Expected deployment time: 25min
lakeFS requires a PostgreSQL database to synchronize actions on your repositories. We will show you how to create a database on AWS RDS, but you can use any PostgreSQL database as long as it’s accessible by your lakeFS installation.
If you already have a database, take note of the connection string and skip to the next step
- Follow the official AWS documentation on how to create a PostgreSQL instance and connect to it. You may use the default PostgreSQL engine, or Aurora PostgreSQL. Make sure you’re using PostgreSQL version >= 11.
Once your RDS is set up and the server is in
Availablestate, take note of the endpoint and port.
- Make sure your security group rules allow you to connect to the database instance.
Save the following configuration file as
--- database: connection_string: "[DATABASE_CONNECTION_STRING]" auth: encrypt: # replace this with a randomly-generated string: secret_key: "[ENCRYPTION_SECRET_KEY]" blockstore: type: s3 s3: region: us-east-1 # optional, fallback in case discover from bucket is not supported
- Download the binary to the EC2 instance.
- Run the
lakefsbinary on the EC2 instance:
lakefs --config config.yaml run
Note: it is preferable to run the binary as a service using systemd or your operating system’s facilities.
To support container-based environments like AWS ECS, lakeFS can be configured using environment variables. Here is a
command to demonstrate starting lakeFS using Docker:
docker run \ --name lakefs \ -p 8000:8000 \ -e LAKEFS_DATABASE_CONNECTION_STRING="[DATABASE_CONNECTION_STRING]" \ -e LAKEFS_AUTH_ENCRYPT_SECRET_KEY="[ENCRYPTION_SECRET_KEY]" \ -e LAKEFS_BLOCKSTORE_TYPE="s3" \ treeverse/lakefs:latest run
See the reference for a complete list of environment variables.
Depending on how you chose to install lakeFS, you should have a load balancer direct requests to the lakeFS server.
By default, lakeFS operates on port 8000, and exposes a
/_health endpoint which you can use for health checks.
- Your security groups should allow the load balancer to access the lakeFS server.
- Create a target group with a listener for port 8000.
- Setup TLS termination using the domain names you wish to use (e.g.
*.s3.lakefs.example.comif using virtual-host addressing).
- Configure the health-check to use the exposed