Quickstart¶

Follow these quickstarts to try out lakeFS Enterprise.

Warning

lakeFS Enterprise Quickstarts are not suitable for production use-cases. See the installation guide to set up a production-grade lakeFS Enterprise installation

lakeFS Enterprise Sample¶

The lakeFS Enterprise Sample is the quickest way to experience the value of lakeFS Enterprise features in a containerized environment. This Docker-based setup is ideal if you want to easily interact with lakeFS without the hassle of integration and experiment with lakeFS without writing code.

By running the lakeFS Enterprise Sample, you will be getting a ready-to-use environment including the following containers:

lakeFS Enterprise (includes additional features)
Postgres: used by lakeFS as a KV store
MinIO container: used as the storage connected to lakeFS
Jupyter notebooks setup: Pre-populated with notebooks that demonstrate lakeFS Enterprise' capabilities
Apache Spark: this is useful for interacting with data you'll manage with lakeFS

Checkout the RBAC demo notebook to see lakeFS Enterprise Role-Based Access Control capabilities in action.

Docker Quickstart¶

Prerequisites¶

Note

In order to use lakeFS enterprise you must have: - Access token to download binaries from Docker hub - License to run lakeFS Enterprise Contact us to gain access for both.

You have installed Docker Compose version 2.23.1 or higher on your machine.
Access to download treeverse/lakefs-enterprise from Docker Hub.
With the token you've been granted, login locally to Docker Hub with docker login -u externallakefs -p <TOKEN>.

The quickstart docker-compose files below create a lakeFS server that's connected to a local blockstore and spin up the following containers:

lakeFS Enterprise
Postgres: used by lakeFS as a KV store

You can choose from the following options:

Recommended: A fully functional lakeFS Enterprise setup without SSO support
Advanced: A fully functional lakeFS Enterprise setup including SSO support with OIDC integration configured

Info

If you can postpone the evaluation of the SSO integration, we suggest starting without it to speed up overall testing. The SSO integration requires additional configurations and is best addressed later.

Recommended (SSO Disabled)Advanced (SSO Enabled)

Create a docker-compose.yaml file with the following content
Run docker compose up in the same directory as the docker-compose.yaml file.
In your browser, go to http://localhost:8080 to access lakeFS UI.

version: "3"
services:
  lakefs:
    image: "treeverse/lakefs-enterprise:latest"
    command: "RUN"
    ports:
      - "8000:8000"
    depends_on:
      - "postgres"
    environment:
      - LAKEFS_LISTEN_ADDRESS=0.0.0.0:8000
      - LAKEFS_LOGGING_LEVEL=DEBUG
      - LAKEFS_AUTH_ENCRYPT_SECRET_KEY=random_secret
      - LAKEFS_AUTH_UI_CONFIG_RBAC=internal
      - LAKEFS_DATABASE_TYPE=postgres
      - LAKEFS_DATABASE_POSTGRES_CONNECTION_STRING=postgres://lakefs:lakefs@postgres:5432/postgres?sslmode=disable
      - LAKEFS_BLOCKSTORE_TYPE=local
      - LAKEFS_BLOCKSTORE_LOCAL_PATH=/home/lakefs
      - LAKEFS_BLOCKSTORE_LOCAL_IMPORT_ENABLED=true
      - LAKEFS_AUTH_POST_LOGIN_REDIRECT_URL=http://localhost:8000/
      - LAKEFS_FEATURES_LOCAL_RBAC=true
      - LAKEFS_LICENSE_CONTENTS=<license token>
    configs:
      - source: lakefs.yaml
        target: /etc/lakefs/config.yaml
  postgres:
    image: "postgres:11"
    ports:
      - "5433:5432"
    environment:
      POSTGRES_USER: lakefs
      POSTGRES_PASSWORD: lakefs

configs:
  lakefs.yaml:
    content: |
      auth:
        ui_config:
          login_cookie_names:
            - internal_auth_session

This setup uses OIDC as the SSO authentication method, thus requiring a valid OIDC configuration.

Create a docker-compose.yaml with the content below.
Create a .env file with the configurations below in the same directory as the docker-compose.yaml, docker compose will automatically use that.
Run docker compose up in the same directory as the docker-compose.yaml file.
Validate the OIDC configuration:
In your browser, go to http://localhost:8080 to access lakeFS UI
Complete the Setup process, and login with your Admin credentials
Logout and try to login again, you will be redirected to the OIDC login page.

.env

LAKEFS_AUTH_PROVIDERS_OIDC_CLIENT_ID=<your-oidc-client-id>
LAKEFS_AUTH_PROVIDERS_OIDC_CLIENT_SECRET=<your-oidc-client-secret>
# The name of the query parameter that is used to pass the client ID to the logout endpoint of the SSO provider, i.e client_id
LAKEFS_AUTH_PROVIDERS_OIDC_LOGOUT_CLIENT_ID_QUERY_PARAMETER=
LAKEFS_AUTH_PROVIDERS_OIDC_URL=https://my-sso.com/
LAKEFS_AUTH_LOGOUT_REDIRECT_URL=https://my-sso.com/logout
# Optional: display a friendly name in the lakeFS UI by specifying which claim from the provider to show (i.e name, nickname, email etc)
LAKEFS_AUTH_OIDC_FRIENDLY_NAME_CLAIM_NAME=
LAKEFS_LICENSE_CONTENTS=<license token>

docker-compose.yaml

version: "3"
services:
  lakefs:
    image: "treeverse/lakefs-enterprise:latest"
    command: "RUN"
    ports:
      - "8080:8080"
    depends_on:
      - "postgres"
    environment:
      - LAKEFS_LISTEN_ADDRESS=0.0.0.0:8000
      - LAKEFS_LOGGING_LEVEL=DEBUG
      - LAKEFS_LOGGING_AUDIT_LOG_LEVEL=INFO
      - LAKEFS_AUTH_ENCRYPT_SECRET_KEY=shared-secret-key
      - LAKEFS_AUTH_LOGOUT_REDIRECT_URL=${LAKEFS_AUTH_LOGOUT_REDIRECT_URL}
      - LAKEFS_AUTH_UI_CONFIG_LOGIN_URL=http://localhost:8000/oidc/login
      - LAKEFS_AUTH_UI_CONFIG_LOGOUT_URL=http://localhost:8000/oidc/logout
      - LAKEFS_AUTH_UI_CONFIG_RBAC=internal
      - LAKEFS_AUTH_OIDC_FRIENDLY_NAME_CLAIM_NAME=${LAKEFS_AUTH_OIDC_FRIENDLY_NAME_CLAIM_NAME}
      - LAKEFS_AUTH_PROVIDERS_OIDC_ENABLED=true
      - LAKEFS_AUTH_PROVIDERS_OIDC_POST_LOGIN_REDIRECT_URL=http://localhost:8000/
      - LAKEFS_AUTH_PROVIDERS_OIDC_URL=${LAKEFS_AUTH_PROVIDERS_OIDC_URL}
      - LAKEFS_AUTH_PROVIDERS_OIDC_CLIENT_ID=${LAKEFS_AUTH_PROVIDERS_OIDC_CLIENT_ID}
      - LAKEFS_AUTH_PROVIDERS_OIDC_CLIENT_SECRET=${LAKEFS_AUTH_PROVIDERS_OIDC_CLIENT_SECRET}
      - LAKEFS_AUTH_PROVIDERS_OIDC_CALLBACK_BASE_URL=http://localhost:8000
      - LAKEFS_AUTH_PROVIDERS_OIDC_LOGOUT_CLIENT_ID_QUERY_PARAMETER=${LAKEFS_AUTH_OIDC_LOGOUT_CLIENT_ID_QUERY_PARAMETER}
      - LAKEFS_ENTERPRISE_LICENSE_SERVER_URL=https://license.lakefs.io
      - LAKEFS_LICENSE_CONTENTS=${LAKEFS_LICENSE_CONTENTS}
      - LAKEFS_AUTH_PROVIDERS_OIDC_LOGOUT_CLIENT_ID_QUERY_PARAMETER=${LAKEFS_AUTH_PROVIDERS_OIDC_LOGOUT_CLIENT_ID_QUERY_PARAMETER}
      - LAKEFS_DATABASE_TYPE=postgres
      - LAKEFS_DATABASE_POSTGRES_CONNECTION_STRING=postgres://lakefs:lakefs@postgres:5432/postgres?sslmode=disable
      - LAKEFS_BLOCKSTORE_TYPE=local
      - LAKEFS_BLOCKSTORE_LOCAL_PATH=/tmp/lakefs/data
      - LAKEFS_BLOCKSTORE_LOCAL_IMPORT_ENABLED=true
      - LAKEFS_FEATURES_LOCAL_RBAC=true
    entrypoint: ["/app/wait-for", "postgres:5432", "--", "/app/lakefs", "run"]
    configs:
      - source: lakefs.yaml
        target: /etc/lakefs/config.yaml
  postgres:
    image: "postgres:11"
    ports:
      - "5433:5432"
    environment:
      POSTGRES_USER: lakefs
      POSTGRES_PASSWORD: lakefs

#This tweak is unfortunate but also necessary. logout_endpoint_query_parameters is a list
#of strings which isn't parsed nicely as env vars.
configs:
  lakefs.yaml:
    content: |
      auth:
        ui_config:
          login_cookie_names:
            - internal_auth_session
            - oidc_auth_session
        oidc:
          # friendly_name_claim_name: "name"
          default_initial_groups:
            - Admins
        providers:
          oidc:
            logout_endpoint_query_parameters:
              - returnTo
              - http://localhost:8000/oidc/login

Kubernetes Helm Chart Quickstart¶

In order to use lakeFS Enterprise, we provided out of the box setup, see lakeFS Helm chart configuration.

The values below create a fully functional lakeFS Enterprise setup without SSO support. The created setup is connected to a local blockstore, and spins up the following pods:

lakeFS Enterprise
Postgres: used by lakeFS as a KV store

Info

If you can postpone the evaluation of the SSO integration, we suggest starting without it to speed up overall testing. The SSO integration requires additional configurations and is best addressed later. To try lakeFS Enterprise SSO capability on a Kubernetes cluster, check out the production deployment guide.

Prerequisites¶

You have a Kubernetes cluster running in one of the platforms supported by lakeFS.
Helm is installed
Access to download treeverse/lakefs-enterprise from Docker Hub.
lakeFS Enterprise license Contact us to gain access to lakeFS Enterprise.

Instructions¶

Add the lakeFS Helm repository with helm repo add lakefs https://charts.lakefs.io
Create a values.yaml file with the following content and make sure to replace <lakefs-enterprise-docker-registry-token> with the Docker Hub token you received, <lakefs.acme.com> and <ingress-class-name>.
In the desired K8S namespace run helm install lakefs lakefs/lakefs -f values.yaml
In your browser, go to the Ingress host to access lakeFS UI.

enterprise:
  enabled: true

image:
  privateRegistry:
    enabled: true
    secretToken: <lakefs-enterprise-docker-registry-token>

lakefsConfig: |
  logging:
    level: "DEBUG"
  blockstore:
    type: local
  auth:
    ui_config:
      rbac: internal

ingress:
  enabled: true
  ingressClassName: <ingress-class-name>
  annotations: {}
  hosts:
    - host: <lakefs.acme.com>
      paths:
       - /

# useDevPostgres is false by default and will override any other db configuration, 
# set false or remove for configuring your own db
useDevPostgres: true