lakeFS Enterprise Configuration Reference¶

lakeFS Enterprise configuration extends lakeFS's configuration and uses the same config file.

lakeFS Configuration¶

For a complete list of configuration options, see the lakeFS Server Configuration. The sections below provide additional configuration references that complement the main configuration guide.

Reference¶

This reference uses . to denote the nesting of values.

Client IP resolution¶

lakeFS resolves the client IP of each request once, and every feature that reads a client address uses that result. A client can place any value in a forwarding header such as X-Forwarded-For, so ip_strategy decides which headers, if any, are believed.

• ip_strategy (one of ["direct", "proxy", "platform"] : "proxy") - Where the client address of a request is taken from.

  directproxyplatform

  The address of the TCP connection, with no header consulted. Use this when nothing sits between clients and lakeFS, and for local development.

  X-Forwarded-For, then X-Real-IP, honored only when the connection comes from an address in trusted_proxies. Use this behind a load balancer, a reverse proxy, or a Kubernetes ingress, and set trusted_proxies to the addresses that proxy uses. An empty list honors no header, so resolution behaves as under direct apart from counting requests that carried one, as described under Monitoring client IP resolution.

  The header named by trusted_platform_header, read from any connection and taking precedence over the proxy strategy, which handles requests that arrive without it. Use this behind a CDN that reports the client address in its own header, and only where lakeFS cannot be reached around the CDN, since a client that reaches the server directly could supply the header itself.

• trusted_proxies (list of strings : ["0.0.0.0/0", "::/0"]) - CIDR blocks or single addresses of the proxies in front of lakeFS, read by the proxy strategy.

  Warning

  The default trusts every network, which is not safe: any client that can reach lakeFS can choose the address that policies evaluate and audit logs record by sending an X-Forwarded-For header of its own. Set trusted_proxies to the addresses your proxies actually use, or to an empty list to ignore forwarding headers altogether. Trusting every network is appropriate only when a proxy that replaces X-Forwarded-For with the address it observed is the sole path to lakeFS.

• trusted_platform_header (string : "") - Name of the provider header the platform strategy reads, commonly CF-Connecting-IP, True-Client-IP, or X-Cloudfront-Viewer-Address. Required under the platform strategy.

• client_ip_debug_header (string : "") - Name of a response header that carries the address lakeFS resolved for the request, such as X-Lakefs-Client-Ip. It is returned on authenticated requests only, so an anonymous caller cannot use it to learn whether a forged forwarding header was believed. Empty adds no header.

Each key has a matching environment variable, so ip_strategy can be set through LAKEFS_IP_STRATEGY, and trusted_proxies accepts a comma-separated list through LAKEFS_TRUSTED_PROXIES:

ip_strategy: "proxy"
trusted_proxies:
  - "10.0.1.0/24"       # the ingress subnet
  - "192.168.1.50"      # a single reverse proxy

Under the proxy strategy lakeFS reads X-Forwarded-For from its rightmost entry, the one added by the proxy nearest the server, towards the left, skipping trusted proxy addresses, and uses the first address that is not one. Entries a client wrote sit further left and are never reached, and when every entry is a trusted proxy the leftmost is used. Every host in a trusted range can state the client address, so list the addresses your proxy actually uses rather than a whole subnet, and in Kubernetes, where pods and the ingress often share subnets, restrict access to the lakeFS service with a NetworkPolicy so that only the ingress can reach it.

Monitoring client IP resolution¶

Each server logs where it takes the client address from as it starts, naming the strategy and the trusted networks. To see what a particular caller resolves to, set client_ip_debug_header and read the address back from the response to any authenticated request.

The lakefs_client_ip_forwarding_anomalies_total counter, labelled by reason, counts requests whose X-Forwarded-For or X-Real-IP could not be used as configured. Once trusted_proxies names specific networks, untrusted_peer is the one that identifies a misconfiguration: it rises when requests carry a forwarding header over a connection outside the list, which almost always means a proxy in front of lakeFS is missing from it. Under the default, which trusts every network, no connection is untrusted and the reason never occurs.

database¶

Configuration section for lakeFS Enterprise database options.

database.redis¶

• database.redis.endpoint (string : "localhost:6379") - Redis server endpoint in host:port format
• database.redis.enable_tls (bool : false) - Enable TLS for Redis connections (required for AWS MemoryDB)
• database.redis.cluster_mode (bool : false) - Enable Redis Cluster mode support
• database.redis.username (string : "") - Username for Redis authentication
• database.redis.password (string : "") - Password for Redis authentication
• database.redis.pool_size (int : 10) - Connection pool size for Redis connections
• database.redis.min_idle_conns (int : 0) - Minimum number of idle connections in the pool
• database.redis.dial_timeout (duration : 5s) - Timeout for establishing new connections
• database.redis.read_timeout (duration : 3s) - Timeout for socket reads
• database.redis.write_timeout (duration : 3s) - Timeout for socket writes
• database.redis.database (int : 0) - Database number to select (0-15 for most Redis configurations)
• database.redis.namespace (string : "") - Prefix for all keys used by the application
• database.redis.tls_skip_verify (bool : false) - Skip certificate verification (for development only)
• database.redis.batch_size (int : 1000) - Default batch size for Redis KV operations

auth¶

Configuration section for SSO authentication services, like SAML or OIDC.

• auth.logout_redirect_url (string : "/auth/login") - The URL to redirect to after logout when using SSO authentication services, like SAML or OIDC.
  The configuration depends on the authentication provider:
  • For OIDC: The logout URL of the OIDC provider (e.g., Auth0 logout endpoint).
  • For SAML: The URL within lakeFS where the IdP should redirect after logout (e.g., /auth/login).
• auth.login_max_duration (time duration : "336h") - The maximum duration a client may request for a login token issued through the external principals API.

• auth.allowed_authentication_methods (list of strings : []) - Restricts which authentication methods clients may use. When the list is empty (the default), all methods are allowed. Supply one or more methods to permit only those; requests using any other method are rejected.
  Supported values:

  • "access_key" - Access key ID and secret access key. Covers the S3 gateway, API basic authentication, and the built-in username/password login.
  • "sso" - Single sign-on via OIDC or SAML.
  • "external_principal" - External principal login (e.g. AWS IAM).

  For example, to disable access keys and allow only SSO and external principals:

  auth:
    allowed_authentication_methods:
      - sso
      - external_principal
  

  The web UI adjusts to the restriction: when no credential-based login can succeed, the login form is hidden and only the allowed methods are offered. Separately configured LDAP or remote authentication is not covered by this setting and keeps working, including its login form.

  Note

  This setting is supported only with the local RBAC auth service (features.local_rbac, the default for self-managed installations). It is rejected at startup when using the external auth API service or basic auth, since neither honors the restriction.

auth.authentication_api¶

• auth.authentication_api.endpoint (string : "") - URL to external Authentication Service described at authentication.yml.
• auth.authentication_api.external_principals_enabled (bool : false) - If true, external principals API will be enabled, e.g auth service and login api's.

auth.remote_authenticator¶

• auth.remote_authenticator.enabled (bool : false) - If specified, also authenticate users via this Remote Authenticator server.
• auth.remote_authenticator.endpoint (string : "" - required) - Endpoint URL of the remote authentication service (e.g. https://my-auth.example.com/auth).
• auth.remote_authenticator.default_user_group (string : "Viewers") - Create users in this group (i.e Viewers, Developers, etc).
• auth.remote_authenticator.request_timeout (time duration : 10s) - If specified, timeout for remote authentication requests.

auth.oidc¶

• auth.oidc.default_initial_groups (string[] : []) - By default, OIDC users will be assigned to these groups.
• auth.oidc.initial_groups_claim_name (string : "") - Use this claim from the ID token to provide the initial group for new users. This will take priority if auth.oidc.default_initial_groups is also set.
• auth.oidc.friendly_name_claim_name (string : "") - If specified, the value from the claim with this name will be used as the user's display name.
• auth.oidc.email_claim_name (string : "") - If specified, the value from the claim with this name will be used as the user's email when the user is first created.
• auth.oidc.persist_friendly_name (bool : false) - If set to true, the friendly name is persisted to the KV store and can be displayed in the user list. This is meant to be used in conjunction with auth.oidc.friendly_name_claim_name.
• auth.oidc.validate_id_token_claims (map[string]string : {}) - When a user tries to access lakeFS, validate that the ID token contains these claims with the corresponding values.

auth.cookie_auth_verification¶

• auth.cookie_auth_verification.validate_id_token_claims (map[string]string : {}) - When a user tries to access lakeFS, validate that the ID token contains these claims with the corresponding values.
• auth.cookie_auth_verification.default_initial_groups (string[] : []) - By default, users will be assigned to these groups.
• auth.cookie_auth_verification.initial_groups_claim_name (string : "") - Use this claim from the ID token to provide the initial group for new users. This will take priority if auth.cookie_auth_verification.default_initial_groups is also set.
• auth.cookie_auth_verification.friendly_name_claim_name (string : "") - If specified, the value from the claim with this name will be used as the user's display name.
• auth.cookie_auth_verification.email_claim_name (string : "") - If specified, the value from the claim with this name will be used as the user's email when the user is first created.
• auth.cookie_auth_verification.persist_friendly_name (bool : false) - If set to true, the friendly name is persisted to the KV store and can be displayed in the user list. This is meant to be used in conjunction with auth.cookie_auth_verification.friendly_name_claim_name.
• auth.cookie_auth_verification.external_user_id_claim_name - (string : "") - If specified, the value from the claim with this name will be used as the user's id name.
• auth.cookie_auth_verification.auth_source - (string : "") - If specified, user will be labeled with this auth source.

auth.ui_config¶

• auth.ui_config.login_url (string : "") - An absolute or relative URL to your IdP’s login page, used to authenticate to lakeFS via SSO with OIDC or SAML.
• auth.ui_config.logout_url (string : "") - URL to redirect users to when they logout from lakeFS (defaults to "/logout" if not specified).
• auth.ui_config.use_login_placeholders (bool: false) - If set to true, the login page will show placeholders for the Access Key ID and Secret Access Key (Username and Password) (defaults to "false" if not specified).
• auth.ui_config.login_url_method (string : "redirect") - Controls how lakeFS handles login when an auth.ui_config.login_url (SSO via OIDC or SAML) is configured. This parameter is only relevant when auth.ui_config.login_url is set.
  Supported values:
  • "redirect" - On login, users are redirected to the configured auth.ui_config.login_url (default).
  • "select" - On login, login selection page is used to select between build-in lakeFS and SSO login.

auth.providers¶

Configuration section for external identity providers used for authentication services, such as LDAP, SAML or OIDC.

auth.providers.ldap¶

Configuration section for LDAP

• auth.providers.ldap.server_endpoint (string : "" - required) - The LDAP server address, e.g. 'ldaps://ldap.company.com:636'.
• auth.providers.ldap.bind_dn (string : "" - required) - The bind string, e.g. 'uid=<bind-user-name>,ou=Users,o=<org-id>,dc=<company>,dc=com'.
• auth.providers.ldap.bind_password (string : "" - required) - The password for the user to bind.
• auth.providers.ldap.username_attribute (string : "" - required) - The user name attribute, e.g. 'uid'.
• auth.providers.ldap.user_base_dn (string : "" - required) - The search request base dn, e.g. 'ou=Users,o=<org-id>,dc=<company>,dc=com'.
• auth.providers.ldap.user_filter (string : "" - required) - The search request user filter, e.g. '(objectClass=inetOrgPerson)'.
• auth.providers.ldap.connection_timeout_seconds (int : 0 - required) - The timeout for a single connection.
• auth.providers.ldap.request_timeout_seconds (int : 0 - required) - The timeout for a single request.
• auth.providers.ldap.default_user_group (string : "" - required) - The default group for the users initially authenticated by the remote service.

auth.providers.saml¶

Configuration section for SAML

• auth.providers.saml.sp_root_url (string : "") - The base lakeFS-URL, e.g. 'https://<lakefs-url>'.
• auth.providers.saml.sp_x509_key_path (string : "") - The path to the private key, e.g '/etc/saml_certs/rsa_saml_private.cert'.
• auth.providers.saml.sp_x509_cert_path (string : "") - The path to the public key, '/etc/saml_certs/rsa_saml_public.pem'.
• auth.providers.saml.sp_sign_request (bool : false) Some IdP require the SLO request to be signed.
• auth.providers.saml.sp_signature_method (string : "") Optional valid signature values depending on the IdP configuration, e.g. 'http://www.w3.org/2001/04/xmldsig-more#rsa-sha256'.
• auth.providers.saml.idp_metadata_url (string : "") - The URL for the metadata server, e.g. 'https://<adfs-auth.company.com>/federationmetadata/2007-06/federationmetadata.xml'.
• auth.providers.saml.idp_metadata_file_path (string : "") - The path to the Identity Provider (IdP) metadata XML file, e.g. '/etc/saml/idp-metadata.xml'.
• auth.providers.saml.idp_skip_verify_tls_cert (bool : false) - Insecure skip verification of the IdP TLS certificate, like when signed by a private CA.
• auth.providers.saml.idp_authn_name_id_format (string : "") - The format used in the NameIDPolicy for authentication requests. (e.g., "urn:oasis:namesSAML:1.1:nameid-format:unspecified")
• auth.providers.saml.idp_request_timeout (time duration : 10s) The timeout for remote authentication requests.
• auth.providers.saml.post_login_redirect_url (string : "") - The URL to redirect users to after successful SAML authentication, e.g. 'http://localhost:8000/'.

auth.providers.oidc¶

Configuration section for OIDC

• auth.providers.oidc.url (string : "") - The OIDC provider url, e.g. 'https://oidc-provider-url.com/'
• auth.providers.oidc.client_id (string : "") - The application's ID
• auth.providers.oidc.client_secret (string : "") - The application's secret
• auth.providers.oidc.callback_base_url (string : "") - A default callback address of the lakeFS server
• auth.providers.oidc.callback_base_urls (string[] : []) - If callback_base_urls is configured, check current host is whitelisted otherwise use callback_base_url (without 's'). These config keys are mutually exclusive

• auth.providers.oidc.authorize_endpoint_query_parameters (map[string]string : {}) - key/value parameters that are passed to a provider's authorization endpoint
• auth.providers.oidc.logout_endpoint_query_parameters (string[] : []) - The query parameters that will be used to redirect the user to the OIDC provider after logout, e.g. ["returnTo", "https://<lakefs.ingress.domain>/oidc/login"]
• auth.providers.oidc.logout_client_id_query_parameter (string : "") - The claim name that represents the client identifier in the OIDC provider
• auth.providers.oidc.additional_scope_claims (string[] : []) - Specifies optional requested permissions, other than openid and profile that are being used
• auth.providers.oidc.post_login_redirect_url (string : "") - The URL to redirect users to after successful OIDC authentication, e.g. 'http://localhost:8000/'

auth.external_aws_auth¶

Configuration section for authentication to lakeFS using the AWS presigned get-caller-identity request:
External Principals AWS Auth

• auth.external_aws_auth.enabled (bool : false) - If true, external principals API will be enabled, e.g auth service and login api's
• auth.external_aws_auth.get_caller_identity_max_age (time duration : 24h) - The maximum age of a presigned GetCallerIdentity request that the server accepts. lakeFS clients presign requests that expire after 15 minutes, and setting a value close to that instead of the default limits how long a leaked presigned request stays usable
• auth.external_aws_auth.valid_sts_hosts (string[] : []) - The default are all the valid AWS STS hosts (sts.amazonaws.com, sts.us-east-2.amazonaws.com etc.)
• auth.external_aws_auth.required_headers (map[string]string : {}) - Headers that must be present by the client when doing login request. For security reasons it is recommended to set X-LakeFS-Server-ID: <lakefs.ingress.domain>, lakeFS clients assume that's the default
• auth.external_aws_auth.optional_headers (map[string]string : {}) - Optional headers that can be present by the client when doing login request
• auth.external_aws_auth.http_client.timeout (time duration : 10s) - The timeout for the HTTP client used to communicate with AWS STS
• auth.external_aws_auth.http_client.skip_verify (bool : false) - Skip SSL verification with AWS STS

auth.prefetched_cache¶

Configuration section for the prefetched cache. The prefetched cache is used to cache entries from the auth KV store in memory to reduce the number of calls to the KV store.

• auth.prefetched_cache.enabled (bool : false) - Enable the prefetched cache
• auth.prefetched_cache.interval (duration : 30s) - The interval at which to refresh the cache
• auth.prefetched_cache.ttl (duration : 1m) - The time to live for each cache entry
• auth.prefetched_cache.size (int : 10000) - The maximum number of entries in the cache
• auth.prefetched_cache.eviction_jitter (duration : 3s) - The maximum random jitter to add to the eviction interval to prevent thundering herds
• auth.prefetched_cache.negative_cache_size (int : 40000) - The maximum number of missing entries the cache will remember

backpressure¶

Configuration section for backpressure settings. When enabled, backpressure will limit the number of requests each server will handle concurrently through s3 gateway

• backpressure.enabled (bool : false) - Enable backpressure
• backpressure.upload_part_max_concurrent (int : 0) - The maximum number of concurrent upload part requests. 0 means no limit

blockstores¶

• blockstores.signing.secret_key (string : required) - A random generated string that is used for HMAC signing when using get/link physical address
• blockstores.stores ([{id: string, type: string, ...}] : required) - Defines multiple storage backends used in a multi-storage backend setup. Each storage backend must have a unique id and a valid configuration.

Common Fields for All Stores¶

• blockstores.stores[].id (string : required) - Unique identifier for the storage backend.
• blockstores.stores[].backward_compatible (bool : false) - Optional. Defaults to false. Used to migrate from a single to a multi-storage backend setup.
• blockstores.stores[].description (string : ) - A human-readable description of the storage backend.
• blockstores.stores[].type (string : required) - (one of ["local", "s3", "gs", "azure", "mem"] : required). Block adapter to use. This controls where the underlying data will be stored.

blockstore.s3.credentials_vending¶

Configuration for vending short-lived, table-scoped S3 credentials to Iceberg REST Catalog clients. See Iceberg credentials vending. For multi-storage backend setups, use the equivalent blockstores.stores[].s3.credentials_vending.* fields.

• blockstore.s3.credentials_vending.role_arn (string : ) - ARN of the IAM role lakeFS assumes (via STS AssumeRole) when vending credentials. Setting a non-empty value enables credential vending; an empty value (the default) disables it.
• blockstore.s3.credentials_vending.session_duration (time duration : "1h") - Lifetime of the vended STS session. Minimum 15m; configuring a shorter value prevents lakeFS from starting.
• blockstore.s3.credentials_vending.external_id (string : ) - External ID passed to STS AssumeRole, normally when the role's trust policy requires one.
• blockstore.s3.credentials_vending.endpoint (string : ) - Overrides the S3 endpoint returned to clients in the vended configuration. Useful when lakeFS and its clients reach S3 through different (internal/external) addresses.

features¶

• features.local_rbac (bool: true) - Backward compatibility if you use an external RBAC service (such as legacy fluffy). If false lakeFS will expect to use auth.api and all fluffy related configuration for RBAC.

graveler.compaction¶

Configuration section for staging compaction.

• graveler.compaction.auto_trigger (bool : false) - Compact a branch automatically once its staged mutations cross the threshold.
• graveler.compaction.threshold (int : 0) - Staged mutations per branch before a compaction starts. Zero selects the default of 50,000.

audit_log¶

Configuration section for the Audit Log Iceberg System Table.

• audit_log.enabled (bool : false) - Enable the audit log Iceberg pipeline. When enabled, lakeFS captures all audit events and persists them to an Iceberg table.
• audit_log.retention_days (int : 90) - Default retention period in days for snapshot expiration. Set to 0 for infinite retention.
• audit_log.storage_namespace (string : "") - Storage location for the lakefssystem repository. If empty, derived from blockstore.default_namespace_prefix by appending /lakefssystem (e.g. s3://my-bucket/lakefssystem).
• audit_log.storage_id (string : "") - Storage backend ID for multi-storage backend setups. Must correspond to the backend that hosts the storage_namespace above.

audit_log.flush¶

• audit_log.flush.interval (duration : 1m) - How often to flush buffered audit events.
• audit_log.flush.batch_size (int : 100000) - Flush when this many events accumulate. Events are flushed when either the interval or the batch size threshold is reached, whichever comes first.

audit_log.maintenance¶

Configuration for audit maintenance. See Maintenance for details. When running maintenance as an external job, set enabled to false to disable the in-process scheduler.

• audit_log.maintenance.enabled (bool : true) - Run maintenance in-process when the audit_log.enabled setting is true. This setting has no effect when audit logging is disabled.
• audit_log.maintenance.schedule (string : "0 * * * *") - Cron expression to schedule maintenance, defaulting to every hour.
• audit_log.maintenance.metrics_push_url (string : "") - Prometheus Pushgateway URL to push maintenance metrics to after each run (used by the external job mode). If empty, metrics are not pushed.

iceberg_catalog¶

Configuration section for the Iceberg REST Catalog

• iceberg_catalog.token_duration (duration : 1h) - Authenticated token duration

ui¶

Configuration section for custom object viewers. How objects select a viewer is described in Matching rules.

• ui.custom_viewers ([{name: string, url: string, ...}] : []) - Viewer entries. Give each viewer at least one content type or extension. Viewer names, content types, and extensions must be unique across viewer entries.
• ui.custom_viewers[].name (string : required) - Unique name identifying the viewer.
• ui.custom_viewers[].url (string : required) - URL loaded in the viewer iframe. See Serve the page.
• ui.custom_viewers[].content_types (string[] : []) - Exact values matched against the object's full content type.
• ui.custom_viewers[].extensions (string[] : []) - Lowercase final file suffixes, without the leading dot.
• ui.custom_viewers[].allow_same_origin (bool : false) - Preserves the viewer's origin instead of assigning an opaque one. See Relaxing the sandbox with allow_same_origin.

Using Environment Variables¶

All the configuration variables can be set or overridden using environment variables. To set an environment variable, prepend LAKEFS_ to its name, convert it to upper case, and replace . with _:

For example, auth.logout_redirect_url becomes LAKEFS_AUTH_LOGOUT_REDIRECT_URL, auth.external.aws_auth.enabled becomes LAKEFS_AUTH_EXTERNAL_AWS_AUTH_ENABLED, etc.

To set a value for a map[string]string type field, use the syntax key1=value1,key2=value2,....