# SCIM for Self-Managed lakeFS Enterprise

> [!NOTE]
>
> **Available in lakeFS Enterprise. [Contact us](<https://lakefs.io/book-a-demo/>).**

> [!WARNING]
>
> **Private preview**
>
> SCIM for self-managed lakeFS Enterprise is in private preview, and interfaces and behaviour can change before general availability. The first sync takes over matching users and groups that already exist; see [Known limitations](<#known-limitations>) and [Adopting existing users and groups](<#adopting-existing-users-and-groups>) before relying on the feature.

SCIM for self-managed lakeFS Enterprise is a [SCIM 2.0](<https://datatracker.ietf.org/doc/html/rfc7644>) server built into lakeFS Enterprise, which an identity provider (IdP) calls to create, update, deactivate and delete lakeFS users and groups as assignments change in the directory. For lakeFS Cloud, where SCIM is set up from the cloud admin console, see [SCIM for lakeFS Cloud](<https://docs.lakefs.io/security/scim/index.md>).

## Capabilities

The IdP manages users, groups and group memberships. [RBAC](<https://docs.lakefs.io/security/rbac/index.md>) policies, access keys and [tenant](<https://docs.lakefs.io/security/tenants/index.md>) membership stay in lakeFS. A user or group is *provisioned* once the IdP has created or [adopted](<#adopting-existing-users-and-groups>) it. The IdP sees and changes only provisioned users and groups, and every other user or group is hidden from it until it is adopted.

### User provisioning

- **Create users:** Users assigned in the IdP are created in lakeFS with a generated ID, and with the IdP's `userName` as their friendly name.
- **Update user attributes:** Changes to `userName`, `externalId`, the primary email and `active` are synced to lakeFS.
- **Deactivate users:** A deactivated user is refused on the REST API, the web UI, the S3 gateway and the [Iceberg REST catalog](<https://docs.lakefs.io/iceberg/index.md>), and holds no license seat. It keeps its credentials, policies and memberships, so reactivating it restores access. Only the IdP can deactivate or reactivate a user. A deactivation takes effect within the [auth cache TTL](<https://docs.lakefs.io/reference/configuration/#authcache>), about 20 seconds by default, or longer when [`auth.prefetched_cache`](<https://docs.lakefs.io/reference/enterprise-configuration/#authprefetched_cache>) is enabled.
- **Delete users:** A deleted user is removed from lakeFS together with its credentials, policies and memberships, including any access keys an adopted user had before SCIM. lakeFS deletes a user only when the IdP sends a SCIM `DELETE`, so check whether your IdP deactivates or deletes a user that you unassign.
- **User adoption:** Matching users that already exist are [adopted](<#adopting-existing-users-and-groups>) instead of re-created.

### Group provisioning

- **Create groups:** Groups assigned in the IdP are created in lakeFS with their members.
- **Update group name:** When a provisioned group is renamed in the IdP, it is renamed in lakeFS.
- **Add/remove members:** When members are added to or removed from a group in the IdP, they are added to or removed from the group in lakeFS.
- **Delete groups:** A deleted group is removed from lakeFS with its policy attachments and memberships, and its members stay in place. This applies to adopted groups too, as the warning under [Adopting existing users and groups](<#adopting-existing-users-and-groups>) describes.
- **Group adoption:** Matching groups that already exist are [adopted](<#adopting-existing-users-and-groups>) instead of re-created.

### Adopting existing users and groups

When the IdP provisions a user, lakeFS first looks for an existing user that is not provisioned, and adopts it in place of creating a new one:

1. If one existing user holds the request's `externalId`, lakeFS adopts it. For a user created by an SSO login, the external ID is the identifier that login presented.
2. Otherwise, if a provisioned user already holds the `userName`, the request is refused with `409`.
3. Otherwise, if one existing user holds the request's primary email, lakeFS adopts it.
4. Otherwise, lakeFS creates a new user.

```
flowchart TD
    A["The IdP provisions a user"] --> B{"One existing user<br/>holds the externalId?"}
    B -- yes --> ADOPT["Adopt that user"]
    B -- no --> C{"A provisioned user<br/>holds the userName?"}
    C -- yes --> R["Refuse with 409"]
    C -- no --> D{"One existing user holds<br/>the primary email?"}
    D -- yes --> ADOPT
    D -- no --> NEW["Create a new user"]
```

*lakeFS checks the external ID before the email, so a user that both match is adopted by its external ID.*

An adopted user keeps its ID, credentials, policies and group memberships, and takes its name, external ID, email and active state from the request. If several existing users share the external ID or email, the request is refused with `409` until the duplicate is resolved.

A user that has neither an email nor an external ID, such as one created with `lakectl auth users create`, can't be adopted. The IdP creates a new user for that person, and the existing user keeps its credentials and its license seat until you delete it. lakeFS has no way to add an email or external ID to an existing user, so decide before the first sync whether to delete such users. `GET /api/v1/auth/users` returns the `email` and `external_id` of every user, which shows how each one will match.

> [!WARNING]
>
> **Adoption by email replaces the user's external ID**
>
> SSO login finds a user by external ID, and adoption overwrites it with the request's `externalId`. Adoption by email is safe only when the IdP sends the SSO login identifier as `externalId`. See [Linking provisioned users to SSO logins](<#linking-provisioned-users-to-sso-logins>).

Groups are matched by `displayName`. An adopted group keeps its ID, policies and existing members, and the members in the request are added to them. Existing members that are not provisioned stay in the group and are not reported to the IdP.

> [!WARNING]
>
> **Groups are adopted by name**
>
> An IdP group adopts any existing lakeFS group with the same name, in any case, including [preconfigured groups](<https://docs.lakefs.io/security/rbac/#preconfigured-groups>) such as `Admins`. If the IdP later deletes it, lakeFS deletes the group with its policy attachments and all its memberships. Give IdP groups names that differ from existing lakeFS groups unless you want the IdP to manage them, and keep at least one administrator who is not provisioned and whose policies are attached directly.

### Permissions for provisioned users and groups

SCIM does not provision policies, so unless you set the defaults below, a new user or group has no permissions until a policy is attached in lakeFS. Attach policies to the provisioned groups, and let the IdP control access through group membership. A group that SCIM creates has a generated ID, which `lakectl auth groups list` shows next to its name:

```bash
lakectl auth groups list
lakectl auth groups policies attach --id <group-id> --policy FSReadAll
```

Use the same ID when you [attach the group to a tenant](<https://docs.lakefs.io/security/tenants/#creating-tenants-and-managing-membership>). A group that the IdP deletes and pushes again gets a new ID, without the policies and tenant memberships of the old one.

To give every provisioned user a baseline, create a group that holds only the policies every provisioned user needs, and set the optional `auth.providers.scim.default_user_group` to its name. The group must not be provisioned:

```bash
lakectl auth groups create --id scim-users
lakectl auth groups policies attach --id scim-users --policy FSReadAll
```

Every user that SCIM creates joins the group, while an adopted user keeps the memberships it already has. The IdP cannot take the group over, so a group pushed with the same name is refused with `409`. If the group is deleted, user creates fail with `500` and lakeFS refuses to start until the group is created again.

To give every group that SCIM creates a baseline, set the optional `auth.providers.scim.default_group_policy` to the name of an existing policy. lakeFS attaches it to each group SCIM creates, while an adopted group keeps its own policies and gains none. If the policy is deleted, group creates fail with `500` and lakeFS refuses to start until the policy is created again.

## Known limitations

The following are limited or not supported yet in the private preview:

- **Static bearer tokens.** Tokens are set in the server configuration. They do not expire, and changing them requires a restart.
- **Settings without effect.** `auth.providers.scim.revoke_credentials_on_deactivate` and its Helm value `enterprise.auth.scim.revokeCredentialsOnDeactivate` are accepted but have no effect yet.
- **Local changes.** lakeFS does not block local changes to provisioned users and groups, and the IdP is not notified of them.
- **Nested groups.** A group cannot be a member of another group.
- **Email type.** lakeFS keeps one address for each user and returns it as primary, without the `type` or `display` the IdP sent.
- **SCIM protocol.** `/Bulk` and `/.search` are not implemented, and filters support only `eq` terms joined by `and`. The `attributes` and `excludedAttributes` parameters are ignored, apart from `excludedAttributes=members` on groups.

## Enabling SCIM in lakeFS Enterprise

### Prerequisites

- lakeFS Enterprise v1.110.0 or later, which the lakeFS Helm chart deploys from chart version 1.15.2.
- A [lakeFS Enterprise license](<https://docs.lakefs.io/admin/install/enterprise/#lakefs-enterprise-license>) that includes SCIM. The server refuses to start with SCIM enabled without one, and if the license stops covering SCIM while lakeFS runs, SCIM requests are refused with `403`.
- Room in the license for the users and groups the IdP pushes. Each active user takes a seat from the license's user limit, and a provisioned user also takes one from its limit for provisioned users, while a deactivated user takes none. Every group counts toward the group limit. To see the limits, decode the token that [the license API](<https://docs.lakefs.io/admin/install/enterprise/#license-configuration-in-lakefs-enterprise>) returns and look for `number_allowed_users`, `number_allowed_groups` and `number_provisioned_users`.
- [SSO](<https://docs.lakefs.io/security/sso/#sso-for-lakefs-enterprise>) configured with the same IdP, as provisioned users log in through SSO.
- No duplicate users or groups, as provisioning a duplicate is refused with `409`.

Run the task below with the lakeFS configuration, for example in a lakeFS pod. It is safe while lakeFS serves requests. It lists users that share an email or external ID, and groups whose names differ only by case. lakeFS can't edit those fields or rename a group, so resolve each duplicate by deleting one of the records. Then run the task again without `--dry-run`, as lookups keep refusing a resolved duplicate until you do.

```bash
lakefs migrate task run identity-indexes --dry-run
```

### Server configuration

Enable SCIM in the lakeFS configuration with at least one bearer token, which can be any random string without whitespace:

```yaml
auth:
  providers:
    scim:
      enabled: true
      tokens:
        - "<scim-bearer-token>"
      default_user_group: "scim-users"
```

To keep tokens out of the configuration file, set `LAKEFS_AUTH_PROVIDERS_SCIM_TOKENS` to a single token or a JSON list of tokens. See the [configuration reference](<https://docs.lakefs.io/reference/enterprise-configuration/#authprovidersscim>) for every key.

With the lakeFS Helm chart, set the same values under `enterprise.auth.scim`:

```yaml
enterprise:
  enabled: true
  auth:
    scim:
      enabled: true
      tokens:
        - <scim-bearer-token>
      defaultUserGroup: scim-users
```

To keep the tokens in an existing secret instead, store them as a JSON list under a key of the secret, and name that key in `secretKeys.scimTokens`. The chart then ignores `enterprise.auth.scim.tokens`:

```bash
kubectl create secret generic my-lakefs-secrets --from-literal=scim_tokens='["<scim-bearer-token>"]'
```

```yaml
existingSecret: my-lakefs-secrets
secretKeys:
  scimTokens: scim_tokens
```

The chart has no values for `auth.providers.scim.default_group_policy` or [`auth.oidc.external_user_id_claim_name`](<#linking-provisioned-users-to-sso-logins>). Add them to `lakefsConfig`, which the chart passes to lakeFS as its configuration file:

```yaml
lakefsConfig: |
  auth:
    oidc:
      external_user_id_claim_name: oid  # with Microsoft Entra ID
    providers:
      scim:
        default_group_policy: FSReadAll
```

### SCIM base URL

The IdP sends its requests to:

```text
https://<lakefs-host>/api/scim/v2
```

Use the host that serves the lakeFS API and web UI, not a host dedicated to the S3 gateway by [`gateways.s3.domain_name`](<https://docs.lakefs.io/reference/configuration/#gateways>). Serve it over HTTPS, and make sure the IdP can reach every path under `/api/scim/v2/`. To check the endpoint and the token:

```bash
curl -H "Authorization: Bearer <scim-bearer-token>" \
  https://<lakefs-host>/api/scim/v2/ServiceProviderConfig
```

A `200` response with a JSON body means both work, `401` means the token is wrong, and an HTML page means the request did not reach SCIM.

## Setting up SCIM provisioning in your identity provider

Add SCIM provisioning to the lakeFS application in the IdP with:

- **Base URL:** the [SCIM base URL](<#scim-base-url>).
- **Authentication:** one of the configured tokens, as a bearer token.
- **Attribute mappings:** as described under [supported attributes](<#supported-attributes>) and [linking provisioned users to SSO logins](<#linking-provisioned-users-to-sso-logins>).

lakeFS provisions every user and group the IdP sends, so choose which ones to provision in the IdP. Start with a small group to check how existing users are [adopted](<#adopting-existing-users-and-groups>). The IdP settings and attribute rules on the [lakeFS Cloud SCIM page](<https://docs.lakefs.io/security/scim/index.md>) don't apply to self-managed installations.

### Linking provisioned users to SSO logins

At login, lakeFS looks up the user whose external ID equals the identifier the login presents. For OIDC, that identifier is the claim named by [`auth.oidc.external_user_id_claim_name`](<https://docs.lakefs.io/reference/enterprise-configuration/#authoidc>), which is `sub` when unset. For SAML, it is the attribute named by [`auth.cookie_auth_verification.external_user_id_claim_name`](<https://docs.lakefs.io/reference/enterprise-configuration/#authcookie_auth_verification>). Map the same identifier to `externalId` in the IdP's SCIM settings. Otherwise the login does not find the provisioned user. SSO then creates a second user, with the initial groups your SSO configuration sets, or the login fails when the new user's ID is already taken.

Microsoft Entra ID issues a different `sub` for each application, so with Entra and OIDC, set `auth.oidc.external_user_id_claim_name` to `oid`, which carries the user's object ID, and map `objectId` to `externalId` in the IdP's SCIM settings. If your users already log in over OIDC, set the key right after the first full sync. Until then, their logins fail but change nothing. Setting it before the sync makes each login create a second user, which the sync then adopts in place of the original.

Test the link with one user before the first full sync, or right after it if you set the key after the sync. Once the IdP has provisioned that user, their SSO login should not add a user to `lakectl auth users list`.

## Supported attributes

lakeFS keeps the user attributes that its `/Schemas` endpoint marks as `readWrite`:

```bash
curl -s -H "Authorization: Bearer <scim-bearer-token>" \
  https://<lakefs-host>/api/scim/v2/Schemas/urn:ietf:params:scim:schemas:core:2.0:User \
  | jq -r '.attributes[] | select(.mutability == "readWrite") | .name'
```

The other attributes of the core user schema and the enterprise extension are `writeOnly`: lakeFS accepts them and drops them, so the IdP's default mappings for them can stay. Don't map attributes under any other schema, such as a custom extension in Microsoft Entra ID. A PATCH that names one is refused with `400` as a whole, so a deactivation sent in the same PATCH doesn't apply.

### User resource attributes

- `userName`: The name of the user, stored as its friendly name. It must be unique among provisioned users, regardless of case. REQUIRED.
- `externalId`: The identifier of the user in the IdP. lakeFS uses it to adopt existing users and to [link SSO logins](<#linking-provisioned-users-to-sso-logins>).
- `emails`: lakeFS stores the address marked as primary, or the first address when none is marked, and uses it to adopt existing users.
- `active`: Whether the user can access lakeFS. It defaults to `true`.

### Group resource attributes

- `displayName`: The name of the group. It must be unique in the installation, regardless of case. REQUIRED.
- `externalId`: The identifier of the group in the IdP.
- `members`: The members of the group. Each member must be a provisioned user. A request that adds any other member is refused with `400` and changes none of the members, so a user who failed to provision holds back every group that includes them.

## Operating SCIM provisioning

### Rotating SCIM tokens

lakeFS reads the tokens at startup and accepts any of them, so a token can be rotated without interrupting provisioning:

1. Add the new token next to the current one, and restart lakeFS.
2. Replace the token in the IdP.
3. Remove the old token, and restart lakeFS.

With the Helm chart, a `helm upgrade` that changes the tokens the chart holds rolls out new lakeFS pods, which picks up the new tokens. Tokens in an `existingSecret` are not visible to the chart, so restart the lakeFS pods after you change the secret.

### Monitoring SCIM requests

The `scim_requests_total` [Prometheus metric](<https://docs.lakefs.io/admin/monitoring/index.md>) counts every SCIM request by `method` and response `code`, including requests refused for a bad token. Requests are also logged with `service_name` set to `scim`, at the level set by [`logging.audit_log_level`](<https://docs.lakefs.io/reference/configuration/#logging>). That level defaults to `DEBUG`, below the default `logging.level` of `INFO`, so SCIM requests are hidden. Setting it to `INFO` shows them, along with every lakeFS API and S3 gateway request.

### Turning SCIM off

Setting `auth.providers.scim.enabled` to `false` stops serving the SCIM endpoint and leaves provisioned users and groups as they are. A user the IdP deactivated stays deactivated, and only the IdP can reactivate it, so reactivate those users in the IdP first, or delete them. External IDs that adoption replaced are not restored.

### Troubleshooting provisioning errors

| Response | Cause |
| --- | --- |
| `400` | A request adds a group member who is not a provisioned user, a PATCH names an attribute outside the schemas at `/Schemas`, such as a custom extension attribute, or the filter is not supported. |
| `401` | The token is not one of the configured tokens. |
| `403` | The license does not include SCIM, or a user or group limit of the license is reached, which can also refuse a reactivation. |
| `409` | Another provisioned user or group holds the name, external ID or email, or several existing records match. Resolve the duplicate, then run `lakefs migrate task run identity-indexes`. A group with the name of the default user group is also refused. |
| `500` | lakeFS cannot find the default user group or the default group policy, or the request failed on the server. The lakeFS log records the cause. |
| `501` | The endpoint is not implemented. |
| An HTML page | The request did not reach the SCIM endpoint. Check that SCIM is enabled and that the base URL is correct. |