# Authenticate to lakeFS with AWS IAM Roles

> [!NOTE]
>
> **Available in lakeFS Team and lakeFS Enterprise on AWS. [Start a free trial](<https://lakefs.io/try/>) or [contact us](<https://lakefs.io/book-a-demo/>).**

## Overview

lakeFS supports authenticating users programmatically using AWS IAM roles instead of using static lakeFS access and secret keys. The method enables you to bound IAM principal ARNs to lakeFS users. A single lakeFS user may have many AWS principal ARNs attached to it. When a client is authenticated to a lakeFS server with an AWS session, the actions performed by the client are on behalf of the user attached to the ARN.

### Using Session Names

The bound ARN can be attached to a single lakeFS user with or without SessionName, serving different users. For example, consider the following mapping:

| Principal ARN | lakeFS User |
| --- | --- |
| `arn:aws:sts::123456:assumed-role/Dev` | `foo` |
| `arn:aws:sts::123456:assumed-role/Dev/john@acme.com` | `john` |

if the bound ARN were `arn:aws:sts::123456:assumed-role/Dev/<SessionName>` it would allow any principal assuming `Dev` role in AWS account `123456` to login to it. If the `SessionName` is `john@acme.com` then lakeFS would return token for `john` user

### How AWS authentication works

The AWS STS API includes a method, `sts:GetCallerIdentity`, which allows you to validate the identity of a client. The client signs a GetCallerIdentity query using the AWS Signature v4 algorithm and sends it to the lakeFS server.

The `GetCallerIdentity` query consists of four pieces of information: the request URL, the request body, the request headers and the request method. The AWS signature is computed over those fields. The lakeFS server reconstructs the query using this information and forwards it on to the AWS STS service. Depending on the response from the STS service, the server authenticates the client.

Notably, clients don't need network-level access themselves to talk to the AWS STS API endpoint; they merely need access to the credentials to sign the request. However, it means that the lakeFS server does need network-level access to send requests to the STS endpoint.

Each signed AWS request includes the current timestamp to mitigate the risk of replay attacks. In addition, lakeFS allows you to require an additional header, `X-LakeFS-Server-ID` (added by default), to be present to mitigate against different types of replay attacks (such as a signed `GetCallerIdentity` request stolen from a dev lakeFS instance and used to authenticate to a prod lakeFS instance).

It's also important to note that Amazon does NOT appear to include any sort of authorization around calls to GetCallerIdentity. For example, if you have an IAM policy on your credential that requires all access to be MFA authenticated, non-MFA authenticated credentials will still be able to authenticate to lakeFS using this method.

## Server Configuration

> [!NOTE]
>
> **lakeFS Cloud**
>
> On the hosted service, installations that run on AWS come with external AWS authentication already enabled, so this section does not apply and there is no server configuration to change. Attaching IAM roles to users requires the IAM role authentication capability in your organization's subscription. Clients must log in through the installation's canonical hostname, since requests arriving through a custom domain or a PrivateLink endpoint fail signature verification.

> [!NOTE]
>
> **Info**
>
> lakeFS Helm chart supports the configuration below since version 1.5.0

To enable AWS IAM authentication in lakeFS Enterprise:

1. Enable external principals in lakeFS configuration
2. Configure external AWS authentication settings

If [`auth.allowed_authentication_methods`](<https://docs.lakefs.io/reference/enterprise-configuration/index.md#auth>) is set, it must include `external_principal` for IAM authentication to work.

**Helm Configuration (`values.yaml`):**

```yaml
ingress:
  enabled: true
  ingressClassName: <class-name>
  hosts:
    - host: <lakefs.ingress.domain>
      paths:
        - /

lakefsConfig: |
  auth:
    # Configure external AWS authentication
    external_aws_auth:
      enabled: true
      # the maximum age in seconds for the GetCallerIdentity request
      #get_caller_identity_max_age: 60
      # headers that must be present by the client when doing login request
      required_headers:
        # same host as the lakeFS server ingress
        X-LakeFS-Server-ID: <lakefs.ingress.domain>
```

> [!NOTE]
>
> By default, lakeFS clients will add the parameter `X-LakeFS-Server-ID: <lakefs.ingress.domain>` to the initial [login request](<https://docs.lakefs.io/reference/api/enterprise/index.md#/auth/externalPrincipalLogin>) for STS.

**Direct Configuration File (`lakefs.yaml`):**

```yaml
auth:
  external_aws_auth:
    enabled: true
    # Optional: max age for GetCallerIdentity requests (default: 24h)
    get_caller_identity_max_age: 3600
    # Required headers for login requests
    required_headers:
      X-LakeFS-Server-ID: <lakefs.ingress.domain>
    # Optional headers that may be present
    optional_headers:
      X-Custom-Header: value
```

## Administration of IAM Roles in lakeFS

Administration refers to the management of the IAM roles that are allowed to authenticate to lakeFS. Operations such as attaching and detaching IAM roles to a user, listing the roles attached to a user, and listing the users attached to a role. This can be done through lakectl, the lakeFS [External Principals API](<https://docs.lakefs.io/reference/api/enterprise/index.md#/auth/createUserExternalPrincipal>), or generated clients.

### Using lakectl

```bash
# Attach IAM role to a lakeFS user (omit --id to use current user)
lakectl auth users aws-iam attach [--id <lakefs-user>] \
    --principal-id 'arn:aws:sts::<id>:assumed-role/<role A>/<optional session name>'
lakectl auth users aws-iam attach [--id <lakefs-user>] \
    --principal-id 'arn:aws:sts::<id>:assumed-role/<role B>'

# Detach an IAM role from a user (omit --id to use current user)
lakectl auth users aws-iam detach [--id <lakefs-user>] \
    --principal-id 'arn:aws:sts::<id>:assumed-role/<role A>'

# List IAM roles attached to a user (omit --id to use current user)
lakectl auth users aws-iam list [--id <lakefs-user>]

# Lookup which lakeFS users are attached to an IAM role
lakectl auth users aws-iam lookup --principal-id 'arn:aws:sts::<id>:assumed-role/<role>'
```

### Using Python SDK

```python
import lakefs_sdk as lakefs

configuration = lakefs.Configuration(host = "...",username="...",password="...")
username = "<lakefs-user>"
api = lakefs.ApiClient(configuration)
auth_api = lakefs.AuthApi(api)

# attach the role(s)to a lakeFS user
auth_api.create_user_external_principal(
    user_id=username, principal_id='arn:aws:sts::<id>:assumed-role/<role A>/<optional session name>')
auth_api.create_user_external_principal(
    user_id=username, principal_id='arn:aws:sts::<id>:assumed-role/<role B>')

# list the roles attached to the user
resp = auth_api.list_user_external_principals(user_id=username)
for p in resp.results:
    # do something
```

### Using Java SDK

The generated `AuthApi` exposes the same external-principal endpoints as the Python SDK. Calls use the SDK's fluent builder pattern and are terminated with `.execute()`.

```java
import io.lakefs.clients.sdk.ApiClient;
import io.lakefs.clients.sdk.AuthApi;
import io.lakefs.clients.sdk.model.ExternalPrincipal;
import io.lakefs.clients.sdk.model.ExternalPrincipalList;

ApiClient apiClient = new ApiClient();
apiClient.setBasePath("https://lakefs.example.com/api/v1");
apiClient.setUsername("<access-key-id>");
apiClient.setPassword("<secret-access-key>");

AuthApi authApi = new AuthApi(apiClient);
String userId = "<lakefs-user>";

// attach the role(s) to a lakeFS user
authApi.createUserExternalPrincipal(userId,
        "arn:aws:sts::<id>:assumed-role/<role A>/<optional session name>")
    .execute();
authApi.createUserExternalPrincipal(userId,
        "arn:aws:sts::<id>:assumed-role/<role B>")
    .execute();

// list the roles attached to the user
ExternalPrincipalList resp = authApi.listUserExternalPrincipals(userId).execute();
for (ExternalPrincipal p : resp.getResults()) {
    // do something
}

// detach a role from the user
authApi.deleteUserExternalPrincipal(userId,
        "arn:aws:sts::<id>:assumed-role/<role A>")
    .execute();
```

> [!NOTE]
>
> Admin calls themselves can be authenticated with static lakeFS credentials (as shown above) or with an AWS IAM role — see [Login with Java](<#login-with-java>) to install `AWSIAMAuthProvider` on the same `ApiClient`.

## Get lakeFS API Token

The login to lakeFS is done by calling the [login API](<https://docs.lakefs.io/reference/api/enterprise/index.md#/auth/externalPrincipalLogin>) with the `GetCallerIdentity` request signed by the client. Currently, the login operation is supported out of the box in:

- [lakectl](<#login-with-lakectl>)
- [lakeFS Hadoop FileSystem](<https://docs.lakefs.io/integrations/spark/index.md#lakefs-hadoop-filesystem>) version 0.2.4, see [Spark usage](<https://docs.lakefs.io/integrations/spark/index.md#usage-with-temporaryawscredentialslakefstokenprovider>)
- [python](<#login-with-python>)
- [Java](<#login-with-java>)
- [lakeFS Mount](<https://docs.lakefs.io/mount/index.md#authentication-configuration>)

For other use cases authenticated to lakeFS via login endpoint, this will require building the request input.

## Login with lakectl

### prerequisites

1. lakeFS must be configured to allow external principals to authenticate. The relevant IAM role should be attached to the appropriate lakeFS user.
2. lakectl must be configured to use IAM authentication.

### lakectl configuration

> [!WARNING]
>
> **Deprecated**
>
> The `lakectl` distributed with lakeFS Community accepts the keys below so that existing configuration files keep loading, then ignores them and logs a warning, since lakeFS Community does not serve the login endpoint they call. Authenticating with AWS IAM needs the `lakectl` that ships with lakeFS Enterprise.

To use IAM authentication, the following configuration fields are available:

- `credentials.provider.type` `(string: '')` - Settings this `aws_iam` will expect `aws_iam` block and try to use IAM.
- `credentials.provider.aws_iam.token_ttl_second` `(duration: 60m)` - Optional: lakeFS token duration.
- `credentials.provider.aws_iam.url_presign_ttl_seconds` `(duration: 15m)` - Optional: AWS STS's presigned URL validation duration.
- `credentials.provider.aws_iam.refresh_interval` `(duration: 15m)` - Optional: Amount of time before token expiration that lakeFS Mount will try to fetch a new session token instead of using the current one.
- `credentials.provider.aws_iam.token_request_headers`: Map of required headers and their values to be signed by the AWS STS request as configured in your lakeFS server. If nothing is set the **default** behavior is adding `x-lakefs-server-id:<lakeFS host>`. If your lakeFS server doesn't require any headers (less secure) you can set this empty by setting `{}` empty map in your config.

These configuration fields can be set via `.lakectl.yaml`:

> [!NOTE]
>
> **Example**
>
> ```yaml
> credentials:
> provider:
>     type: aws_iam          # Required
>     aws_iam:
>       token_ttl_seconds: 60m              # Optional, default: 1h
>       url_presign_ttl_seconds: 15m        # Optional, default: 15m
>       refresh_interval: 5m                # Optional, default: 5m
>       token_request_headers:              # Optional, if omitted then will set x-lakefs-server-id: <lakeFS host> by default, to override default set to '{}'
>       # x-lakefs-server-id: <lakeFS host>     Added by default if token_request_headers is not set  
>       custome-key:  custome-val
> server:
> endpoint_url: <lakeFS endpoint url>
> ```

### Token Caching

To optimize lakectl's IAM authentication, a simple token caching mechanism was introduced. Instead of performing a login request to AWS to get a JWT for lakeFS, lakectl will save the token to `$HOMEDIR/.lakectl/cache/lakectl_token_cache.json`. The token will be used in the next lakectl operations up until one hour after writing. Later than that, lakectl will try and retrieve a new token.

> [!NOTE]
>
> Cache implementation does not support multiple AWS profiles. When switching profiles and for general troubleshooting, try and delete the cache first.

## Login with python

### prerequisites

1. lakeFS should be [configured](<#server-configuration>) to allow external principals to authenticate, and the used IAM role should be [attached](<#administration-of-iam-roles-in-lakefs>) to the relevant lakeFS user
2. The Python SDK requires additional packages to be installed to generate a lakeFS client with the assumed role. To install the required packages, run the following command:

```shell
  pip install "lakefs[aws-iam]"
```

There are two ways in which external principals can be used to authenticate to lakeFS:

1. If no other authentication flow is provided, and the `credentials.provider.type` configuration is set to `aws_iam` in `.lakectl.yaml`, the client will use the machine's AWS role to authenticate with lakeFS:

   ```yaml
   credentials:
       provider:
       type: aws_iam
       aws_iam:
           token_ttl_seconds: 3600      # TTL for the temporary token (default: 3600)
           url_presign_ttl_seconds: 60  # TTL for presigned URLs (default: 60)
           token_request_headers:       # Optional headers for token requests
           HeaderName: HeaderValue
   ```

   Or using environment variables:

   ```bash
   export LAKECTL_CREDENTIALS_PROVIDER_TYPE="aws_iam"
   export LAKECTL_CREDENTIALS_PROVIDER_AWS_IAM_TOKEN_TTL_SECONDS="3600"
   export LAKECTL_CREDENTIALS_PROVIDER_AWS_IAM_PRESIGNED_URL_TTL_SECONDS="60"
   export LAKECTL_CREDENTIALS_PROVIDER_AWS_IAM_TOKEN_REQUEST_HEADERS='{"HeaderName":"HeaderValue"}'
   ```

   To use the client, merely `import lakefs` and use it as you would normally do:

   ```python
   import lakefs

   for branch in lakefs.repository("example-repo").branches():
   print(branch)
   ```

   > **Warning**
   >
   > Please note, using the IAM provider configurations will not work with the lakectl command line tool, and will stop you from running it.
2. Generate a lakeFS client with the assumed role by initiating a boto3 session with the desired role and call `lakefs.client.frow_aws_role`:

   ```python
   import lakefs
   import boto3    

   session = boto3.Session()
   my_client = lakefs.client.from_aws_role(session=session, ttl_seconds=7200, host="<lakefs-host>")

   # list repositories
   repos = lakefs.repositories(client=my_client)
   for r in repos:
       print(r)
   ```

## Login with Java

### prerequisites

1. lakeFS should be [configured](<#server-configuration>) to allow external principals to authenticate, and the used IAM role should be [attached](<#administration-of-iam-roles-in-lakefs>) to the relevant lakeFS user.
2. The lakeFS Java SDK declares `software.amazon.awssdk:sts` as an optional dependency, so applications using IAM authentication must add it explicitly:

   ```xml
   <dependency>
       <groupId>software.amazon.awssdk</groupId>
       <artifactId>sts</artifactId>
       <version>2.33.12</version>
   </dependency>
   ```

### Usage

The `io.lakefs.clients.sdk.auth.iam` package installs an OkHttp interceptor on an `ApiClient` that presigns an `sts:GetCallerIdentity` request, exchanges it for a lakeFS JWT via the [login API](<https://docs.lakefs.io/reference/api/enterprise/index.md#/auth/externalPrincipalLogin>), caches the token, and refreshes it transparently before it expires. AWS credentials, region, and the STS endpoint are resolved from the ambient environment (default credentials provider chain, default region provider chain).

```java
import io.lakefs.clients.sdk.ApiClient;
import io.lakefs.clients.sdk.RepositoriesApi;
import io.lakefs.clients.sdk.auth.iam.AWSIAMAuthProvider;
import io.lakefs.clients.sdk.auth.iam.IAMAuthParams;

ApiClient apiClient = new ApiClient();
apiClient.setBasePath("https://lakefs.example.com/api/v1");

IAMAuthParams params = IAMAuthParams.builder("lakefs.example.com").build();
AWSIAMAuthProvider.install(apiClient, params);

RepositoriesApi repos = new RepositoriesApi(apiClient);
```

The `lakefsHost` value passed to `IAMAuthParams.builder(...)` is automatically added as the `X-LakeFS-Server-ID` header on the signed `GetCallerIdentity` request, matching the default `required_headers` on the [server](<#server-configuration>).

### Configuration

`IAMAuthParams.builder(...)` exposes the same tunables as the Python SDK and lakectl:

| Builder method | Description | Default |
| --- | --- | --- |
| `urlPresignTtl` | Validity of the presigned `GetCallerIdentity` URL before lakeFS forwards it to STS. | `Duration.ofMinutes(1)` |
| `tokenTtl` | Requested lifetime of the lakeFS JWT. | `Duration.ofHours(1)` |
| `refreshInterval` | Window before token expiry during which the next intercepted request triggers a synchronous re-login. Must be strictly less than `tokenTtl`. | `Duration.ofMinutes(5)` |
| `tokenRequestHeaders` | Extra headers to include in the signed STS request. The `X-LakeFS-Server-ID` header is added automatically. | *empty* |

```java
import java.time.Duration;
import java.util.Collections;

IAMAuthParams params = IAMAuthParams.builder("lakefs.example.com")
    .urlPresignTtl(Duration.ofMinutes(2))
    .tokenTtl(Duration.ofHours(2))
    .refreshInterval(Duration.ofMinutes(10))
    .tokenRequestHeaders(Collections.singletonMap("X-Custom-Header", "value"))
    .build();

AWSIAMAuthProvider.install(apiClient, params);
```

> [!NOTE]
>
> The provider keeps the token in memory only — there is no on-disk cache. For long-running JVM processes this is typically what you want. See [Persisting tokens across restarts](<#persisting-tokens-across-restarts>) if your process restarts often and you want to skip the initial login round trip.

### Persisting tokens across restarts

`AuthenticationToken` has two fields — `token` (the JWT) and `tokenExpiration` (Unix epoch seconds) — so you can persist them however you like. The SDK ships Gson, so the simplest path is to serialize the model directly. Capture the provider returned by `install(...)`, save its current token on shutdown, and rehydrate via `initialToken(...)` on startup:

```java
import com.google.gson.Gson;
import io.lakefs.clients.sdk.model.AuthenticationToken;

// On startup — rehydrate (if a cached token exists) and install.
AuthenticationToken restored = null;
if (Files.exists(cachePath)) {
    restored = new Gson().fromJson(Files.readString(cachePath), AuthenticationToken.class);
}

AWSIAMAuthProvider provider = AWSIAMAuthProvider.builder()
    .apiClient(apiClient)
    .params(params)
    .initialToken(restored)   // null is fine — first call will log in
    .install();

// ... use apiClient as normal ...

// On shutdown — save the current token.
AuthenticationToken current = provider.getCurrentToken();
if (current != null) {
    Files.writeString(cachePath, new Gson().toJson(current));
}
```

The provider serves the cached token until it lands inside `refreshInterval` of expiry; the next intercepted request then triggers a synchronous re-login. Persist whatever protections your environment requires (file mode, OS keychain, secret manager) — the JWT is bearer credentials.
