Configuring lakeFS to use presigned URLs

With lakeFS, you can access data directly from the storage and not through lakeFS using a presigned URL. Based on the user’s access to an object in the object store, the presigned URL will get read or write access. The presign support is enabled for block adapter that supports it (S3, GCP, Azure), and can be disabled by the configuration (blockstore.blockstore-name.disable_pre_signed). Note that the UI support is disabled by default.

Using presigned URLs in the UI

For using presigned URLs in the UI:

1. Enable the presigned URL support UI in the lakeFS configuration (blockstore.blockstore-name.disable_pre_signed_ui).
2. Add CORS (Cross-Origin Resource Sharing) permissions to the bucket for the UI to fetch objects using a presigned URL (instead of through lakeFS).
3. The disable_pre_signed needs to be enabled to enable it in the UI.

⚠️ Note Currently DuckDB fetching data from lakeFS does not support fetching data using presigned URL.

Example: AWS S3

  [
    {
        "AllowedHeaders": [
            "*"
        ],
        "AllowedMethods": [
            "GET",
            "PUT"
        ],
        "AllowedOrigins": [
            "lakefs.endpoint"
        ],
        "ExposeHeaders": [
            "ETag"
        ]
    }
  ]

Example: Google Storage

  [
    {
        "origin": ["lakefs.endpoint"],
        "responseHeader": ["ETag"],
        "method": ["PUT", "GET"],
        "maxAgeSeconds": 3600
    }
   ]

Example: Azure blob storage

  <Cors>
      <CorsRule>  
          <AllowedOrigins>lakefs.endpoint</AllowedOrigins>  
          <AllowedMethods>PUT,GET</AllowedMethods>  
          <AllowedHeaders>*</AllowedHeaders>  
          <ExposedHeaders>ETag,x-ms-*</ExposedHeaders>  
          <MaxAgeInSeconds>3600</MaxAgeInSeconds>  
      </CorsRule>  
  </Cors>