Using lakeFS with Trino / Presto¶

Trino and Presto are distributed SQL query engines designed to query large data sets distributed over one or more heterogeneous data sources.

Iceberg REST Catalog¶

The recommended way to use lakeFS with Trino/Presto is through the lakeFS Iceberg REST Catalog. lakeFS stays completely outside the data path, so data is read and written by Trino/Presto executors directly to the underlying object store, while lakeFS versions the table metadata.

Configuration¶

To use the Iceberg REST Catalog, you need to configure Trino/Presto to use the Iceberg REST catalog endpoint:

# example: /etc/trino/catalog/lakefs.properties
connector.name=iceberg

# REST Catalog connection
iceberg.catalog.type=rest
iceberg.rest-catalog.uri=https://lakefs.example.com/iceberg/api
iceberg.rest-catalog.nested-namespace-enabled=true

# REST Catalog authentication
iceberg.rest-catalog.security=OAUTH2
iceberg.rest-catalog.oauth2.credential=${ENV:LAKEFS_CREDENTIALS}
iceberg.rest-catalog.oauth2.server-uri=https://lakefs.example.com/iceberg/api/v1/oauth/tokens

# Object storage access to underlying tables (modify this to match your storage provider)
fs.hadoop.enabled=false
fs.native-s3.enabled=true
s3.region=us-east-1
s3.aws-access-key=${ENV:AWS_ACCESS_KEY_ID}
s3.aws-secret-key=${ENV:AWS_SECRET_ACCESS_KEY}

Usage¶

Once configured, you can use the Iceberg REST Catalog to query and update Iceberg tables.

USE "repo.main.inventory";
SHOW TABLES;
SELECT * FROM books LIMIT 100;

USE "repo.new_branch.inventory";
SHOW TABLES;
SELECT * FROM books LIMIT 100;