Link Search Menu Expand Document

Using lakeFS with Glue ETL

AWS Glue is a fully managed extract, transform, and load (ETL) service. With AWS Glue ETL, you can run your ETL jobs as soon as new data becomes available in Amazon S3 by invoking your AWS Glue ETL jobs from an AWS Lambda function.

Configuration

Since Glue ETL is essentially running Spark jobs, to configure Glue ETL to work with lakeFS, all you need to do it apply the lakeFS Spark configuration to your Glue ETL script.