Link Search Menu Expand Document

Using lakeFS with Glue ETL

AWS Glue is a fully managed extract, transform, and load (ETL) service. With AWS Glue ETL you can run your ETL jobs as soon as new data becomes available in Amazon S3 by invoking your AWS Glue ETL jobs from an AWS Lambda function.


Since Glue ETL is essentially running Spark jobs, to configure Glue ETL to work with lakeFS, you should apply the lakeFS Spark configuration to your Glue ETL script.