Link Search Menu Expand Document

Airflow Hooks

Table of contents

  1. Action file Airflow hook properties
  2. Hook Record in configuration field

Airflow Hook triggers a DAG run in an Airflow installation using Airflow’s REST API. The hook run succeeds if the DAG was triggered, and fails otherwise.

Action file Airflow hook properties

Property Description Data Type Example Required Env Vars Support
url The URL of the Airflow instance String http://localhost:8080 true no
dag_id The DAG to trigger String example_dag true no
username The name of the Airflow user performing the request String admin true no
password The password of the Airflow user performing the request String admin true yes
dag_conf DAG run configuration that will be passed as is JSON   false no
wait_for_dag Wait for DAG run to complete and reflect state (default: false) Boolean   false no
timeout Time to wait for the DAG run to complete (default: 1m) String (golang’s Duration representation)   false no

Example:

...
hooks:
  - id: trigger_my_dag
    type: airflow
    description: Trigger an example_dag
    properties:
       url: "http://localhost:8000"
       dag_id: "example_dag"
       username: "admin"
       password: "{{ ENV.AIRFLOW_SECRET }}"
       dag_conf:
          some: "additional_conf"
...

Hook Record in configuration field

lakeFS will add an entry to the Airflow request configuration property (conf) with the event that triggered the action.

The key of the record will be lakeFS_event and the value will match the one described here