Abstract: This session will talk about the awesome new features the community has built that were recently released in. Apache Airflow 2.3.
- Dynamic Task Mapping
- First-class support for DB Downgrades
- Pruning old DB records (No need of using Maintenance DAGs anymore)
- Building Connections using JSON
- UI Improvements
The talk will also cover the growth of Airflow Community over years and why Airflow is still the defacto tool for Workflow Orchestration.
Bio: Kaxil is currently working as the Director of Airflow Engineering Team @ Astronomer. Currently, he is one of the top three committers of the Airflow Project based on the number of commits. He is one of the release managers of Airflow. Most prominent works include co-authoring DAG Serialization, Scheduler HA, Secrets Backend.
He did his Masters in Data Science & Analytics from Royal Holloway, University of London. Started as a Data Scientist and then gained experience in Data Engineering, BigData and DevOps space. He began working on Airflow in 2017 while working at Data Reply as a BigData consultant and became a PMC member in 2018 and now works full-time at astronomer.io making Airflow better for everyone. He is a huge cricket fan and his favourite cricketers are Rahul Dravid and Virat Kohli.
Director of Airflow Engineering | Astronomer