It's been almost 1 year since I've adopted Apache Airflow to my company. This post would be a quick lookback for this decision. Although the Airflow is in action, it has been only used as an alternative to the cron daemon. One of the biggest reasons was that every team members were busy to handling… Continue reading A year with the Apache Airflow
Although Airflow can send an email upon task failure, sometime it's not good enough for you. You may have other kind of communication channels like Slack, or you may just want to post it to the SNS like Twitter. In this case, on_success_callback and on_failure_callback could be a good choice. These callback functions are called… Continue reading Enhance Airflow Task Monitoring with Custom Callback Functions and Webhooks
Apache Airflow is a platform to programmatically author, schedule and monitor workflows. For whom has lots of scheduled jobs to run, Airflow is quite a mind-blowing platform. Although it also has some weird points, I think the good overcomes the bad. Among the many characteristics of Airflow, let's focus on job scheduling here. Actually, setting… Continue reading Understanding Airflow Job Scheduling: Timetable Concept Explained
Kirin's Technology Essay 