Although Airflow can send an email upon task failure, sometime it's not good enough for you. You may have other kind of communication channels like Slack, or you may just want to post it to the SNS like Twitter. In this case, on_success_callback and on_failure_callback could be a good choice. These callback functions are called… Continue reading Enhance Airflow Task Monitoring with Custom Callback Functions and Webhooks
Apache Airflow is a platform to programmatically author, schedule and monitor workflows. For whom has lots of scheduled jobs to run, Airflow is quite a mind-blowing platform. Although it also has some weird points, I think the good overcomes the bad. Among the many characteristics of Airflow, let's focus on job scheduling here. Actually, setting… Continue reading Understanding Airflow Job Scheduling: Timetable Concept Explained
"Hello, World" is the message being used when you are learning a software language for the first time. Almost every software language provides a small program printing this message1) to illustrate its basic syntax. Although I have almost 20 years of experiences in software development and data engineering, most of my experiences are from non-English… Continue reading Hello, World
Kirin's Technology Essay 