Skip to content

Monitor Apache Airflow with StatsD | Prometheus | Grafana

2021 16 min MaxcoTec Learning en YouTube

#apacheairflow #airflowforbeginners #airflowmetrics #statsd #prometheus #grafana This is the second part of Airflow Monitoring Metric, in which we will continue working on building airflow metrics. Watch the first part here: https://youtu.be/xyeR_uFhnD4 We will learn about how to monitor airflow cluster by taking advantages of useful airflow statsd metrics. This video covers following; 00:00 Intro 00:48 write Prometheus service 02:26 check Prometheus metrics 04:19 write Grafana service 05:07 configure grafana data source 05:44 build grafana dashboard 07:12 configuring dashboard 09:10 writing custom metrics 12:52 test custom metrics 14:09 build grafana panel for custom metrics 15:43 outro --------------------------------------------------- Learn Something New Below: https://maxcotec.com/blog/introduction-to-daos-everything-you-need-to-know/ https://maxcotec.com/blog/what-is-ncrave-social-promo-facebook/ ๐Ÿ‘ If this video was helpful to you, please don't forget to thumbs-up, โŒจ๏ธ leave a comment and ๐Ÿ”— share with your friends. โญ Support our work by hitting that subscribe button. ๐Ÿ”” hit bell icon to stay alert on more upcoming interesting stuff. Get complete code from here; https://github.com/maxcotec/Apache-Airflow Grafana and StatsD configs https://github.com/databand-ai/airflow-dashboards Statsd python client https://github.com/jsocol/pystatsd Watch Part 1 here: https://youtu.be/xyeR_uFhnD4 Watch how to run airflow locally via Docker https://youtu.be/28UI_Usxbqo

Chapters

  1. 0:00 Intro
  2. 0:48 write Prometheus service
  3. 2:26 check Prometheus metrics
  4. 4:19 write Grafana service
  5. 5:07 configure grafana data source
  6. 5:44 build grafana dashboard
  7. 7:12 configuring dashboard
  8. 9:10 writing custom metrics
  9. 12:52 test custom metrics
  10. 14:09 build grafana panel for custom metrics
  11. 15:43 outro
Channel
MaxcoTec Learning
Views
169

More like this