Best Practicies & Success Stories
ponedjeljak, 16. listopad 2023., 17:00
Thousands of ETL delivery jobs. Hundreds of clusters.Computational infrastructure hitting AWS region limits, legacy technology meeting bleeding edge improvements, throttled AWS API calls, tight SLAs, ... Squeezing the last drop from Airflow, Luigi and other orchestration tools to meet the demands.On top of that, three acquisitions, three different tech stacks being merged into one...This is a talk about bad choices made, good decisions implemented, but above all, about what has been learned along the way.