DataEngineerBootcamp

Week 5 Data Pipeline Maintenance

Homework

Imagine you’re in a group of 4 data engineers, you will be in charge of creating the following things:

You are in charge of managing these 5 pipelines that cover the following business areas:

• Profit
• Unit-level profit needed for experiments
• Aggregate profit reported to investors
• Growth
• Aggregate growth reported to investors
• Daily growth needed for experiments
• Engagement
• Aggregate engagement reported to investors

You are in charge of figuring out the following things:

• Who is the primary and secondary owners of these pipelines?
• What is an on-call schedule that is fair
  • Think about holidays too!
• Creating run books for all pipelines that report metrics to investors
  • What could potentially go wrong in these pipelines?
  • (you don’t have to recommend a course of action though since this is an imaginatione exercise)

Create a markdown file submit it!