DataEngineerBootcamp

The Data Engineering Handbook

This repo has all the resources you need to become an amazing data engineer!

Getting started

If you are new to data engineering, start by following this 2024 breaking into data engineering roadmap

If you are here for the 6-week free YouTube boot camp you can check out

For more applied learning:

Check out the projects section for more hands-on examples!
Check out the interviews section for more advice on how to pass data engineering interviews!
Check out the books section for a list of high quality data engineering books
Check out the communities section for a list of high quality data engineering communities to join
Check out the newsletter section to learn via email

Resources

Great list of over 25 books

Great list of over 10 communities to join:

Top must-join communities for DE:

Top must-join communities for ML:

Companies:

Orchestration
- Mage
- Astronomer
- Prefect
- Dagster
- Airflow
- Kestra
- Shipyard
- Hamilton
Data Lake / Cloud
- Tabular
- Microsoft
- Databricks
- Onehouse
- Delta Lake
- Ilum
Data Warehouse
Data Quality
- dbt
- Metaplane
- Gable
- Great Expectations
- Streamdal
- Coalesce
- Soda
- DQOps
- HEDDA.IO
- Dingo
Education Companies
Analytics / Visualization
- Preset
- Starburst
- Metabase
- Looker Studio
- Tableau
- Power BI
- Hex
- Apache Superset
- Evidence
- Redash
- Lightdash
Data Integration
- Cube
- Fivetran
- Airbyte
- dlt
- Sling
- Meltano
Semantic Layers
- Cube
- dbt Semantic Layer
Modern OLAP
- Apache Druid
- ClickHouse
- Apache Pinot
- Apache Kylin
- DuckDB
- QuestDB
- StarRocks
LLM application library
Real-Time Data
Data Lineage
- OpenLineage

Data Engineering blogs of companies:

Data Engineering Whitepapers:

Here’s the mostly comprehensive list of data engineering creators: (You have to have at least 5k followers somewhere to be added!)

YouTube

Name	LinkedIn Profile	Follower Count
Zach Wilson	Zach Wilson	400,000+
Chip Huyen	Chip Huyen	250,000+
Shashank Mishra	Shashank Mishra	100,000+
Seattle Data Guy	Ben Rogojan	100,000+
TrendyTech	Sumit Mittal	100,000+
Darshil Parmar	Darshil Parmar	100,000+
Andreas Kretz	Andreas Kretz	100,000+
ByteByteGo (Alex Xu)	Alex Xu	100,000+
Azure Lib (Deepak Goyal)	Deepak Goyal	100,000+
Alex Freberg	Alex Freberg	100,000+
SQLBI (Marco Russo)	Marco Russo	50,000+
Ankit Bansal	Ankit Bansal	50,000+
Marc Lamberti	Marc Lamberti	50,000+
Ankur Ranjan	Ankur Ranjan	48,000+
ITVersity (Durga Gadiraju)	Durga Gadiraju	48,000+
Prashanth Kumar Pandey	Prashanth Kumar Pandey	37,000+
Alex Merced	Alex Merced	30,000+
Ijaz Ali	Ijaz Ali	24,000+
Mehdi Ouazza	Mehdi Ouazza	20,000+
Ananth Packkildurai	Ananth Packkildurai	18,000+
Ansh Lamba	Ansh Lamba	13,000+
Manojkumar Vadivel	Manojkumar Vadivel	12,000+
Advancing Analytics	Simon Whiteley	10,000+
Li Yin	Li Yin	10,000+
Jaco van Gelder	Jaco van Gelder	10,000+
Joseph Machado	Joseph Machado	10,000+
Eric Roby	Eric Roby	10,000+
Simon Späti	Simon Späti	10,000+
Constantin Lungu	Constantin Lungu	10,000+
Lakshmi Sontenam	Lakshmi Sontenam	9,500+
Soumil Shah	Soumil Shah	8,000+
Arnaud Milleker	Arnaud Milleker	7,000+
Dimitri Visnadi	Dimitri Visnadi	7,000+
Lenny	Lenny A	6,000+
Dipankar Mazumdar	Dipankar Mazumdar	5,000+
Daniel Ciocirlan	Daniel Ciocirlan	5,000+
Hugo Lu	Hugo Lu	5,000+
Tobias Macey	Tobias Macey	5,000+
Marcos Ortiz	Marcos Ortiz	5,000+
Julien Hurault	Julien Hurault	5,000+
John Kutay	John Kutay	5,000+
Hassaan Akbar	Hassaan Akbar	5,000+
Subhankar	Subhankar	5,000+
Nitin	Nitin	N/A
Hassaan	Hassaan	5000+
Javier de la Torre	Javier	5000+

X/Twitter

Name	X/Twitter Profile	Follower Count
ByteByteGo	alexxubyte	100,000+
Dan Kornas	@dankornas	66,000+
Zach Wilson	EcZachly	30,000+
Seattle Data Guy	SeattleDataGuy	10,000+
SQLBI	marcorus	10,000+
Joseph Machado	startdataeng	5,000+
Alex Merced	@amdatalakehouse	N/A
John Kutay	@JohnKutay	N/A
Mehdi Ouazza	mehd_io	N/A

Instagram

Name	Instagram Profile	Follower Count
Sundas Khalid	sundaskhalidd	300,000+
Zach Wilson	eczachly	150,000+
Andreas Kretz	learndataengineering	5,000+
Alex Merced	@alexmercedcoder	N/A

TikTok

Name	TikTok Profile	Follower Count
Zach Wilson	@eczachly	70,000+
Alex Freberg	@alex_the_analyst	10,000+
Mehdi Ouazza	@mehdio_datatv	N/A

Great Podcasts

The Data Engineering Show
Data Engineering Podcast
DataTopics
The Data Engineering Side Of Data
DataWare
The Data Coffee Break Podcast
The Datastack show
Intricity101 Data Sharks Podcast
Drill to Detail with Mark Rittman
Analytics Power Hour
Catalog & cocktails
Datatalks
Data Brew by Databricks
The Data Cloud Podcast by Snowflake
What’s New in Data
[Open Source Data by Datastax](https://www.datastax.com/resources/podcast/open-source-data)
Streaming Audio by confluent
The Data Scientist Show
MLOps.community
Monday Morning Data Chat
The Data Chief

Great list of 20+ newsletters

Top must follow newsletters for data engineering:

Glossaries:

Design Patterns

Courses / Academies

DataExpert.io course use code HANDBOOK10 for a discount!
LearnDataEngineering.com
Technical Freelancer Academy Use code zwtech for a discount!
IBM Data Engineering for Everyone
Qwiklabs
DataCamp
Udemy Courses from Shruti Mantri
Rock the JVM teaches Spark (in Scala), Flink and others
Data Engineering Zoomcamp by DataTalksClub
Efficient Data Processing in Spark
Scaler
DataTeams - Data Engingeer hiring platform
Udemy Courses from Daniel Blanco