E333 - The Hidden Problems Behind Every Data Pipeline

#333: Pete Hunt, CEO of Dagster and early React team member, explores the evolution from Facebook's early React development through trust and safety infrastructure at Twitter, to building modern data orchestration tools. The conversation reveals how similar infrastructure problems plague every industry - whether you're launching rockets or managing porta-potties, the core challenges remain consistent: late data, quality issues, and mysterious errors that require both automated solutions and human oversight.

The discussion dives into the technical realities of scaling systems, from the microservices complexity trap to the current AI adoption wave. Hunt shares candid insights about leadership challenges...