Why great data engineers think like product managers
When I started out, I thought being a good data engineer was all about clean pipelines and perfect schemas. Get the modelling right, keep the jobs green, and the work is done.
Over time, I realised something that reshaped how I approach the job: the best data engineers don't just move data, they move decisions.
Think in outcomes, not objects
A table is an object. A pipeline is an object. A schema is an object. It's easy to spend an entire career optimising objects and never ask what they're for.
The engineers I learn the most from think one layer up. They ask:
- Who's actually using this data?
- What decision depends on it?
- How does it impact the business?
Those three questions change everything. They turn "build the table the ticket asked for" into "understand the decision this table exists to support", and those are not the same task. One produces a correct artifact. The other produces a useful one.
When ETL jobs become features
Once you start thinking like a product owner, your ETL jobs stop being just processes. They become features that deliver real business value. A pipeline isn't plumbing anymore, it's the thing that lets someone in finance close the books a day earlier, or lets a team catch a problem before a customer does.
That shift in framing is subtle but powerful. It changes what you prioritise, what you monitor, and what you consider "done." A job that runs successfully but feeds a decision nobody trusts isn't finished. A job that quietly powers a confident, correct decision every single day, that's the work that matters.
Because data that's correct is great. But data that drives action? That's powerful.
I'm Yash Agarwal, a Data Engineer II at Amdocs in Pune, India. I write about building reliable, large-scale data platforms and the mindset that makes data engineering matter. You can find more of my work on my portfolio or connect with me on LinkedIn.