What I've learned in 2 years as a data engineer (at 24)
When I started my career, I was convinced that mastering tools was everything. Python, SQL, Snowflake, Teradata, cloud platforms, I chased them all, certain that the next tool was the thing standing between me and being "good."
Then I got to work on projects that quietly rearranged that belief:
- Migrating 1000+ tables and 45B+ records for AT&T
- Building a production ready, GenAI-based pipeline
- Optimising billing systems that handle millions of dollars daily
And somewhere in the middle of all that, I realised something I didn't expect.
Tools change. Fundamentals compound.
Tools will keep evolving. The framework that's essential today will be legacy in a few years, and the cloud service everyone's learning right now will be renamed twice before it stabilises. Chasing tools is running on a treadmill that keeps speeding up.
What actually matters, what made those projects succeed at enterprise scale, was never the tool. It was problem-solving, system design, and ensuring data integrity. Those are the skills that survive every migration, every platform change, every hype cycle. They're what's left standing when the logos on your résumé go out of date.
That's what truly future proofs a career in data: not the surface area of tools you've touched, but the depth of the thinking you bring to them.
If you're just starting out
Here's the advice I'd give my younger self:
Focus on strong fundamentals. The tools will follow, but your foundation stays with you forever. Learn why a system behaves the way it does, not just which button produces the result. Understand how data moves, where it breaks, and what "correct" really means at scale. Get those right and you can pick up any tool in a week. Skip them and no tool will save you.
Two years in, that's the lesson I'm most grateful I learned early.
I'm Yash Agarwal, a Data Engineer II at Amdocs in Pune, India. I write about building reliable, large-scale data platforms and growing a career in data. You can find more of my work on my portfolio or connect with me on LinkedIn.