Feeling stagnated as a Data Engineer at a startup — need career advice on switching
Hi everyone, looking for some genuine career advice here.
I'm currently working at a startup as a Data Engineer for the past one year. The company does client data analysis through ML models. My role mainly involves gathering data from client sources (APIs, Google Drive, etc.) and transforming it into a model-acceptable format so the ML team can extract insights. I primarily work with PostgreSQL and Python for this.
Apart from that, I also do some backend work on the company's product (mainly their website), but outside of these two things, there isn't much else on my plate.
Before this, I worked at TCS in a Digital role for about 5 months, but I was on the bench the entire time. I only got exposure to AWS, Docker, and Java/JDBC/Servlets fundamentals — never actually got staffed on a project. Eventually I left and joined my current company.
Now I'm planning to switch to a proper Data Engineer role because I feel stagnated in terms of growth. When I look at other data engineers around me, they're working with PySpark, Databricks, and large-scale production pipelines — stuff I have zero hands-on exposure to. My work is fairly basic in comparison, and my salary reflects that too (which, honestly, feels fair given the scope of my current work).
So I wanted to ask this community:
- What should I focus on learning to actually become job-ready for a real Data Engineer role (PySpark, Databricks, pipeline orchestration, etc.)?
- Should I try building projects on my own to simulate that production-pipeline experience, or is there a better way to get exposure?
- Any advice on how to position my current experience (which is more data-wrangling than true DE work) when applying for DE roles elsewhere?
Would really appreciate insights from people who've been in a similar spot or are currently working as DEs in bigger setups. Thanks in advance!
[Used AI to phrase my query better, as im bad at it 😭 ]