u/OkRock1009

Hello everyone. I am a 2025 passout. My CTC is 3.7 LPA

My tech stack is Python, SQL, Bigquery, Datastream, Looker

I now have 8 months of experience working as a GCP data engineer for a mid sized service based company. I have worked on a couple of projects. One being an ETL project and the another one being replicating data from MYSQL to BigQuery in real time. Other than this i have also implemented BigQuery cost optimization strategies for a few clients. Thinking of switching by this year end. What exactly should I focus on?

Hello Everyone!

I want to basically replicate data from my cloud sql instance to Big Query. The problem is since the initial load is expensive , I am gonna use a dump for that and only want the real time data to be captured.

I want it to create empty datasets and tables in Big Query automatically without the initial historical data. Any other solution?