Hello all,
Any senior azure data engineer here?
I am preparing for Azure data engineer interviews(immediate joiner)with no hands on experience. Need your help for preparing techno managerial round. Please reach out to me.
Hi everyone,
I may have an upcoming interview for a Senior Data Platform Engineer role, and I’m trying to prepare properly.
My background is mainly as an Azure Platform Engineer — I’ve worked with Azure networking, CI/CD, security, infrastructure, App Services, AKS, monitoring, etc. However, I was told that a big portion of the interview may focus on Databricks and data engineering concepts.
For those who work with Azure Databricks or interview candidates for similar roles:
I’d also appreciate any advice on how platform engineering experience can be positioned well for a data platform role.
Thanks in advance!
Hello everyone . I’m interested in a Data Engineering career. I’ve spent the past 8 months learning many of the common stacks listed in DE jobs, including Azure Databricks, and I was wondering if you have any advice for someone trying to get into it without tech background. Whether it’s types of projects you suggest having on my Portfolio or anything that can help getting recruited.
Thank you!
Hi everyone,
I’m currently preparing for Data Engineering interviews(currently DE at product based company with 3 yoe), especially rounds focused on system design, scalability, distributed systems, and large-scale data processing.
I’m comfortable with:
Python, SQL, PySpark
Batch + streaming pipelines
Medallion architecture (Bronze/Silver/Gold)
Lambda/Kappa architecture
CDC, incremental loads vs full loads
Partitioning, bucketing, optimization concepts
Data warehousing vs OLTP systems
Airflow orchestration and cloud ecosystem understanding
However, one challenge I’m facing is that in my actual work experience, I haven’t directly handled truly massive-scale systems like:
500TB / petabyte-scale pipelines
billions of events per day
Extremely high-throughput streaming systems
In interviews, I’m seeing a lot of questions like:
“How would you scale your pipeline from 500GB to multiple TBs?”
“How would you handle billions of rows in S3 without full scans?”
“How would you detect duplicates/missing data efficiently?”
“How would you design real-time fraud detection pipelines?”
“How would you optimize shuffle-heavy Spark jobs?”
“How would you handle skew, late-arriving data, retries, SLA failures, and observability?”
My questions are:
What is the best way to genuinely build system design thinking for large-scale data engineering systems if you haven’t worked at that scale directly?
How do experienced engineers think through:
scalability
distributed bottlenecks
non-functional requirements
reliability
throughput vs latency tradeoffs
operational concerns during interviews?
Is there a practical way to get hands-on exposure to these concepts outside production environments? For example:
open-source projects
simulation environments
cloud labs
datasets
personal projects
system design exercises
Kafka/Spark mini architectures etc.
How would you recommend preparing specifically for:
incremental loads vs full loads
CDC pipelines
data quality validation at scale
Spark optimization
partitioning strategy
real-time + analytical hybrid systems
handling missing/corrupted/late data
observability and monitoring
near real-time data pipelines with high traffic
Would really appreciate advice from people who’ve worked on large-scale systems or cracked strong DE/system design interviews recently.
Thanks in advance!
Hi Folks,
Has anyone recently appeared for a Data Engineer interview at DGLiger or interviewed there in the past?
If yes, could you please share your interview experience — like the interview rounds, technical questions asked, difficulty level, and the kind of skills/topics they focused on (SQL, Python, ETL, Spark, Cloud, etc.)?
It would really help people preparing for the process. Thanks in advance!
Hey everyone,
I'm a data engineer and I can build complete data pipelines for your company. I also do Power BI/Tableau dashboards for businesses. If you need anything automated, I can set up Power Automate workflows too. Here's what I'm good at:
SQL, pyspark, databricks, agentbricks, ADF, power bi, tableau, servicenow, troubleshooting, performance tuning, informatica iics.
Got any work for me? DM me!
Hi everyone,
I’m a Data Engineer from Pakistan with experience in ETL pipelines, SQL, Python,Talend, Kafka, AWS, and BI systems. Currently working in the analytics/data engineering domain and exploring opportunities in Saudi Arabia or GCC countries.
I’d appreciate any guidance, recruiter contacts, or referrals. Thank you.
I want to switch my role to Data Engineer and would appreciate guidance from people in the data engineering industry.
Skills I currently have:
Database:
SQL Server, good with both theory and practical concepts
Studied SQL up to an advanced level and practiced on LeetCode
Data Warehouse:
Basic understanding
Built one project for practical experience
DSA:
I’ve been coding in Java and feel comfortable with it. Planning to continue DSA in Java
Would like to know how much DSA is required for Data Engineering and if there’s a specific list/topics I should cover
Cloud:
Completed Stephan Maarek’s Cloud Practitioner course. Planning to revisit and strengthen concepts again
Python:
Basic knowledge (Beginner)
Planning to start learning:
Spark
Airflow
Please also suggest good learning resources/material for Spark and Airflow.
Please let me know:
What more I need to do
What can be dropped
What type of projects I should add to strengthen my profile
Hi everyone,
I recently got accepted for a Data Engineering internship at a small startup, and I’m both excited and a bit overwhelmed. Since the company is still small, there isn’t really a structured mentorship or established data infrastructure yet, so I’ll have to handle a lot of things independently.
I’ll basically be starting from scratch:
The data comes in many formats (PDFs, presentations, images, logs, documents, etc.), so I’m trying to figure out the best way to structure everything professionally from the beginning.
I have theoretical knowledge and some academic/project experience, but this is my first real-world experience where the work will actually be used in production, so I want to avoid bad decisions early on.
I’d really appreciate advice on:
Any advice, roadmap suggestions, GitHub repos, YouTube channels, or real-world tips would help a lot. Thanks!
I want to switch my role to Data Engineer and would appreciate guidance from people in the data engineering industry.
Skills I currently have:
Database:
SQL Server, good with both theory and practical concepts
Studied SQL up to an advanced level and practiced on LeetCode
Data Warehouse:
Basic understanding
Built one project for practical experience
DSA:
I’ve been coding in Java and feel comfortable with it. Planning to continue DSA in Java
Would like to know how much DSA is required for Data Engineering and if there’s a specific list/topics I should cover
Cloud:
Completed Stephan Maarek’s Cloud Practitioner course. Planning to revisit and strengthen concepts again
Python:
Basic knowledge (Beginner)
Planning to start learning:
Spark
Airflow
Please also suggest good learning resources/material for Spark and Airflow.
Please let me know:
What more I need to do
What can be dropped
What type of projects I should add to strengthen my profile
Wanted to switch from my current company but not getting opportunities on LinkedIn or Naukri. Can anyone suggest me what should I do? 🥹 I have 1.5 years experience as a Azure Databricks Engineer.
📊 Analyst Community
Position: Analyst
Service line: BFSI - BFS (TGH)
Location: Gurgaon / Bengaluru
Experience: 1 to 3 years
Ideal match:
Strong problem-solving skills and analytical capability
Strong oral and written communication skills
Business research or advisory experience preferred
Graduate degree in engineering/management from a reputed institute
Position: Senior Analyst
Service line: CSA, DAAI, HLS, EIT, MEI, SVM, HRT
Location: Gurgaon / Bengaluru
Experience: 1 to 4 years
Ideal match:
Strong problem-solving skills and analytical capability
Strong oral and written communication skills
Business research or advisory experience preferred
Graduate/PG degree in engineering/management from a reputed institute
Position: Practice Director
Service line: CSA, BFSI, DAAI, HLS, HRT
Location: Gurgaon / Bengaluru
Experience: 5 to 9 years
Ideal match:
Business research or advisory experience is required
Strong accountability, stakeholder & people management
Graduate/postgraduate degree in engineering/management
💼 Sales
Position: Business Development Representative
Location: Gurgaon / Bengaluru
Experience: 1 to 4 years
Ideal match:
Experience in B2B business development/outbound prospecting
Strong communication and stakeholder engagement
Experience with CRM tools (Salesforce, LinkedIn Sales Navigator, Gong, etc.)
Comfortable with target-driven environments
Proactive, organised, adaptable
Position: Client Director - Enterprise
Location: Gurgaon / Bengaluru
Experience: 10 to 15 years
Ideal match:
Experience in consulting/research/advisory sales
Strong enterprise account management & revenue growth
Experience selling advisory/research solutions
Strong executive stakeholder management
Position: Client Director – Service Provider
Location: Gurgaon / Bengaluru
Experience: 10 to 15 years
Ideal match:
Experience in consulting/research/advisory sales
Strong enterprise account growth and client acquisition
Experience with service providers/GSIs/vendors preferred
Strong stakeholder management
Position: Business Development Director
Location: US (Remote)
Experience: 4 to 6 years
Ideal match:
Experience in B2B new logo sales
Strong pipeline generation and enterprise sales
Strong communication and consultative selling skills
Position: Client Director (Farmer-Sales)
Location: US (Remote)
Experience: 5+ years
Ideal match:
Experience in B2B sales/account management
Strong client retention and account growth experience
Strong stakeholder management
⚙️ Strategic / Corporate Enablers
Position: Senior Finance Manager
Location: Gurgaon
Experience: 8 to 12 years
Ideal match:
CA/CMA with finance/controllership experience
Strong experience in global AR/O2C processes
Experience with multi-entity/multi-currency accounting
ERP/NetSuite experience preferred
Strong analytical and stakeholder management skills
Position: Full Stack Developer
Location: Gurgaon
Experience: 5 to 12 years
Ideal match:
Experience with Python + React/Angular/Vue
Hands-on LLM integration, RAG, LangChain
Azure deployment, CI/CD, Linux
Strong database and API integration skills
Position: Data Engineer
Location: Gurgaon
Experience: 7 to 10 years
Ideal match:
Snowflake ELT & performance optimisation
SaaS API ingestion experience
Strong SQL, Python & data modelling
Experience in data quality & CI/CD practices
Position: Solution Architect
Location: Gurgaon
Experience: 12 to 15 years
Ideal match:
Enterprise architecture experience
Snowflake-based data platform design
Strong API & integration architecture
Strong stakeholder management
Position: Director - Design
Location: Gurgaon
Experience: 10 to 15 years
Ideal match:
Experience in design/creative operations
Process optimisation & workflow management
Exposure to automation/AI workflows preferred
Strong cross-functional collaboration
#hiring
For urgent hiring requirements, creating a pool of tech talent.
if you have expertise in Azure, AWS, Databricks, Sigma Computing and have 10+ years of experience. DM your resume.
key requirement:
All open positions are for EST timezone.
Experience 10+
dm only when you are open for long term commitment. (minimun 6 months)
pay is little better than market standards for contract position.
we have hiring drive this Sunday for initial round.
Hi everyone, I have roughly 10 years of experience in the industry and finally hit staff level at work, worked at a FAANG company for 3 years but the rest have been unicorns.
I want to gauge what the interest is in sharing my experience and offering services like coaching, training, career guidance, resume advice. Is there a hole in the market here? Would any of this be valuable to the community? Let me know what you think.
Looking for an experienced freelance Data Engineer for a remote engagement.
Must have strong hands-on experience in:
• DBT (very important)
• Advanced SQL (very important)
• Amazon Redshift
• AWS
• Apache Airflow
• Python
• Git
• Dimensional Modeling / Data Warehousing
• Strong business understanding
Please reach out only if you have worked extensively on DBT and SQL in production-level projects.
If interested, DM me with:
• Updated resume
• Years of experience
• Details of DBT/SQL projects you’ve worked on
Timings 7 Pm to 1:30 AM IST
Remote freelance opportunity.
Hi everyone,
I’m a software engineer based in India with around 4 years of experience, currently working at Accenture.
My primary skills are:
SQL
Power BI
Advanced Excel
Python
I recently completed my maternity break and am now looking for remote/work-from-home opportunities in India. My current compensation is around 10 LPA.
I would really appreciate guidance on
Companies in India that are actively hiring for remote data/BI/analytics roles
Which platforms or communities are currently working best for remote tech hiring
I’m mainly targeting roles like:
Data Analyst
BI Developer
Reporting Analyst
Power BI Developer
SQL/Python-based analytics roles
If anyone has recently switched or hired for similar profiles, your suggestions would help a lot.
Senior DE with about 10y of working experience, currently looking for remote jobs in Canada. How is the market right now?
Hey guys , I need guidance
I have 2.2 YOE experience working in Databricks, Python, pyspark, sql , Azure , ADF current - 5 fixed
Now I got my first offer with 16 fixed but tech stack is python, oracle sql, bash, autosys and maybe pyspark in product based firm/Bank
Shall I take this offer or not?
I'm currently exploring new Data Engineer opportunities and would really appreciate any referrals if your company is hiring!
A bit about me:
~3 years of experience as a Data Engineer (2 years 8 months)
Strong hands-on experience with Databricks, Apache Spark, and PySpark for large-scale data processing
Proficient in Python and SQL for pipeline development and data transformation
Cloud experience on Microsoft Azure (Data Factory, ADLS, Synapse, etc.)
Built RAG (Retrieval-Augmented Generation) pipelines and have foundational AI/LLM integration skills — increasingly relevant as data teams adopt AI workflows
Current role: Senior Data Analyst Interested in healthcare/insurance domains.
Any help would be appreciated. Glad if i can get some pointers for improving my resume aswell.
Hi Folks,
I recently interviewed at Everest group (a research advisory firm) as a Data Engineer having 3.5 YOE. If anyone can help me to decide whether to join or not, it would be really helpful.
The data team in the company is really small with 6-8 people only and looking to expand to a team of 13-14 people.
I am currently in a data analytics firm with a Good project.