r/ETL

Hey everyone,

I'm evaluating data integration tools heading into Q3 2026 and would love to hear what's actually working for people right now. The landscape has shifted a lot in the last year or two (more reverse ETL, more zero-copy/data sharing, AI-assisted pipelines, etc.) and I want to cut through the marketing.

A few things I'd love your input on:

- What tool(s) are you using and roughly what's your stack/scale?

- What do you love about it?

- What are the gotchas or things you wish you'd known before adopting it?

- Anything you've migrated away from and why?

Open to hearing about Scaylor, Fivetran, Airbyte, Estuary, Hevo, Matillion, dbt + custom, Meltano, or anything else I'm not thinking of.

Thanks in advance!

Im curious about the practical strategies used in production ETL systems when source tables or API structures change unexpectedly.

HI,

I am looking for an on premise ETL tool primarily to handle Transforming and loading data. And possibly something that can be automated/schedule to execute Stored Procedures and queries.

We don't need cloud storage or reporting, that is done through Microsoft Fabric and PowerBI.

(current fabric licenses are allocated through our parent company, and I can not use them - Some weird "separation of entity" legal red tape as they are based outside of the US.)

Data Sources: .CSV files and SalesForce.

Destination: SQL server and if possible, a push back to Salesforce.

We have a very small budget of 10K annually. Total of 2 users.

Any recommendations would be helpful. (SSIS isn't possible, since we use an Azure SQL and thus can't bill it under the parent companies Microsoft licenses).

Been deep into Base44 lately and honestly enjoying the whole vibe coding/building space.

I’ve been working on platforms involving job boards, CRM-style workflows, automations, dashboards, onboarding systems, AI-assisted forms, candidate management, subscriptions, and internal admin tools — mostly focused on making things practical and usable instead of overcomplicated.

Still learning new things every day, but if anyone here is building something on Base44 (or thinking about it) and wants another brain to bounce ideas with, troubleshoot workflows, improve UI/UX, or help structure systems, feel free to reach out.

Happy to connect with fellow builders and those curious to develop their own getting little help in the space 👌

I’m curious about the practical approaches used in production ETL systems to detect bad or inconsistent data before it impacts downstream analytics.

Has anyone had an issue when trying fetch 20k+ records from BiqQuery to Postgres DB? Everything works fine if I keep it under 10k, using Table Input + SQL, but as soon as I try more records the pipeline fails. Odd Java error message. Ultimately, I am looking to move like 500k records from BQ to Postgres DB.

Most pipelines break because an upstream source changed without warning. I built a platform to catch these issues before they crash your ETL.

What it does:

• Schema Monitoring: Detects renamed columns, dropped fields, or type changes in real-time.
• Uptime Checks: Verifies your APIs and Databases are online before the pipeline runs.
• Instant Alerts: Notifies you the moment drift is detected or any problem with the source.
• Simple Setup: Connect your SQL DBs or REST APIs in under 2 minutes.

Would you use it and what features would make this a "must-have" for your workflow? Thanks!