What do you do with the error rows from a nightly CSV upsert?
Partner drops a CSV on SFTP every night, ~40k rows of order line items, we upsert into a custom object on an external ID through Bulk API.
Most nights a few dozen rows fail. Usually a lookup that doesn't resolve, sometimes an external ID with trailing whitespace nobody trimmed upstream. Bulk API hands back the error file, and right now that file lands in a folder on the same box the script runs on, where it is read by nobody.
So the org ends up incomplete and there's no sign of it from the inside. A report comes back short and there's nothing in Salesforce explaining why.
Trying to work out where failed rows should actually live. Write them to a custom object so ops can fix and requeue from the UI, push a rejects file back to the partner's SFTP and make it their problem, or fail the whole load once the error rate crosses some threshold.
Anyone built the requeue-from-UI version? Wondering whether it's worth it or whether it becomes a second product nobody maintains.