Building a Bank Payment File Pipeline Where "No File" Is an Answer

At a Canadian fintech I worked on the path that takes a daily batch of loan funding payments and gets it to the bank. The payments go out as a CPA005 file, the fixed-width EFT format Canadian banks use for batch credits and debits.

The loan-servicing platform we use generates the funding batch once per business day, in the late afternoon Eastern time. Our service's job is everything after that: pick the file up, check it, fix what needs fixing, deliver it, and make sure someone knows when any of that doesn't happen.

Here's how it fits together, and what I learned building it.

The pipeline

The flow is four steps:

1. Retrieve. Pick up the day's file once the platform's batch has run.
2. QC. Validate the file's structure and contents. A file that fails is not uploaded to the bank, and it raises an alert to the operations and tech teams.
3. Re-sequence. Write the correct file creation number into the file.
4. Deliver. Upload the QC'd file to a dedicated SFTP directory that a downstream job syncs onward.

The pipeline records file events in an audit table, so for any day you can see what arrived, what passed, and where it went.

QC: what a fixed-width file can get wrong

CPA005 is fixed-width. Every line is exactly 1,464 characters, and every field lives at a fixed offset. There's an A header record, C or D detail records for credits and debits, and a Z trailer.

The QC checklist we built against covered:

• Structure. Header, details and trailer present. Every line exactly 1,464 characters, so no shifted columns. Plain ASCII.
• Header. Originator ID, creation date and destination institution correct. File sequence number unique.
• Detail records. Institution number is 3 digits, transit is 5 digits, account number present and a valid length, transaction code valid, amount numeric and greater than zero, no invalid characters in name or reference fields.
• Totals. The trailer's record count and total amount match what's actually in the detail records.
• Duplicates. No repeated transactions, and the file itself hasn't been sent before.
• Dates. Effective date isn't a weekend or holiday, and isn't in the past.

Most of these are simple string slicing and arithmetic. The value isn't in any one check. It's that a bad file stops before it reaches the bank instead of after.

Re-sequencing: read the real files first

The bank expects a file creation number that increments with each file: 0001, 0002, 0003. It sits at positions 21 to 24 of the header, right after the record type, record count and originator ID.

The ticket described it as a header field. When we looked at real sample files, the same four digits repeated at positions 21 to 24 on every detail record and on the trailer too. Changing only the header would have produced a file that disagreed with itself, which the bank could reject.

So re-sequencing overwrites that slice on every record:

const LINE_LENGTH = 1464;

export function resequence(lines: string[], fileNumber: number): string[] {
  const value = String(fileNumber).padStart(4, '0');
  return lines.map((line) => {
    if (line.length !== LINE_LENGTH) {
      throw new Error(`Expected ${LINE_LENGTH} chars, got ${line.length}`);
    }
    return line.slice(0, 20) + value + line.slice(24);
  });
}

Before calling it done, I validated the sequencing with the business owner against real files rather than signing off on my own reading. That conversation also settled two operational details: the counter sits at 0000 before launch, and there's a reset path for go-live.

Delivery: keep directions separate

The QC'd file goes to its own SFTP directory. We couldn't reuse the existing one, because that directory faces the servicing platform. Keeping inbound and outbound apart means nothing downstream can mistake an unvalidated file for a validated one.

There's also a small admin page to re-trigger the scan by hand. During QA I checked that running it again on the same day doesn't produce a duplicate file. If you give people a "run it again" button, re-running has to be safe.

The hard part: silent failure

This is the problem that took the most thought.

If the platform generates a file every business day, a missing file is an obvious alarm. But it doesn't. On a day with no funding transactions, there's no file at all. So at the cutoff, an empty directory could mean two things:

• Nothing needed to be funded today. Fine.
• The batch process broke and nobody knows. Not fine.

The usual answer is a heartbeat: have the producer send an empty file, or a completion signal, every day regardless. We asked. The platform couldn't send a blank file or heartbeat, and suggested we track transactions on our side and alert at cutoff if the two didn't line up.

So that's what the detector does. At the cutoff, it asks: did the platform record any funding transactions today, and did we retrieve a file today?

Transactions today	File retrieved	Meaning
No	No	Quiet day. OK.
Yes	Yes	Normal day. OK.
No	Yes	Shouldn't happen. Log a warning.
Yes	No	Silent failure. Alert.

A few details mattered:

• "Today" means today in Toronto. The cutoff and the date both use America/Toronto, not UTC, so a late-evening run doesn't land on the wrong calendar day.
• It's idempotent. Running the detector twice on the same day doesn't send two alerts.
• Each of the four states has a unit test. The detector is small, and the cost of it being wrong is a funding day nobody notices.

The alerts go out by Slack and email. That service had neither when we started, so it got a small notifications module. Alert delivery is deliberately simple and goes to two channels, so one failing doesn't silence the other.

It worked the way it should

Later, someone asked why there was no funding file that day. The answer was quick: no file had been generated, the system had said so, and that was the expected outcome. That's what this kind of work is for. Not drama, just a fast, confident answer.

Takeaways

• Validate before delivery, not after. A QC step that blocks a bad file is worth more than any alert about one.
• Check the spec against real files. The "header-only" field was on every record.
• Make re-runs safe before you give anyone a re-run button.
• Absence is ambiguous. If "no output" is a valid outcome, you need a second source of truth to tell a quiet day from a broken one.
• Pin your dates to a time zone. Batch systems run on business days in a specific place, not on UTC.