The Inbound EDI Pipeline

Updated Jul 25, 2026

EDI

The Inbound EDI Pipeline

Every inbound document goes through the same stages in the same order, whether it arrived over the hosted file gateway, was pulled from a Plex mailbox, was fetched by a retrieval script, or was replayed by hand from the console.

parse → resolve partner → duplicate check → resolve spec → map
      → conformance → routing decision → record → acknowledge → turnaround

1. Parse

The file is split into interchanges, groups and transaction sets. Delimiters are read from the ISA/UNA header rather than assumed. Composite elements are split; EDIFACT release characters are unescaped. A file can contain several interchanges, and each transaction set is handled on its own.

When it goes wrong: nothing is recorded, because there was no document to record — the call fails with a parse error naming the problem. Almost always this is a truncated file or a mangled ISA header. Check the raw bytes; the file is still in the gateway transfer or the retrieval log.

2. Resolve the trading partner

The interchange sender qualifier and ID are looked up against your partner registry.

When it goes wrong: the document is recorded with status review, partner —, and the note "no trading partner matched the interchange sender". Nothing is lost. Create the partner with the exact values from the raw payload and replay.

3. Duplicate check

This runs before mapping and before routing, because a re-delivery must not re-trigger anything.

Each transaction gets a fingerprint: a hash over its transaction set id and its own segments, canonicalised so the digest does not depend on delimiters. The fingerprint deliberately excludes the envelope and all control numbers — partners re-envelope a re-send with fresh ISA13/GS06/ST02 values, so hashing those would mean the check never fired on the exact case it exists for.

A hit is recorded with status duplicate and a note naming the original transaction, and nothing downstream runs. A third delivery still points at the first, not at the second.

Partners re-send an entire file whenever they miss an acknowledgement. This check is what stops a purchase order being processed twice, and for Plex mailbox pickup it is load-bearing rather than a safety net.

When it goes wrong: if a genuinely new document is flagged duplicate, the partner re-sent identical content with only the envelope changed. Confirm against the raw payloads of both rows, then replay the newer one if you do want it processed — replay deliberately runs without the duplicate check.

4. Resolve the mapping spec

The partner's routing map is consulted for this transaction set and direction. The row names a spec.

When it goes wrong: no row for that document gives "partner X has no route for 856 inbound"; a row naming a spec that does not exist or is disabled gives "spec … not found". Both land in review. Add the row or fix the spec, then replay.

5. Map

The spec is applied to that one transaction, producing the business object. A file containing three 850s maps them separately, so one bad order does not spoil the other two.

When it goes wrong: "mapping failed: …" and review. This is a spec problem, not a partner problem — open the spec, paste the raw payload from the console into Test mapping, and fix the paths.

6. Conformance

Envelope validation and content conformance are two different questions, and DataMagik keeps them apart:

Check	Asks
Valid (envelope)	Do the segment counts and control numbers add up?
Conformant (content)	Did the partner actually send the segments and elements this document type requires — and did the spec extract the fields the downstream system needs?

An 850 with no PO1 lines, or a BEG with no PO number, is arithmetically perfect and business-useless. Built-in rules ship for X12 850, 855, 856, 810, 820, 830, 846, 852, 860, 862, 865, 875, 940, 944, 945, 997 and 999, and for EDIFACT ORDERS, ORDRSP, DESADV, DELFOR, DELJIT, INVOIC, RECADV and CONTRL. They are deliberately conservative — only what a partner is genuinely obliged to send — because an over-strict rule manufactures exceptions out of good documents.

Two flags are recorded, not one: conformant and conformance checked. An unknown transaction set has no rules, and "we did not check" must never read as "it conforms".

A partner who legitimately differs can have the check switched off by unticking Check content conformance on their routing row.

7. Routing decision

If everything above passed, the row's configured action applies — review, Plex, or script. If anything failed, the action is downgraded to review regardless of what was configured.

Why downgrade rather than pass it through. The configured action exists for documents you understand. A document that did not match a partner, did not map, or does not conform is one you do not understand yet — and the cost of sending a half-understood order downstream is a wrong shipment, while the cost of downgrading it is somebody reading it. The acknowledgement agrees with the decision: a document sent to review on content grounds is rejected in the 997, not accepted. An acknowledgement that says something different from what you did is the worst thing an acknowledgement can say.

8. Record

A row is written for every transaction — including duplicates and failures — carrying the mapped object, the issues, the raw payload and the fingerprint. If the status is review, an approval task is raised for the partner's configured exception approver.

9. Acknowledge

With Send acknowledgements ticked on the partner's routing row, DataMagik reserves a real control number from the partner's counter, rebuilds the acknowledgement with it and with any content rejections, delivers it over the partner's transport, and logs it as an outbound transaction. See Acknowledgements.

An inbound acknowledgement is never itself acknowledged — that is how you build an infinite loop with a trading partner.

10. Turnaround

A routing row can answer an inbound document with an outbound one automatically — the usual case being an 850 in, an 855 back. Put the outbound transaction set in Turn around as on the partner's routing row. Defaults for the fields you are answering with are set on the spec:

{ "turnaroundDefaults": { "purpose": "00", "ackType": "AC" } }

The inbound mapped data is re-keyed into the outbound spec's shape and the defaults applied. The defaults deliberately override what the inbound document carried: the fields a turnaround configures are the ones you are answering with, and echoing the partner's own purpose code back defeats the point.

Only a cleanly routed document turns around. A review, a duplicate or a failed envelope has not been accepted, and answering it with an 855 would tell the partner you accepted an order you did not.

What a replay does and does not do

Replay re-runs the whole decision — partner, spec, mapping, conformance, routing — against your current configuration, and records the fresh attempt. It deliberately transmits nothing: no acknowledgement, no turnaround, and no control number is consumed. Fixing a spec should not flood a partner with duplicate 997s and 855s, and reserving a control number on a replay would leave a permanent gap in a sequence partners check.

Next: The EDI Console is where you watch all of this and act on it.