When the System Has No API: Webhook and File-Based Integration Patterns for Legacy and Niche Platforms

The document extraction worked perfectly. Ninety-four purchase orders came in as scanned PDFs, the model pulled vendor names, line items, quantities, and totals with better than 99% field accuracy, and the whole batch cleared validation in under four minutes. Then the project stalled for six weeks.

The reason was not the AI. The reason was that the customer's ERP, a heavily customized 2004 installation running on a server in a locked room in Ohio, had no API. No REST endpoint, no SOAP service, no SDK, no partner program. The vendor that built it went out of business in 2011. The only person who understood the schema retired in 2019 and answers email roughly once a quarter.

This is not an unusual story. It is closer to the default. Every automation project eventually meets a system that refuses to be integrated the easy way, and the difference between a project that ships and a project that dies in pilot usually comes down to how quickly the team stops waiting for an API that is never going to exist and starts designing around its absence.

The API-first assumption quietly breaks most automation projects

Modern integration thinking starts from a comfortable premise. Systems expose APIs, APIs are documented, documentation is accurate, and connecting two systems is mostly a matter of authentication and field mapping. That premise holds beautifully in a stack built entirely from software purchased after 2018.

Very few enterprises have that stack.

Walk into a mid-sized freight brokerage and you will find a transportation management system from 2009 sitting next to a modern CRM. Walk into a regional insurance carrier and you will find a policy administration system where the "integration layer" is a nightly flat file and a set of stored procedures nobody wants to touch. Walk into a hospital system and you will find HL7 interfaces that predate the people maintaining them. Walk into a county recorder's office and you will find a web portal, a fax number, and nothing else.

These systems are not going away. They hold the record of truth for billions of dollars in transactions, they pass audits, and replacing them costs more than the automation project that triggered the conversation. So the integration has to meet them where they are.

The good news is that "no API" almost never means "no integration path." It means the path is different, and it means the reliability engineering has to move from the platform into your own architecture. That trade is manageable once you name it.

First, find out what kind of "no API" you are dealing with

The phrase covers at least six distinct situations, and each one points toward a different pattern. Sorting this out in week one saves months.

Genuinely no programmatic interface. The system was built before web services were common practice. There is a UI, a database, and possibly a reporting module that exports files. This is the classic legacy case and it is the most honest one, because nobody wastes time hunting for documentation that does not exist.

Read-only API. The vendor offers a data export endpoint for reporting tools but nothing that writes back. You can pull, you cannot push. Plenty of niche vertical platforms sit here, especially in practice management, church management, dispatch, and field service software. The extraction half of your workflow is solved and the delivery half is not.

API exists but is gated commercially. The endpoints are real and documented, but access sits behind an enterprise tier, a partner agreement, or a per-call fee that makes the economics absurd at volume. The technical problem is solved and the procurement problem is not, which is a different fight and often a longer one.

API exists but is effectively unusable. No sandbox, no error documentation, undocumented required fields, a three-week ticket turnaround for credential issues, or rate limits set at a level that assumes nobody would ever send more than a few hundred records a day. Teams burn quarters here because the API technically exists, so nobody is willing to declare it dead.

Vendor policy blocks automation. Some platforms prohibit programmatic access in their terms of service, particularly in regulated categories or where the vendor sells its own integration product. This is worth checking early, because a pattern that works perfectly in testing can get an account suspended in production.

On-premise with no network path. The system has an interface, but it lives inside a network segment that security will not open. The integration problem becomes a network and governance problem wearing a technical disguise.

Only the last four are negotiable. For the first two, the design work starts immediately, and it starts with a decision about which layer of the system you are going to talk to instead.

Climbing down the ladder, one rung at a time

The core discipline is simple. Start at the top rung and move down only when the rung above is genuinely unavailable. Teams get into trouble by skipping straight to UI automation because it looks fast in a demo, and then discovering three months later that a vendor UI update broke every workflow at once.

Rung two: outbound webhooks, even when there is no inbound API

This is the most frequently missed opportunity in the whole stack. A surprising number of systems that cannot accept data can still emit it. Webhook support shows up in places where a full API does not, because the vendor built notifications for their own alerting features and exposed the configuration.

Look for anything described as notifications, callbacks, event subscriptions, alerts, or "post to URL." Look inside workflow builders, because platforms with a low-code automation module almost always include an HTTP action, and that action is a webhook whether the vendor calls it one or not. Zapier and similar connectors are another tell. If a platform has a public connector on a general automation marketplace, some interface exists underneath it, and the connector documentation often reveals the endpoint shape.

Outbound webhooks solve the trigger half of an integration cleanly. When a new claim is filed, a new invoice posts, or a document lands in a queue, the source system tells you. Your platform picks up the event, retrieves and processes the document, extracts the fields, and then faces the delivery problem separately. Splitting trigger from delivery is useful even when both are hard, because the two halves fail for different reasons and should be monitored separately.

Building the receiving end properly matters more than it looks. A webhook receiver that works in a demo and a webhook receiver that survives production are different pieces of software.

Verify signatures on every inbound call, because an unauthenticated endpoint that writes into a processing pipeline is an open door. Most platforms sign payloads with an HMAC using a shared secret, and checking that signature costs almost nothing. Respond fast, ideally under a second, by acknowledging receipt and queuing the work rather than processing inline. Senders time out, and a timeout usually triggers a retry, which means slow processing quietly turns into duplicate processing.

Assume duplicates anyway. Webhook delivery is at-least-once in practice regardless of what the documentation promises. Every event needs an idempotency key, usually the event ID from the source, stored and checked before work begins. Handle out-of-order arrival too, because an update event can beat the create event that preceded it, and a pipeline that assumes ordering will corrupt records in ways that are painful to unwind.

Log the raw payload before parsing it. When a vendor silently changes a field name in a minor release, the raw log is the only evidence of what actually arrived, and the difference between a two-hour fix and a two-day investigation is whether that log exists.

Rung three: file-based exchange, the pattern that never dies

File transfer is the oldest integration pattern still in daily use and it remains the most reliable answer for legacy systems. Almost every enterprise platform built in the last forty years can write a file or read one. That universality is exactly why it survives.

The pattern takes several shapes. Batch export and import runs on a schedule, with the source system dropping a CSV or fixed-width file to an SFTP location and the destination picking it up on a defined window. Watched folders trigger processing the moment a file appears, which gets close to real time without requiring either side to speak HTTP. Scheduled report generation covers systems that can email or export reports but cannot write to a share, and the report itself becomes the data feed. Print-to-file capture, where a system's print output is redirected to a spool directory and parsed, sounds crude but works well for green-screen applications with no other export path.

For a document processing workflow, this usually runs in both directions. Scanned documents arrive in a watched folder, get classified and extracted, and the structured output is written back as a file the destination system already knows how to ingest. The legacy platform never learns that AI touched anything. It sees the same nightly import file it has been consuming since 2007, which is precisely the point, because that file format already passed whatever validation and audit process the organization built around it.

Getting file-based integration right depends on a handful of details that separate a pipeline that runs for years from one that breaks the first busy Monday.

Write atomically. Never write directly to the filename the consumer is watching. Write to a temporary name, close the handle, then rename. Rename is atomic on most filesystems, so the consumer either sees no file or sees a complete one. Half-read files are the single most common failure in watched-folder architectures and this one habit eliminates the entire class.

Use manifests for multi-file batches. When a batch spans several files, write a manifest last that lists filenames, record counts, and checksums. The consumer waits for the manifest before starting. This also gives you a control total, which is how partial transfers get caught the same day instead of during a quarterly reconciliation.

Name files so a human can debug them at 2am. A convention like source_documenttype_YYYYMMDDHHMMSS_sequence.csv costs nothing to adopt and answers most support questions without opening a log. Sequence numbers matter, because gaps in a sequence are the cheapest possible detection mechanism for a missed batch.

Pin down encoding and delimiters before the first test. UTF-8 with BOM versus without, CRLF versus LF, comma versus pipe, quoting rules for embedded delimiters. Legacy systems tend to be strict and silent about all of these. A vendor address containing a comma has broken more integrations than any algorithm ever will.

Move processed files immediately. Successful files go to an archive directory partitioned by date. Failed files go to a quarantine directory with an error file beside them explaining what went wrong. A source directory that fills up with thousands of processed files becomes slow to scan and impossible to reason about.

Decide what an empty file means. A zero-record file and no file at all are different signals, and they need different alarms. Silence is ambiguous. Silence that lasts a week is a data loss incident that nobody noticed.

Rung four: direct database access

When a system has no API and no export path worth using, the database underneath it sometimes becomes the interface. This works, and it carries real risk, so it needs guardrails.

Read access to a replica is the safe version, and it is often enough. A read-only replica lets you pull master data, validate extracted values against real customer or vendor records, and reconcile what you sent against what actually landed. It cannot break the source system, which makes the security conversation far shorter.

Writing directly into an application database is a different matter. Application logic lives in the application, not the schema. Business rules, audit triggers, derived fields, and cache invalidation all sit above the tables. Insert a row directly and you may produce a record that looks correct in the database and behaves incorrectly in the UI, or one that never appears at all. If direct writes are unavoidable, write into a staging table the application already reads from, or into a table you control that a scheduled job then processes through the application's own import routine. Keep your writes on the outside of the application's logic, never in the middle of it.

Schema drift is the long-term risk. Vendor updates change columns without notice because nobody is supposed to be reading them. Version-check on every run, fail loudly on unexpected schema, and never let a silent column rename turn into months of quietly wrong data.

Rung five: UI automation, the last resort that sometimes wins

Screen-level automation, whether classic RPA or a browser agent, is the bottom rung for good reasons. It is brittle against interface changes, slow relative to every other option, hard to scale, and frequently blocked by session timeouts, multi-factor prompts, and bot detection.

It still earns its place in specific situations. Government portals with no other submission path. Vendor systems where a partner agreement is years away. Low-volume, high-value processes where twenty submissions a day through a UI beats a six-month integration project. Bridge periods where a real integration is funded but not yet built and the business cannot wait.

If it is the right call, treat it as production software rather than a recorded macro. Target elements by stable identifiers instead of screen coordinates. Build explicit waits rather than fixed sleeps. Capture a screenshot at every failure, because the screenshot is usually the entire diagnosis. Run against a test account first, every time. And keep a written record of every action taken, because "the robot did it" is not an acceptable answer to an auditor.

The reliability layer is where these integrations live or die

Every rung below the first shifts responsibility onto you. A well-built API gives you delivery guarantees, error semantics, and retry behavior. A watched folder gives you a directory. The difference has to be made up in architecture.

Four things carry most of the weight.

Idempotency everywhere. Every record needs a stable business key, something like invoice number plus vendor ID plus amount, hashed and stored. Before writing anything downstream, check the key. This single mechanism protects against duplicate webhooks, replayed files, and the operator who reruns yesterday's batch because they were not sure it completed.

Retry with backoff and a dead-letter path. Transient failures deserve retries with exponential spacing. Permanent failures deserve a queue where a human can see them. The mistake is treating both the same way, which produces either infinite retry loops on records that will never succeed, or immediate abandonment of records that would have gone through on the second attempt. Every dead-lettered record needs an owner and a review cadence, otherwise the dead-letter queue becomes a place where data goes to be forgotten.

Reconciliation as a scheduled job, not a quarterly panic. Count what you sent. Count what arrived. Compare daily. For file-based flows this can be as simple as comparing manifest totals against a destination-side count query. The discrepancy report is boring on almost every run, and on the run where it is not boring, it saves the quarter.

Monitor for silence, not just for errors. The most dangerous integration failure is the one that produces no error at all. A scheduled export that stops running produces nothing to alert on. Staleness alarms fix this. If the expected file has not arrived within its window, page someone. If yesterday's volume dropped more than fifty percent against a trailing average, flag it. Absence of data is a signal, and it needs to be treated as one.

Security and governance do not get a pass because the pattern is old

File-based and database-level integration tends to slip past the review that an API integration would receive, largely because it looks like plumbing rather than software. That is backwards. These patterns move the same regulated data with fewer built-in controls.

Flat files sitting on an SFTP server frequently contain names, account numbers, medical details, and financial records in plain text. Encryption at rest on the landing zone, key-based authentication rather than passwords, IP allowlisting, and a retention policy that actually deletes archived files should all be non-negotiable. Directory permissions deserve real attention, because the account that writes files rarely needs to read the whole archive, and the account that reads rarely needs delete rights.

Audit trails matter just as much. When data moves through a file drop, the chain of custody has to be reconstructable. Which file, written when, by which process, containing which records, processed at what time, with what outcome. Regulated industries will ask for this, and the answer needs to come from a log rather than from memory. This is also where field-level lineage earns its cost, because tracing a single wrong value back to the source document page it came from turns a compliance conversation into a five-minute lookup.

Credential rotation is the quiet one. SFTP keys and service accounts for legacy systems tend to be created once and never touched, often tied to a named employee who left two years ago. Inventory them, rotate them, and move them to service identities.

What this looks like in a document workflow

For a platform like Artificio, most of this lands on the delivery side rather than the extraction side. Reading a scanned bill of lading, an ACORD form, or a batch record is a solved problem. Getting the structured output into the system of record, when that system is a decade past its last integration update, is where the engineering time actually goes.

The practical approach is to treat the output format as configurable rather than fixed. The same extraction result should be able to leave as a JSON payload to a modern endpoint, a pipe-delimited flat file matching a legacy import spec, an SFTP drop on a nightly window, a database staging insert, or an email with a structured attachment. Nothing about the model changes. Only the last mile does.

That flexibility is what lets a deployment move forward while the API conversation with the vendor continues in parallel. Ship the file-based path in week three, run the business on it, and swap in the API later if it ever arrives. Plenty of those file-based paths are still running years later, quietly, because they work.

The customer with the 2004 ERP in Ohio ended up on a watched-folder pattern with a manifest file and a nightly reconciliation job. Documents come in, extraction runs, a fixed-width file lands in the directory the ERP has been importing from since before the automation project existed. The integration has run for eighteen months. It has never been down for more than one batch window.

No API required.

Where to start on your own stack

Take the three systems that would benefit most from automation and find out which category each one falls into before writing any code. Check for outbound webhooks hiding in notification settings. Check whether an existing export or report can become a feed. Ask what nightly jobs already move files, because an organization that has been doing file-based integration for years usually has infrastructure and conventions worth reusing.

Then pick the highest rung that is genuinely available and design the reliability layer as part of the build rather than as a follow-up. Idempotency keys, dead-letter handling, daily reconciliation, and staleness alarms are not enhancements. On any rung below the first, they are the integration.

The systems that resist integration are usually the ones holding the most valuable data, which is why they are still running. Meeting them on their own terms is not a compromise. It is the job.