Most enterprise AI lives next to systems older than the engineers running them. Patterns that work:
Integration patterns
1. REST API in front of AI
- AI service exposes REST; legacy calls it.
- Synchronous: simple but ties latency.
- Async: legacy submits job, polls or listens for callback.
- Use OpenAPI for contract. Generate clients.
2. Message queue (IBM MQ, Kafka, RabbitMQ)
- Legacy puts request on queue → AI worker consumes → response on response queue.
- Decouples latency, smooths load, gives natural retry semantics.
- IBM MQ in particular: prevalent in finance/insurance; understand its transaction semantics (XA, units of work).
3. ETL augmentation
- Batch job: nightly, AI processes records in bulk, writes back annotations / embeddings / summaries.
- Cheapest pattern; doesn't add real-time latency.
- Good for: classification, enrichment, summarization at scale.
4. iPaaS / API gateway
- Apigee, MuleSoft, IBM API Connect mediate between legacy and AI services.
- Centralized auth, rate limiting, observability.
Async pattern in detail
Legacy → enqueue request → AI worker → embed → retrieve → LLM → enqueue response
← poll/webhook ←
- Use idempotency keys so retries don't double-process.
- Dead-letter queue for failed messages with retries exhausted.
- Circuit breaker in legacy for AI-service downtime.
Specific challenges
Schema rigidity
Legacy expects { "amount": "DECIMAL(10,2)" }. AI returns "twenty-five dollars". Always validate AI outputs against the legacy schema before handing back; reject + retry on mismatch.
Encoding
Legacy systems often use EBCDIC, ISO-8859-1, or Windows-1256 (common for Arabic). Normalize at boundary, not inside AI logic.
Transaction boundaries
AI calls aren't transactional. If your business needs atomicity (debit + credit + log + AI annotation), the AI step is best outside the DB transaction.
Audit trail
Legacy may have its own audit. AI service must log enough to reconstruct decisions for compliance.
A realistic IBM enterprise pattern
Mainframe (CICS / DB2)
↓ (IBM MQ)
Integration layer (Cloud Pak for Integration)
↓
AI Service (watsonx.ai with Granite, RAG over watsonx.data)
↓ (response queue)
Mainframe
The AI service treats MQ as both input and output transport. Audit goes to a separate stream into watsonx.governance.
What to avoid
- Synchronous chains over 5s. Legacy times out. Async or break the work up.
- Pushing model output directly into legacy without validation. Schema drift = downstream breakage.
- Mixing AI failure modes with legacy failure modes. Surface AI errors distinctly so SREs can triage.