Prompt Engineering Across Providers

Prompts that win on GPT-4o don't always win on Claude or Gemini. Differences that matter:

System prompt behavior

• OpenAI: system is a strong steer; later user messages can override softly. Function definitions take precedence over wording.
• Anthropic: system is treated very seriously and is harder to override mid-conversation. Use <instructions> XML tags inside system for nested constraints — Claude is XML-friendly.
• Google Gemini: systemInstruction parameter; behavior similar to OpenAI but slightly weaker steering historically.
• Llama / Qwen / Mistral: depends on chat template. Always use the official template (tokenizer.apply_chat_template) or you'll silently get junk.

Tool / function calling

• OpenAI: schema-rich, well-documented, parallel tool calls supported.
• Anthropic: similar capability via tools API; very reliable; supports tool result blocks.
• Gemini: similar; historically slightly less reliable but now competitive.
• Open-source: varies wildly. Llama 3.x has good native tool support; Qwen 3 has excellent. Always test with your real schemas.

Structured output / JSON mode

• OpenAI: response_format: { type: "json_schema", schema: ... } — strict, validated, very reliable.
• Anthropic: tool use is the canonical structured-output mechanism — define a tool whose schema is your output shape.
• Gemini: responseMimeType: "application/json" plus a schema; good support.
• Open-source: instructor-style libraries (instructor, outlines) using grammar-constrained decoding. Slower but local.

XML vs Markdown vs JSON in prompts

• Claude loves XML: <doc>, <query>, <example> — parses cleanly and steers reliably.
• OpenAI / Gemini handle Markdown well but XML is also fine.
• For mixed-output prompts, XML > Markdown > prose.

Few-shot vs zero-shot

• Modern frontier models reduce the need for many-shot. 0-2 shots is often enough.
• For structured tasks (extraction, classification with edge cases), 3-5 strong examples still helps.
• Open small models benefit from more examples than frontier closed.

Migration checklist

When swapping providers:

1. Run your eval set on the new provider with identical prompts. Note baseline.
2. Adjust system-prompt structure to provider preference (XML for Claude, JSON-mode for OpenAI).
3. Re-test tool calls — argument-shape behavior differs.
4. Check refusal patterns — different models refuse different things.
5. Tune temperature — distributions differ; what's 0.5 on one model is 0.7 on another in effect.