A control layer for model routing and versioned prompt artifacts that preserves capability differences, evidence, budgets, and agent workflow boundaries.
What this pool coordinates
Model pools choose an eligible inference route for a task. Prompt pools resolve an approved, versioned instruction artifact for that task. Together they let applications and agents move quickly without copying prompts, embedding provider logic, or bypassing organizational policy. The design remains honest about model differences and records exactly what ran.
Architecture priorities
Describe modalities, tools, structured output, context class, region, data policy, latency class, and verified integration status. Preserve provider-specific extensions so normalization does not erase important features.
Clients declare hard constraints and preferences. The router filters by policy and capability, then ranks candidates using task-specific evaluation evidence, health, budget, and latency. Operators can reproduce decisions with the policy version and candidate snapshot.
Prompt versions are immutable and carry variable schemas, output contracts, owners, evaluations, and release state. Aliases such as production point to an approved version and can roll back without deleting history.
Workflow manifests reference prompts, model requirements, tools, budgets, and transition rules. Maximum iterations, elapsed time, credits, and tool calls live in trusted policy rather than model-visible text.
Reference modules
Use these modules as separate services or bounded components. Their contracts matter more than how many repositories contain them.
Verified model features and adapter compatibility.
Hard filters, preferences, evaluation scores, and failover.
Immutable versions, aliases, diffs, and approvals.
Task-specific quality, format, tool, safety, latency, and cost tests.
Step identifiers, loop limits, tools, and workflow budgets.
Prompt version, route, policy, usage, and result validation.
Conceptual request
This example is an architecture pattern, not a live PoolAPI.com endpoint. It shows explicit identifiers, units, limits, and idempotency rather than hidden defaults.
POST /model-routes
{
"task": "structured_extraction",
"prompt_alias": "invoice.production",
"requirements": {
"structured_output": true,
"region": "us"
},
"budget": {"maximum_credits": 120}
}Frequently asked questions
No. It provides a common requirement and policy envelope while preserving model- and provider-specific capabilities.
Only when the next route meets the original hard requirements and retrying will not duplicate side effects.
It enables review, evaluation, canary release, rollback, and exact attribution without redeploying every client.
Not safely. Workflow limits belong in trusted control-plane policy and must not be overridden by prompt content.
Field notes
A practical blueprint for routing across LLMs without hiding capability differences, cost controls, reliability signals, or evaluation evidence.
How to turn prompts into governed, testable, versioned artifacts that agents and applications can retrieve without copying fragile text everywhere.
How to model pooled API credits without double-spend, ambiguous balances, or retry bugs—and how to separate limits from accounting.