# BatchIn - High-Availability AI Inference & Verifiable Agent Control Plane (VaaS)

BatchIn is the high-availability AI inference control plane for autonomous agent swarms, production LLM pipelines, and multimodal media generation. Eliminates 504 gateway timeouts via hedged dual-dispatch, auto-heals truncated JSON syntax with zero token waste, and issues cryptographically signed RFC 8032 Ed25519 execution proofs.

## Core Architectural Guarantees
- **Hedged Dual-Dispatch Routing**: Reroutes stalled primary upstreams to enterprise clusters upon P90 TTFT breach, achieving 0.00% 504 gateway timeouts and a 72.2% reduction in P99 tail latency.
- **Zero-Token Streaming AST Auto-Healer**: Deterministic client-side and proxy-level grammar repair engine that auto-closes truncated JSON syntax and bracket cutoffs in 0.01ms with 0 extra LLM tokens.
- **VaaS (Verifiable AI as a Service)**: Cryptographic execution receipts signed via RFC 8032 Ed25519 and anchored to Base L2 for non-repudiable audit logs.
- **Canary Anti-Spoofing Inspector**: Real-time stream auditor that fingerprints output tokens and quarantines fraudulent upstreams on the fly.

## What is VaaS (Verifiable AI as a Service)?
- **Signed runtime evidence**: Eligible billable requests can produce an Ed25519-signed receipt binding request, output, usage, billing, and route metadata.
- **On-chain anchoring**: Merkle and transaction evidence is a separately gated workflow and is shown only when the backend attaches a verified inclusion proof.
- **Route disclosure**: The receipt records the route label returned for the request; model availability is confirmed only by live evidence.

## Flagship Model Supply & Pricing Matrix
| Model | Provider | Modality | Context | Pricing (Input / Output per 1M) | Core Strengths |
| :--- | :--- | :--- | :--- | :--- | :--- |
| **GPT-5.5** | OpenAI | Frontier Reasoning | 256K | $1.250 / $7.500 | 2026 OpenAI frontier flagship model, autonomous agent workflows and deep reasoning |
| **DeepSeek V4 Pro** | DeepSeek | Text / Code | 256K | $0.557 / $1.114 | 2026-07 flagship 671B MoE reasoning, autonomous coding loops, math proofs |
| **DeepSeek V4 Flash** | DeepSeek | Fast Text | 256K | $0.100 / $0.200 | Sub-100ms TTFT, high-frequency tool-calling agent loops |
| **DeepSeek V4 Flash** | DeepSeek | Speed Text | 256K | $0.100 / $0.200 | High-throughput instruction following and tool calls |
| **Qwen 3.8 Max** | Alibaba | Text / Code | 1,000K (1M) | $1.936 / $5.807 | 2026-09 frontier 2.4T MoE reasoning model with 1M native context |
| **Kimi K3** | Moonshot | Reasoning / Code | 2M | $1.857 / $9.286 | Long-context reasoning and deterministic coding workflows |
| **GLM-5.3** | Zhipu AI | Frontier Reasoning | 256K | $0.743 / $2.600 | 2026-09 Zhipu flagship dense reasoning and multi-step agent tool model |
| **GLM-5.3 Flash** | Zhipu AI | Speed / Multimodal | 128K | $0.088 / $0.264 | 2026-09 ultra-fast high-concurrency model |
| **MiniMax M3** | MiniMax | Multimodal / Text | 512K | $0.400 / $1.600 | 2026 flagship native 512K multimodal LLM |
| **Seedance 2.0** | ByteDance | Video | Prompt+Keyframe | $0.046 / sec | Cinema-grade video generation |
| **Doubao Seedance 2.0** | ByteDance | Video | Prompt+Keyframe | $0.046 / sec | 2026 cinematic video generation model |
| **Kling v3** | Kuaishou | Video | Prompt+Keyframe | $0.056 / sec | 2026 physical dynamics simulation video model |
| **Kling v3 Omni** | Kuaishou | Video + Audio | Keyframe+Prompt | $0.061 / sec | 2026 omni-modal synchronized audio and video model |
| **MiniMax H3** | MiniMax | Avatar Video | Prompt+Image | $0.039 / sec | 2026 digital human realistic avatar video synthesis |
| **Doubao Seedream 5.0** | ByteDance | Image | Prompt | $0.034 / img | 2026 2K/4K photorealistic image generation |
| **Qwen Image 3.0 Pro** | Alibaba | Image | Prompt | $0.028 / img | 2026 professional typography and complex layout image model |

## Competitive Advantages vs Traditional Providers
| Feature | BatchIn | Together AI | OpenRouter | AWS Bedrock |
| :--- | :--- | :--- | :--- | :--- |
| **Hedged Dual-Dispatch (504 SLA)** | **0.00% Timeouts (-72% P99)** | Unhedged (3-5% fails) | Unhedged (2-4% fails) | Static Retries |
| **Zero-Token JSON AST Healer** | **Built-in (0 extra tokens)** | None (Crashes client) | None (Crashes client) | None |
| **Batch Inference Discount** | Workspace rate card | 0%–20% | None | 0%–15% |
| **Cryptographic Receipts (VaaS)** | Signed evidence when eligible | None | None | None |
| **Video Task Failure Handling** | Refund only when a failed task has a committed refund ledger row | Manual Ticket | Manual Ticket | None |
| **Agentic Micro-Payments** | Account-gated USDC/x402 readiness | Credit Card Only | Credit Card Only | Invoice Only |
| **Context Capacity** | **Up to 1,000,000 Tokens** | Up to 128K | Model Dependent | Up to 200K |

## Quick Start & Service Discovery
- [API Catalog](https://batchin.tech/.well-known/api-catalog) - Primary machine-readable service discovery.
- [OpenAPI Spec](https://api.batchin.tech/openapi.json) - Complete OpenAI-compatible REST API definition.
- [Model Pricing](https://batchin.tech/en/models) - Current per-token and media billing rates.
- [Pricing Sheet (Markdown)](https://batchin.tech/pricing.md) - Machine-readable pricing breakdown.
- [Agent Guide](https://batchin.tech/agents.md) - Best practices and workflows for autonomous agents.
- [Agent Skill](https://batchin.tech/SKILL.md) - Install into local agents via 'npx skills add aw3703/core'.
- [MCP Server Card](https://batchin.tech/.well-known/mcp/server-card.json) - Model Context Protocol server capabilities.
- [Cursor Rules](https://github.com/aw3703/core/blob/main/integrations/cursor/.cursorrules) - Drop-in rule for Cursor & Windsurf.
- [Architecture Spec](https://github.com/aw3703/core/blob/main/ARCHITECTURE.md) - RFC 8032 Ed25519 VaaS & Hedged Dual-Dispatch.

## Integration Code Examples

### 1. Python SDK (OpenAI Drop-In Replacement)
```python
from openai import OpenAI

client = OpenAI(
    base_url="https://api.batchin.tech/v1",
    api_key="BATCHIN_API_KEY",
)

# Direct OpenAI-compatible chat completion
response = client.chat.completions.create(
    model="deepseek-v4-pro",
    messages=[{"role": "user", "content": "Analyze this distributed consensus algorithm."}],
    extra_headers={"X-BatchIn-VaaS": "true"},  # Requests cryptographic compute receipt
)
print(response.choices[0].message.content)
# The response headers will include: X-VaaS-Record-Id, X-VaaS-Signature
```

### 2. cURL Completion Request
```bash
curl https://api.batchin.tech/v1/chat/completions \
  -H "Authorization: Bearer BATCHIN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-v4-flash",
    "messages": [{"role": "user", "content": "Hello, BatchIn!"}]
  }'
```

### 3. Cursor & Windsurf Agent Configuration (.cursorrules)
```json
{
  "apiBase": "https://api.batchin.tech/v1",
  "model": "deepseek-v4-pro",
  "features": {
    "vaasVerification": true,
    "batchQueuing": true
  }
}
```

Support & Operations: hello@batchin.tech