Developer Documentation & API

BatchIn Developer Documentation

Complete docs organized around quick start, API reference, tutorials, and cookbook content.

Get API Key Explore Models

Runtime Contract & Specifications

Production posture

Built for stable production-scale text and multimodal traffic.

Capability mix

Text and multimodal traffic share one Model API contract.

Agent protocols

OpenAI-compatible access, MCP, and llms.txt are exposed together.

Traffic policy

TTFT, backpressure, slow-consumer isolation, and regional rate limits.

Public Addresses & Protocols

OpenAPI

https://api.batchin.tech/openapi.json

API base

https://api.batchin.tech/v1

MCP

https://api.batchin.tech/v1/mcp

Public endpoints

/v1/chat/completions · /v1/responses · /v1/embeddings · /v1/images · /v1/audio/* · /v1/videos

Concurrency & Billing Semantics

Global and regional endpoints serve their audiences with a unified API experience.

Scoped rate limits and timeout controls apply by key, workspace, and region with clear failure classification.

Standard enterprise traffic uses the high-performance shared core; dedicated capacity routes to isolated pools.

Human billing and agent settlement stay separated, backed by Ed25519 cryptographic receipts.

Quickstart Guide

Quick Start (3 Languages)

1) Install

pip install openai

2) First Request

from openai import OpenAI

client = OpenAI(
  base_url="https://api.batchin.tech/v1",
  api_key="YOUR_API_KEY"
)

resp = client.chat.completions.create(
  model="deepseek-v4-pro",
  messages=[{"role": "user", "content": "Hello from BatchIn"}]
)
print(resp.choices[0].message.content)

3) Streaming

stream = client.chat.completions.create(
  model="deepseek-v4-pro",
  messages=[{"role": "user", "content": "Write a fast python async worker"}],
  stream=True
)

for chunk in stream:
  delta = chunk.choices[0].delta.content
  if delta:
    print(delta, end="")

Sovereign Routing Lanes & Gateway Specification

4 Sovereign Lanes & X-BatchIn-Lane Header

Pass the X-BatchIn-Lane HTTP header with any OpenAI-compatible client, or configure default lane policies on your API keys.

💎 Pro Quality LaneOfficial FP16

Direct official priority queue, zero quantization loss, uncompromised deep reasoning IQ.

SLA99.95% Availability

TTFT280-420ms

🚀 Turbo Lane<140ms TTFT

Ultra-fast first-token response (<140ms) with 160+ tok/s high-throughput persistent multiplexing.

SLASub-140ms TTFT SLA

TTFT90-140ms

💰 Economy Batch Lane50% Discount

Up to 50% wholesale cost reduction for asynchronous, non-blocking offline batch workloads.

SLA24h Asynchronous SLA

TTFTBatch Queued

🛡️ Smart Failover Lane99.99% Uptime

Automated multi-node health circuit breaking with zero-downtime hedged failover.

SLAZero-Downtime SLA

TTFTSub-250ms Hedged

Code Sample: quality_first

X-BatchIn-Lane: quality_first

# Python (OpenAI SDK Drop-in)
from openai import OpenAI

client = OpenAI(
    base_url="https://api.batchin.tech/v1",
    api_key="your_batchin_api_key",
    default_headers={"X-BatchIn-Lane": "quality_first"}
)

response = client.chat.completions.create(
    model="deepseek-v4-pro",
    messages=[{"role": "user", "content": "Design an async lock-free queue in Rust."}],
)
print(response.choices[0].message.content)

"AI Coding IDE & Agent Integrations"

"Cursor, Cline, Windsurf & Continue Setup"

Point baseURL to BatchIn and use the live model catalog, workspace rate card, and signed receipt evidence.

Use BatchIn inside Cursor to replace expensive Claude/GPT subscriptions with ultra-fast DeepSeek V4 & Qwen 3.8 Max.

# Cursor IDE Configuration Guide:
# 1. Open Cursor Settings (Cmd+, or Ctrl+,) -> Models
# 2. Toggle "OpenAI API Key" to ON
# 3. Click "Override OpenAI Base URL" and enter:
https://api.batchin.tech/v1

# 4. In "OpenAI API Key", paste your BatchIn API Key:
bk_your_batchin_api_key

# 5. Under "Model Names", add:
deepseek-v4-pro      # 1.8T MoE for complex architecture & deep reasoning
qwen3.8-max          # 1M context for full-repo understanding
deepseek-v4-flash    # Use measured latency from your workspace telemetry

🔒 Metadata-only receipt evidence by default"Ephemeral RAM-only processing. We never log or train on your code"

SOC-2 & GDPR Ready

Platform Playbooks

Dedicated Endpoints & Concurrency Pools

See the self-serve path for dedicated endpoints, private throughput usage, billing, and settlement.

Open

Dedicated Capacity Path

Understand dedicated endpoints, private deployment boundaries, key formats, and usage separation before you start a capacity rollout.

Open

Route Control + Failover Path

Review how BatchIn positions route control, batch lanes, and fallback behavior before moving production traffic.

Open

Audit Trace Verification Path

Open the public verification flow, inspect a signed evidence pack, and reproduce the trust check in the browser.

Open

OTLP Observability Path

Review the reserved OTLP ingest/export surface and the planned bridge from native traces into external observability tools.

Open

API Reference

GET/v1/users/me

Bearer

Get current user profile for authenticated API key.

Code Samples

curl -X GET https://api.batchin.tech/v1/users/me \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \

Open Try it

POST/v1/keys

Bearer

Create a new API key with optional rate limit and monthly budget.

Code Samples

curl -X POST https://api.batchin.tech/v1/keys \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "name": "Production Key",
  "rate_limit_rpm": 120,
  "monthly_budget_cents": 500000
}'

Parameters

Name	Type	Required	Description
name	string	No	Key name
rate_limit_rpm	number	No	Requests per minute limit
monthly_budget_cents	number	No	Monthly budget in cents

Open Try it

GET/v1/keys

Bearer

List all API keys for current user.

Code Samples

curl -X GET https://api.batchin.tech/v1/keys \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \

Open Try it

DELETE/v1/keys/{id}

Bearer

Delete one API key by id.

Code Samples

curl -X DELETE https://api.batchin.tech/v1/keys/{id} \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \

Parameters

Name	Type	Required	Description
id	path:uuid	Yes	API key id

Open Try it

POST/v1/chat/completions

Bearer

Text and multi-turn chat completion with SSE streaming support.

Code Samples

curl -X POST https://api.batchin.tech/v1/chat/completions \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "qwen3.8-max",
  "messages": [{"role":"user","content":"Explain attention in 3 bullets"}],
  "temperature": 0.7,
  "stream": false
}'

Parameters

Name	Type	Required	Description
model	string	Yes	Model ID
messages	array	Yes	Chat message array
temperature	number	No	Sampling temperature
max_tokens	number	No	Max output tokens
stream	boolean	No	Enable streaming

Open Try it

POST/v1/responses

Bearer

Responses API compatibility layer with non-streaming response objects, SSE streaming events, and runtime receipt headers.

Code Samples

curl -X POST https://api.batchin.tech/v1/responses \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "qwen3.8-max",
  "input": "Explain batch inference in 3 bullets",
  "max_output_tokens": 256
}'

Parameters

Name	Type	Required	Description
model	string	Yes	Model ID
input	string\|array	Yes	Input text or message array
max_output_tokens	number	No	Max output tokens
stream	boolean	No	Return Responses events as text/event-stream.

Open Try it

POST/v1/completions

Bearer

OpenAI-compatible text completions with prefix/suffix FIM-style passthrough.

Code Samples

curl -X POST https://api.batchin.tech/v1/completions \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "qwen3.8-max",
  "prompt": "Complete this sentence: Batch processing helps",
  "suffix": "for large workloads.",
  "max_tokens": 128
}'

Parameters

Name	Type	Required	Description
model	string	Yes	Completion model id
prompt	string\|array	Yes	Prompt or prefix content
suffix	string	No	FIM suffix
max_tokens	number	No	Max completion tokens

Open Try it

POST/v1/embeddings

Bearer

Convert text into embeddings.

Code Samples

curl -X POST https://api.batchin.tech/v1/embeddings \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "bge-m3",
  "input": ["batch inference", "retrieval augmented generation"]
}'

Parameters

Name	Type	Required	Description
model	string	Yes	Embedding model id
input	string\|array	Yes	Text input to embed

Open Try it

POST/v1/audio/speech

Bearer

Convert text into synthesized speech with standard TTS-compatible speech engines.

Code Samples

curl -X POST https://api.batchin.tech/v1/audio/speech \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "tts-1",
  "input": "Welcome to BatchIn",
  "voice": "alloy",
  "response_format": "mp3"
}'

Parameters

Name	Type	Required	Description
model	string	Yes	Speech model id
input	string	Yes	Text to synthesize
voice	string	No	Voice preset
response_format	string	No	Audio response format

Open Try it

POST/v1/audio/transcriptions

Bearer

Transcribe speech or audio files into text, with compact or verbose segment-aware output.

Code Samples

curl -X POST https://api.batchin.tech/v1/audio/transcriptions \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \
  -F "file=@meeting.wav" \
  -F "model=whisper-large-v3-turbo" \
  -F "response_format=verbose_json" \
  -F "language=en"

Parameters

Name	Type	Required	Description
file	binary	Yes	Audio file upload
model	string	Yes	Transcription model id
language	string	No	Language code
prompt	string	No	Transcription prompt
response_format	string	No	Response format
temperature	number\|string	No	Sampling temperature

Open Try it

POST/v1/videos

Bearer

Submit video generation jobs through the unified Model API video entry.

Code Samples

curl -X POST https://api.batchin.tech/v1/videos \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "prompt": "Cyberpunk city night drone tracking shot, neon reflections",
  "model": "doubao-seedance-2-0-260128",
  "duration_seconds": 5,
  "resolution": "1080p"
}'

Parameters

Name	Type	Required	Description
prompt	string	Yes	Video generation prompt
model	string	No	Video model ID (defaults to doubao-seedance-2-0-260128)
duration_seconds	integer	No	Duration in seconds (default: 5)
resolution	string	No	Resolution (720p, 1080p)

Open Try it

POST/v1/images/generations

Bearer

Generate images via OpenAI Images API standard.

Code Samples

curl -X POST https://api.batchin.tech/v1/images/generations \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "prompt": "Minimalist geometric architectural rendering, dusk light",
  "model": "doubao-seedream-5-0-260128",
  "n": 1,
  "size": "1024x1024"
}'

Parameters

Name	Type	Required	Description
prompt	string	Yes	Image generation prompt
model	string	No	Image model ID
n	integer	No	Number of images
size	string	No	Resolution (1024x1024)

Open Try it

GET/v1/models

Bearer

List all active models with their status, context length, and pricing.

Code Samples

curl -X GET https://api.batchin.tech/v1/models \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \

Open Try it

POST/v1/batches

Bearer

Submit batch jobs with mixed-model tasks and priorities.

Code Samples

curl -X POST https://api.batchin.tech/v1/batches \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "priority": "fill",
  "webhook_url": "https://your-domain.com/webhooks/batchin",
  "tasks": [
    {"custom_id":"task-1","model":"qwen3.8-max","request_body":{"messages":[{"role":"user","content":"translate this"}]}},
    {"custom_id":"task-2","model":"glm-5.3","request_body":{"messages":[{"role":"user","content":"summarize this"}]}}
  ]
}'

Parameters

Name	Type	Required	Description
tasks	array	Yes	Task array (each task can set its own model)
priority	string(high\|low\|fill)	No	Scheduling priority
webhook_url	string(url)	No	Webhook callback URL

Open Try it

GET/v1/batches

Bearer

List batch jobs for current user.

Code Samples

curl -X GET https://api.batchin.tech/v1/batches \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \

Open Try it

GET/v1/batches/{id}

Bearer

Get batch detail including task results.

Code Samples

curl -X GET https://api.batchin.tech/v1/batches/{id} \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \

Parameters

Name	Type	Required	Description
id	path:uuid	Yes	Batch id

Open Try it

POST/v1/batches/{id}/cancel

Bearer

Cancel a pending or running batch job.

Code Samples

curl -X POST https://api.batchin.tech/v1/batches/{id}/cancel \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \

Parameters

Name	Type	Required	Description
id	path:uuid	Yes	Batch id

Open Try it

POST/v1/topup/alipay/checkout

Bearer

Create an Alipay checkout session for Chinese-site credit top-up.

Code Samples

curl -X POST https://api.batchin.tech/v1/topup/alipay/checkout \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "amount_cents": 5000,
  "currency": "cny",
  "locale": "zh-CN"
}'

Parameters

Name	Type	Required	Description
amount_cents	number	Yes	Top-up amount in CNY cents
locale	string	No	Chinese locale, zh-CN or zh-HK

Open Try it

POST/v1/topup/stripe/checkout

Bearer

Create Stripe checkout session for credit top-up.

Code Samples

curl -X POST https://api.batchin.tech/v1/topup/stripe/checkout \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "amount_cents": 5000
}'

Parameters

Name	Type	Required	Description
amount_cents	number	Yes	Top-up amount in cents

Open Try it

GET/v1/topup/history

Bearer

Get top-up transaction history.

Code Samples

curl -X GET https://api.batchin.tech/v1/topup/history \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \

Open Try it

GET/v1/topup/usdc/chains

Bearer

Get supported USDC chains and deposit addresses.

Code Samples

curl -X GET https://api.batchin.tech/v1/topup/usdc/chains \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \

Open Try it

GET/v1/usage/logs?limit=50&offset=0

Bearer

List request-level usage logs.

Code Samples

curl -X GET https://api.batchin.tech/v1/usage/logs?limit=50&offset=0 \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \

Parameters

Name	Type	Required	Description
limit	query:number	No	Page size (1-500)
offset	query:number	No	Offset

Open Try it

GET/v1/usage/summary

Bearer

Get total requests, tokens, and cost summary.

Code Samples

curl -X GET https://api.batchin.tech/v1/usage/summary \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \

Open Try it

GET/v1/usage/by-model

Bearer

Aggregate usage by model dimension.

Code Samples

curl -X GET https://api.batchin.tech/v1/usage/by-model \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \

Open Try it

POST/v1/vaas

Bearer

Create verifiable audit record and signature.

Code Samples

curl -X POST https://api.batchin.tech/v1/vaas \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "input_text": "What is a zero-knowledge proof?",
  "output_text": "A cryptographic method...",
  "model_id": "qwen3.8-max",
  "node_id": "execution-a19c-42ef"
}'

Parameters

Name	Type	Required	Description
input_text	string	Yes	Input text
output_text	string	Yes	Output text
model_id	string	Yes	Model id
node_id	string	No	Masked execution node identifier

Open Try it

GET/v1/vaas/{id}

Public

Fetch audit details by audit_id.

Code Samples

curl -X GET https://api.batchin.tech/v1/vaas/{id} \

Parameters

Name	Type	Required	Description
id	path:uuid	Yes	Audit id

Open Try it

GET/v1/vaas/{id}/verify

Public

Verify audit signature validity.

Code Samples

curl -X GET https://api.batchin.tech/v1/vaas/{id}/verify \

Parameters

Name	Type	Required	Description
id	path:uuid	Yes	Audit id

Open Try it

GET/v1/vaas/{id}/evidence

Public

Return the audit evidence pack used for browser verification and export workflows.

Code Samples

curl -X GET https://api.batchin.tech/v1/vaas/{id}/evidence \

Parameters

Name	Type	Required	Description
id	path:uuid	Yes	Audit id

Open Try it

GET/v1/vaas/pubkey/current

Public

Get public key used for browser-side signature verification.

Code Samples

curl -X GET https://api.batchin.tech/v1/vaas/pubkey/current \

Open Try it

GET/v1/traces?limit=20

Bearer

List native runtime traces with route, prompt-version, and audit linkage.

Code Samples

curl -X GET https://api.batchin.tech/v1/traces?limit=20 \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \

Parameters

Name	Type	Required	Description
limit	query:number	No	Page size (1-100)

Open Try it

GET/v1/traces/{trace_id}

Bearer

Fetch one native runtime trace by trace_id.

Code Samples

curl -X GET https://api.batchin.tech/v1/traces/{trace_id} \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \

Parameters

Name	Type	Required	Description
trace_id	path:string	Yes	Trace ID

Open Try it

GET/v1/prompts

Bearer

Return prompt registries with promoted versions, latest observed versions, and version details.

Code Samples

curl -X GET https://api.batchin.tech/v1/prompts \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \

Open Try it

GET/v1/prompt-regressions

Bearer

Return automatic sampled-replay regression runs with sample counts, similarity, and latency summaries.

Code Samples

curl -X GET https://api.batchin.tech/v1/prompt-regressions \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \

Open Try it

GET/v1/experiments

Bearer

Return automatic experiments for candidate prompt versions, including replay state and auto-promotion results.

Code Samples

curl -X GET https://api.batchin.tech/v1/experiments \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \

Open Try it

POST/v1/otlp/traces

Bearer

Accept OTLP/HTTP JSON or protobuf traces and merge spans into the unified trace view.

Code Samples

curl -X POST https://api.batchin.tech/v1/otlp/traces \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "resourceSpans": []
}'

Open Try it

GET/v1/otlp/export

Bearer

Export spans from the unified trace view as OTLP protobuf or JSON.

Code Samples

curl -X GET https://api.batchin.tech/v1/otlp/export \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \

Open Try it

GET/health

Public

Service health probe endpoint.

Code Samples

curl -X GET https://api.batchin.tech/health \

Open Try it

Tutorials

Batch Tutorial

Upload a JSONL input file (one task per line).
Create a batch job and store batch_id.
Poll status, then download output file.

Responses Tutorial

Send a single input to /v1/responses when you want a lighter response object instead of Chat Completions.
Inspect traceparent and X-BatchIn-* receipt headers to capture route, prompt, and run context.
Follow the trace into /console/traces and open the linked audit proof in /verify.

VaaS Tutorial

Send text to /v1/vaas.
Define policy thresholds using risk_score.
Connect final decisions to audit logs.

Cookbook

Cookbook Recipe

Batch Pipeline for 1M Documents

Split one million records into JSONL tasks, execute by priority tiers, and ingest callbacks automatically.

Try in Playground

Cookbook Recipe

Production RAG Pipeline

Combine embeddings, retrieval, reranking, and generation with observability and rollback controls.

Try in Playground

Cookbook Recipe

Audit Trace Integration

Capture verifiable audit evidence at input, output, and human-review checkpoints with Ed25519 verification.

Try in Playground