Deploy a model. Use it. Stop when you're done.

One dedicated key for your application

Each optimized endpoint can have one dedicated key (vbs_…). Embed it in your application: the same key starts the endpoint, runs inference and stops it, and the URL and key stay the same every time. For example, your application calls Start when a user signs in, sends requests while they work and calls Stop when they are done. The dedicated key cannot change limits, price or keys, delete the endpoint or launch anything else. Dedicated keys are for optimized endpoints; for a custom model, contact us.

1. Get the key once

Create it from an optimized endpoint on the Endpoints page, or with your platform key. GET /console/api/services returns capabilities.canCreate. Choose startup behavior before quoting: for private_only quote rapid_coldboot: false; for qualified_bridge quote rapid_coldboot: true and approve the returned startup provider and capabilities.

POST /console/api/deployments/plan
Authorization: Bearer $INFEROPTIMAL_API_KEY
Content-Type: application/json

{
  "managed_endpoint": "OFFERING_FROM_CATALOG",
  "rapid_coldboot": false,
  "max_runtime_hours": 2,
  "max_spend_usd": 21.60
}

Review the price, startup behavior and blockers, then create the dedicated key with that plan and the same selector. Startup and spending options belong in policy, not selector.

POST /console/api/services
Authorization: Bearer $INFEROPTIMAL_API_KEY
Content-Type: application/json

{
  "name": "My application",
  "plan_id": "plan_FROM_QUOTE",
  "selector": {"managed_endpoint": "OFFERING_FROM_CATALOG"},
  "idempotency_key": "persisted-service-creation-id",
  "policy": {
    "startup": "private_only",
    "maxHourlyUsd": 10.80,
    "maxSpendUsd": 21.60,
    "totalBudgetUsd": 100,
    "maxRuntimeHours": 2,
    "maxConcurrentRequests": 4,
    "idleSeconds": null
  }
}

Use your quoted selector and price; these example amounts are not an offer. The example chooses four simultaneous requests, not a platform default. Prefer the quote’s measured usage.concurrency.agents unless you need fewer, within runtime.execution.maxConcurrentRequests or usage.concurrency.at_most. An engine limit alone is not a measured speed guarantee. Save the returned service_url, base_url, model and api_key in your application's secrets. The key is returned at creation and on identical creation retries for ten minutes; it is not included in listing or inspection. Creation starts no hardware. An endpoint is never stopped for idling by default: omit idleSeconds or send null, and its owner gets one warning email after 30 idle minutes. To opt in to stopping on idle, set a whole number of seconds from 120. Idle time is billed at the endpoint’s rate, and active requests reset the idle clock.

2. Start

Read the endpoint's state with its dedicated key, then start it with the generation and admission.epoch you read.

curl "https://infraoptimal.com/services/svc_YOUR_ENDPOINT" \
  -H "Authorization: Bearer $INFEROPTIMAL_DEDICATED_KEY" \
  --fail-with-body --output service-status.json

It returns state (stopped, starting, ready, stopping or attention_required), generation, admission.epoch, accepts_requests, idle_seconds, idle (its summary says how long it has been idle and its price, in the offering’s unit, or null while serving or stopped) and the URLs. readiness.inference_ready means it can answer; readiness.private_ready means a private replica is ready. readiness.inference_backend identifies private, startup_bridge, mixed, unknown or null when not ready. Connection and agent setup responses include the same summary.

Prepare a Start body once from that saved status (Node 20+). If the request loses its response, resend the saved file; do not regenerate it.

node --input-type=module <<'JS'
import { readFile, writeFile } from 'node:fs/promises';
const status = JSON.parse(await readFile('service-status.json', 'utf8'));
const intent = {
  idempotency_key: crypto.randomUUID(),
  expected_generation: status.generation,
  expected_epoch: status.admission.epoch,
  expires_at: Date.now() + 5 * 60 * 1000
};
await writeFile('service-start.json', JSON.stringify(intent), { mode: 0o600 });
JS

curl -X POST "https://infraoptimal.com/services/svc_YOUR_ENDPOINT/start" \
  -H "Authorization: Bearer $INFEROPTIMAL_DEDICATED_KEY" \
  -H "Content-Type: application/json" \
  --data-binary @service-start.json

expires_at is Unix milliseconds, at most five minutes ahead. Save the whole body before sending and retry it unchanged after a lost response. Start is idempotent and deduplicated: a retried body and simultaneous Starts share one allocation. A Start prepared before a Stop returns 409 service_start_stale; read the state again. Right after a Stop, Start can return 409 service_cleanup_pending with Retry-After while the previous hardware is released. Start keeps the endpoint's hourly ceiling, per-start cap, total authorization, daily start limit and your balance hold, and returns 402 or 429 when one is reached. When state is attention_required, Start with the key returns the reason; fix it, then press Start in the console. Private prewarm is complete when an unstopped replica has phase: "ready" and inference_backend: "private"; state: "ready" alone can mean startup-bridge availability.

3. Send requests

curl "https://infraoptimal.com/services/svc_YOUR_ENDPOINT/v1/chat/completions" \
  -H "Authorization: Bearer $INFEROPTIMAL_DEDICATED_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"svc_YOUR_ENDPOINT","messages":[{"role":"user","content":"Hello"}]}'

Configure your client with base_url, the endpoint ID as model, and an advertised API mode. Anthropic clients such as Claude Code can also call /messages on a chat endpoint: set ANTHROPIC_BASE_URL to the endpoint URL without /v1 (https://infraoptimal.com/services/svc_YOUR_ENDPOINT) and ANTHROPIC_AUTH_TOKEN to the dedicated key. Those requests run as Chat Completions, with the same Start, limits and billing.

Supported api_modes identify openai_chat_completions → /chat/completions, openai_responses → /responses, or /embeddings for synchronous embedding calls. For native Responses during startup, also require the quote’s preparation.startup_capabilities.api_modes to include openai_responses; otherwise wait for private readiness. This release uses one private replica; embedding services admit one active request, which may contain a batch. Asynchronous media jobs, private repository credentials and custom containers use the explicit deployment API.

Keep the inference HTTP connection open: validated requests wait in order for startup or an available concurrency slot. Waiting and inference share policy.requestTimeoutSeconds (default 1800, maximum 3600); X-InferOptimal-Request-Timeout-Ms can set a shorter total deadline. There are 256 waiting places by default, increasing to four times the concurrency limit with a ceiling of 1024. A full waiting queue returns 429 service_queue_full with Retry-After: 5; expiration before dispatch returns 504 service_request_timeout. Eligible private services stream uploads through the gateway and validate near inference, with a current 32 MiB body limit. Startup bridges and older runtimes use the buffered 8 MiB path; concurrent input validation may wait or return 429 service_payload_capacity. Admission order begins after validation or streaming admission; the oldest requests can fill available slots in parallel. The queue stores metadata only: a disconnected HTTP request is not a saved job. Your client or network may have an earlier timeout. No inference is automatically repeated after execution may have begun. A non-streaming Chat Completions or Responses call is not cut off at a proxy timeout: the gateway answers 200 at once, sends spaces while the model works, then the usual JSON answer (leading whitespace is valid JSON, and the OpenAI SDKs read it). If the model does not start answering within 120 seconds the request returns 504 upstream_timeout with no Retry-After, because it may already have run. On a managed session (/optimized/v1) a non-streaming response must still finish within about 100 seconds; send long generations there with stream: true. Whether an endpoint does this is its connection's non_streaming_carried; where it is false, a non-streaming answer is still cut after about 100 seconds, so use stream: true for long generations. If an approved startup provider explicitly refuses a request before execution, the same connection can keep waiting for its private GPU within the original deadline.

A dedicated key does not skip the cold start. Choose qualified_bridge at creation only after approving a quote that offers startup inference and its provider/data handling; no fallback model is selected silently. The serviceChat helper in inferoptimal.mjs also understands explicit retry-safe 503 service_starting responses, without retrying ambiguous inference failures.

4. Stop

curl -X POST "https://infraoptimal.com/services/svc_YOUR_ENDPOINT/stop" \
  -H "Authorization: Bearer $INFEROPTIMAL_DEDICATED_KEY"

Stop cancels waiting requests, lets active ones finish, then releases the hardware and ends billing. The URL and key stay the same, and requests return 410 until the next Start. An endpoint is never stopped for idling unless you set idleSeconds: after 30 idle minutes its owner gets one warning email with what it costs and a Stop link, and using it again re-arms the warning. Stopping an endpoint that is already stopped changes nothing.

Whisper: speech to text

Quote a Whisper checkpoint the engine serves natively (architecture WhisperForConditionalGeneration, for example model: "openai/whisper-large-v3-turbo") on a dedicated GPU row, then launch and retrieve the connection and endpoint key using the lifecycle above. The endpoint speaks the OpenAI audio API: POST {base_url}/audio/transcriptions and POST {base_url}/audio/translations with multipart fields file, model and optionally language, temperature and response_format. Chat routes return 400 route_not_supported. Other speech models are refused at quote time.

curl "$BASE_URL/audio/transcriptions" \
  -H "Authorization: Bearer $INFEROPTIMAL_ENDPOINT_KEY" \
  -F file=@recording.wav -F model=whisper-large-v3-turbo -F response_format=json

Direct uploads are limited to 100 MB, the edge's request-body limit; a larger body gets 413 request_too_large with these instructions. For recordings up to 512 MiB, upload to storage first and transcribe by reference:

# 1. Ask for an upload URL with the exact file size (valid one hour, for this endpoint only).
curl "$BASE_URL/audio/uploads" \
  -H "Authorization: Bearer $INFEROPTIMAL_ENDPOINT_KEY" -H "Content-Type: application/json" \
  -d "{\"bytes\": $(wc -c < meeting.wav)}"
# -> {"upload_id": "upl_...", "upload_url": "https://...", "method": "PUT", "headers": {"Content-Length": "..."}}

# 2. PUT the file to upload_url (it goes straight to storage).
curl -X PUT --upload-file meeting.wav "$UPLOAD_URL"

# 3. Transcribe it: the same route, a JSON body naming upload_id instead of a file.
curl "$BASE_URL/audio/transcriptions" \
  -H "Authorization: Bearer $INFEROPTIMAL_ENDPOINT_KEY" -H "Content-Type: application/json" \
  -d '{"upload_id": "upl_...", "model": "whisper-large-v3-turbo", "response_format": "json"}'

The other JSON fields are passed to the engine as form fields. An upload id only works on the endpoint that issued it, and the stored file is deleted once it has been transcribed; one that is never transcribed is deleted after a day. /audio/translations accepts the same JSON body.

Audio is limited to 2 hours. When a file goes to the engine whole (see below), the engine also refuses audio whose decoded samples exceed 256 MiB, about 70 minutes of 16 kHz mono. Long audio sent to the engine whole loses words: it cuts at fixed 30-second windows (word error rate 3.06% against 1.78% on LibriSpeech test-clean, measured October 22, 2026). The endpoint therefore cuts each transcription at speech pauses first (Silero VAD, pieces of at most 30 seconds, 160 ms minimum silence), transcribes the pieces in parallel and joins the text, for response_format json (the default) and text. Send -F chunking=none to have a file transcribed whole. Translations, verbose_json, srt, vtt and streamed requests go to the engine whole. Transcription responses in json format carry usage.seconds, the duration of the whole input; every audio answer also carries an X-Audio-Duration-Seconds header when the duration is known. The endpoint's audio seconds and request counts appear as audio_usage (unit audio_seconds) in get_deployment stats and get_metrics. Billing is the same as for any dedicated endpoint.

LatentSync 1.6: batch video + audio → MP4

Quote model: "ByteDance/LatentSync-1.6", then launch and retrieve the connection and endpoint key using the lifecycle above. This prepared endpoint runs on InfraOptimal cloud. Use its host as ENDPOINT, without /v1; media jobs use file uploads rather than Chat Completions.

Already launched? Reuse that endpoint and its endpoint key. The optional workload-sizing form is not needed to submit a job. Upload the actual files as multipart fields named video and audio; file paths, URLs or base64 inside JSON are not accepted. Large-file JSON requests use completed upload IDs instead. Let your HTTP client set the multipart Content-Type and boundary.

curl "$ENDPOINT/jobs" \
  -H "Authorization: Bearer $INFEROPTIMAL_ENDPOINT_KEY" \
  -H "Idempotency-Key: customer-job-001" \
  -F video=@face.mp4 -F audio=@speech.wav \
  -F steps=20 -F guidance=1.5

The 202 response includes id, status: "queued" and status_url. Save the returned ID as JOB_ID. Reuse the idempotency key with identical inputs when retrying this submission; different inputs with the same key return 409. Use a new key for a different job.

# Poll until status is "completed"; stop on "failed" or "cancelled".
curl "$ENDPOINT/jobs/$JOB_ID" \
  -H "Authorization: Bearer $INFEROPTIMAL_ENDPOINT_KEY"

# Download the result after completion.
curl -f "$ENDPOINT/jobs/$JOB_ID/result" \
  -H "Authorization: Bearer $INFEROPTIMAL_ENDPOINT_KEY" \
  -o lipsynced.mp4

New LatentSync 1.6 endpoints process video in bounded windows, with no fixed clip-duration limit. Multipart uploads allow 32 MiB per input; the client below automatically uses chunked uploads for larger files, up to 2 GiB per input. Video dimensions must not exceed 1920 pixels on either side. Leave enough time in your accepted endpoint lease for the job to finish; runtime and spending limits still apply. Read /runtime for your endpoint’s input and parameter requirements. The endpoint accepts one running job and eight queued jobs; additional submissions return 429. Existing LatentSync 1.5 deployments and its model selector remain supported.

import { submitMediaJob } from './media-jobs.mjs';
const job = await submitMediaJob({
  baseUrl: process.env.ENDPOINT,
  apiKey: process.env.INFEROPTIMAL_ENDPOINT_KEY,
  videoPath: './face.mp4', audioPath: './speech.wav',
  idempotencyKey: 'customer-job-001',
  parameters: { steps: 20, guidance: 1.5 }
});
console.log('Save this job ID:', job.id);
const status = await job.wait({ timeoutMs: 60 * 60 * 1000 });
if (status.status !== 'completed') throw new Error(status.status);
await job.download('./lipsynced.mp4');

For larger batches, keep excess jobs in your client queue; on 429, wait for a submitted job to finish, or retry with exponential backoff, keeping the same job key and inputs. Download each completed result promptly: it expires one hour after that job finishes, and stopping the endpoint removes access to all results.

# When all jobs are finished, stop the deployment using your platform key.
curl -X POST \
  "https://infraoptimal.com/console/api/deployments/$DEPLOYMENT_ID/stop" \
  -H "Authorization: Bearer $INFEROPTIMAL_API_KEY" \
  -H "Content-Type: application/json" -d '{}'