Realtime conversation

The OpenAI Realtime API provides bidirectional, low-latency voice / text streams, for scenarios like voice assistants and live meeting summarization.

OpenAI officially supports WebRTC / WebSocket / SIP transports and offers next-gen realtime models like gpt-realtime-2 and gpt-realtime-1.5. The OpenPAI gateway currently centers on WebSocket; whether other transports and models are available is subject to the console / announcements.

Connect

wss://openp.ai/v1/realtime?model=gpt-realtime-2

Pass authentication during the handshake via a subprotocol or a header:

Method	Example
Header (Node / curl)	`Authorization: Bearer sk-XXXXXXXXXXXX` + `OpenAI-Beta: realtime=v1`
Browser subprotocol	`["realtime", "openai-insecure-api-key.sk-XXXX", "openai-beta.realtime-v1"]`

The browser-subprotocol method puts the key on the client — for demos only. In production, create a short-lived token through your own backend.

Create an ephemeral token

curl https://openp.ai/v1/realtime/sessions \
  -H "Authorization: Bearer $OPENPAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-realtime-2",
    "voice": "alloy"
  }'

The client_secret.value in the response is a short-lived token you can safely send to the browser as a subprotocol.

Event protocol

JSON events are sent and received in both directions over the WebSocket. Key events:

Client sends	Server replies
`session.update` — set instructions / voice / tools	`session.created` / `session.updated`
`input_audio_buffer.append` — append PCM16 audio	`input_audio_buffer.committed`
`input_audio_buffer.commit` — commit a segment of audio	`conversation.item.created`
`conversation.item.create` — append a text message	`conversation.item.created`
`response.create` — request response generation	`response.created` / `response.audio.delta` / `response.text.delta` / `response.done`

Minimal example (Node.js)

import WebSocket from "ws";

const ws = new WebSocket(
  "wss://openp.ai/v1/realtime?model=gpt-realtime-2",
  { headers: {
    "Authorization": `Bearer ${process.env.OPENPAI_API_KEY}`,
    "OpenAI-Beta": "realtime=v1",
  }}
);

ws.on("open", () => {
  ws.send(JSON.stringify({
    type: "session.update",
    session: { instructions: "You are a helpful assistant", voice: "alloy" }
  }));

  ws.send(JSON.stringify({
    type: "conversation.item.create",
    item: { type: "message", role: "user", content: [{type:"input_text", text:"hi"}]}
  }));

  ws.send(JSON.stringify({ type: "response.create" }));
});

ws.on("message", (data) => {
  const evt = JSON.parse(data.toString());
  if (evt.type === "response.text.delta") process.stdout.write(evt.delta);
});

Audio format

Input: PCM 16-bit, 24kHz, mono, base64-encoded.
Output: likewise PCM 16-bit / 24kHz.

You can convert using tools like webrtc-adapter, MediaRecorder or ffmpeg.

Tool calling

You can declare tools in session.update; during generation the model may produce response.function_call_arguments.delta events. You need to execute the tool and return the result via conversation.item.create (type: function_call_output).

Billing

Audio input / output tokens and text tokens are billed separately, at prices higher than ordinary chat.
No charge while the connection is held open, but if idle for a long time it’s best to close it proactively.

See Model pricing for details.

Overview

OpenAI-compatible

Claude Messages

Google Gemini

Rerank

Video & image generation

Account

Connect

Create an ephemeral token

Event protocol

Minimal example (Node.js)

Audio format

Tool calling

Billing

​Connect

​Create an ephemeral token

​Event protocol

​Minimal example (Node.js)

​Audio format

​Tool calling

​Billing

Connect

Create an ephemeral token

Event protocol

Minimal example (Node.js)

Audio format

Tool calling

Billing