Skip to main content
The OpenAI Realtime API provides bidirectional, low-latency voice / text streams, for scenarios like voice assistants and live meeting summarization.
OpenAI officially supports WebRTC / WebSocket / SIP transports and offers next-gen realtime models like gpt-realtime-2 and gpt-realtime-1.5. The OpenPAI gateway currently centers on WebSocket; whether other transports and models are available is subject to the console / announcements.

​
Connect

wss://openp.ai/v1/realtime?model=gpt-realtime-2
Pass authentication during the handshake via a subprotocol or a header:
MethodExample
Header (Node / curl)Authorization: Bearer sk-XXXXXXXXXXXX + OpenAI-Beta: realtime=v1
Browser subprotocol["realtime", "openai-insecure-api-key.sk-XXXX", "openai-beta.realtime-v1"]
The browser-subprotocol method puts the key on the client — for demos only. In production, create a short-lived token through your own backend.

​
Create an ephemeral token

curl https://openp.ai/v1/realtime/sessions \
  -H "Authorization: Bearer $OPENPAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-realtime-2",
    "voice": "alloy"
  }'
The client_secret.value in the response is a short-lived token you can safely send to the browser as a subprotocol.

​
Event protocol

JSON events are sent and received in both directions over the WebSocket. Key events:
Client sendsServer replies
session.update — set instructions / voice / toolssession.created / session.updated
input_audio_buffer.append — append PCM16 audioinput_audio_buffer.committed
input_audio_buffer.commit — commit a segment of audioconversation.item.created
conversation.item.create — append a text messageconversation.item.created
response.create — request response generationresponse.created / response.audio.delta / response.text.delta / response.done

​
Minimal example (Node.js)

import WebSocket from "ws";

const ws = new WebSocket(
  "wss://openp.ai/v1/realtime?model=gpt-realtime-2",
  { headers: {
    "Authorization": `Bearer ${process.env.OPENPAI_API_KEY}`,
    "OpenAI-Beta": "realtime=v1",
  }}
);

ws.on("open", () => {
  ws.send(JSON.stringify({
    type: "session.update",
    session: { instructions: "You are a helpful assistant", voice: "alloy" }
  }));

  ws.send(JSON.stringify({
    type: "conversation.item.create",
    item: { type: "message", role: "user", content: [{type:"input_text", text:"hi"}]}
  }));

  ws.send(JSON.stringify({ type: "response.create" }));
});

ws.on("message", (data) => {
  const evt = JSON.parse(data.toString());
  if (evt.type === "response.text.delta") process.stdout.write(evt.delta);
});

​
Audio format

  • Input: PCM 16-bit, 24kHz, mono, base64-encoded.
  • Output: likewise PCM 16-bit / 24kHz.
You can convert using tools like webrtc-adapter, MediaRecorder or ffmpeg.

​
Tool calling

You can declare tools in session.update; during generation the model may produce response.function_call_arguments.delta events. You need to execute the tool and return the result via conversation.item.create (type: function_call_output).

​
Billing

  • Audio input / output tokens and text tokens are billed separately, at prices higher than ordinary chat.
  • No charge while the connection is held open, but if idle for a long time it’s best to close it proactively.
See Model pricing for details.