Real-Time Transcription (Streaming STT)

The streaming endpoint transcribes audio while it is being spoken, over a single WebSocket connection. You send raw PCM, and the server sends back partial results that get corrected as more audio arrives, then a final result for each segment.

Use it for live captions, voice agents, contact center monitoring and anything where waiting for the recording to end is not an option. For files that already exist, the REST transcription endpoint is simpler and supports more formats, languages and post-processing.

Closed beta

pulse-stt-streaming-v1 is not listed in the public model catalog yet. Ask support to enable it for your organization before integrating.

At a glance


Endpoint	`wss://api.sippulse.ai/v1/asr/listen`, or `/v1/listen`
Audio	16-bit signed PCM, little endian, interleaved
Model	`pulse-stt-streaming-v1`
Languages	`pt-BR`, `pt`
Channels	up to 8 in a single session
Billing	audio seconds per channel, silence included

Handshake

Everything that can fail is checked before the connection is accepted, so a rejected session comes back as a normal HTTP response on the same socket, with a status code and a readable body. Once you see the 101 Switching Protocols, the session is live and the pod is already connected.

Query parameters

Parameter	Values	Default	Notes
`encoding`	`linear16`	`linear16`	Signed 16-bit PCM, little endian
`sample_rate`	8000 to 48000	`8000`	Must match the audio you send
`channels`	1 to 8	`1`	Interleaved samples; each channel is transcribed separately
`language`	`pt-BR`, `pt`	`pt-BR`
`interim_results`	`true`, `false`	`true`	`false` sends only final results
`endpointing`	560 to 1500	`700`	Milliseconds of silence that close a turn. Values outside the range are clamped to it; `false`, `no` or `0` means no endpointing and becomes `1500`
`model`	model name	account default	Use `pulse-stt-streaming-v1`
`tag`	up to 64 chars	none	Echoed back in `Metadata`; useful to correlate with your own logs

An unknown parameter is ignored, not rejected: the Deepgram SDK sends punctuate, smart_format and diarize by default, and dropping the connection over them would be useless. An invalid value on a parameter we do know is a 400 with the offending field in the message, with one exception: endpointing out of range is clamped instead of rejected.

Parameters that have no equivalent here: smart_format (punctuation comes from the model itself) and multichannel (use channels).

Authentication

The credential is read in this order, and the first one found is the one used:

apiKey query parameter
api-key header
accessToken query parameter
Authorization: Bearer <token> or Authorization: Token <token> header
Sec-WebSocket-Protocol: token, <credential> subprotocol
x-guest-token header

Options 1 and 3 exist because browsers cannot set headers on a WebSocket handshake; option 5 is the same trick the Deepgram browser SDK uses, and the server echoes the token subprotocol back so the browser accepts the upgrade. Option 4 accepts both prefixes, so a Deepgram client keeps its original header.

An API key is the right credential for a server-side integration. A key in a query string ends up in proxy and browser logs, so prefer the header whenever your client can set one.

Sending audio

Send binary frames with raw PCM. Frames of 20 to 100 ms are the sweet spot; the hard limit is 64 KB per frame. For a stereo call at 16 kHz, 40 ms is 16000 * 0.04 * 2 channels * 2 bytes = 2560 bytes.

There is no need to pace the audio exactly, but sending much faster than real time will eventually trip backpressure protection and close the session with 1013.

Two text frames are accepted:

json

{ "type": "Finalize" }

Closes the current turn immediately, without ending the session: use it when your own voice activity detection knows the speaker stopped. Any pending audio is transcribed and emitted as a final result.

json

{ "type": "CloseStream" }

Ends the session. Remaining finals and the Metadata frame are sent, then the connection closes with 1000. Always finish this way instead of dropping the socket, or the final segment and the usage summary are lost.

Any other text frame is discarded with a warning, including Deepgram's KeepAlive. It does count as client activity for the idle timer, so leaving it in a ported client is harmless.

Receiving results

Results

json

{
  "type": "Results",
  "metadata": { "request_id": "4f1a9c2e8b7d4a6f" },
  "channel_index": [0, 2],
  "start": 4.32,
  "duration": 1.86,
  "is_final": true,
  "speech_final": true,
  "channel": {
    "alternatives": [
      {
        "transcript": "qual é o horário de funcionamento",
        "confidence": 0.8594,
        "words": [
          { "word": "qual", "start": 4.32, "end": 4.48, "confidence": 0.8594 },
          { "word": "é", "start": 4.48, "end": 4.56, "confidence": 0.8203 }
        ]
      }
    ]
  }
}

Field	Meaning
`channel_index`	`[channel, total channels]`. Each channel is transcribed independently
`start`, `duration`	Seconds from the beginning of the session
`is_final`	`false` is a partial result that will be rewritten; `true` is a segment that will not change
`speech_final`	`true` when the segment closed the turn, that is, the silence reached `endpointing`
`metadata.request_id`	Trace id of the session, the same value as `trace_id` in `Metadata`. Quote it when opening a support ticket
`alternatives`	Always exactly one alternative. `confidence` is the model's own, from 0 to 1, and is also present on each item of `words`

Rendering rule: replace the current line while is_final is false, commit it when is_final is true, and treat speech_final as end of the speaker's turn.

Metadata

Sent once, right before the session closes:

json

{
  "type": "Metadata",
  "trace_id": "4f1a9c2e8b7d4a6f",
  "channels": 2,
  "duration": [61.44, 61.44],
  "model": "pulse-stt-streaming-v1",
  "tag": "call-8821"
}

duration is the billed audio per channel, in seconds. trace_id is the id to quote when opening a support ticket about a specific session.

Error

Both the pre-handshake HTTP body and any error frame use the same envelope:

json

{ "type": "Error", "code": "INVALID_PARAM", "message": "sample_rate must be between 8000 and 48000, got 96000" }

When the session is refused

Before the 101, failures are HTTP responses, not close codes. Your client library will surface them as a handshake error carrying a status.

Status	Code	What to do
400	`INVALID_PARAM`, `UNSUPPORTED_ENCODING`, `UNSUPPORTED_LANGUAGE`	Fix the query string; the message names the field
401	`UNAUTHORIZED`	Missing, malformed or invalid credential
402	`INSUFFICIENT_CREDIT`	The organization has no credit to start the session
404	`MODEL_NOT_FOUND`	The model is not in your catalog, is inactive, or is not a streaming model
429	`TOO_MANY_SESSIONS`	The organization's concurrent channel quota is in use. Back off and retry, `Retry-After` is set
502	`UPSTREAM_UNAVAILABLE`, `UPSTREAM_REJECTED`	The speech worker could not be reached or refused the session
503	`POD_NOT_READY`, `CREDIT_AUTHORIZATION_UNAVAILABLE`, `SHUTTING_DOWN`	Temporary. Retry after `Retry-After`
500	`INTERNAL_ERROR`, `POD_NOT_CONFIGURED`	Our side. Open a ticket with the timestamp

Note that channels=2 consumes two channels of the quota, not one.

When the session ends

After the 101, the reason arrives as a WebSocket close code.

Code	Meaning
`1000`	Normal end: your `CloseStream`, or the two-hour session ceiling
`1001`	The server is shutting down for a deploy, or your client stopped responding to pings (20 s ping, 120 s idle)
`1008`	Credit ran out mid-session
`1011`	Internal error
`1013`	Backpressure: audio arrived faster than it could be processed, or your client stopped reading results

On 1001 and 1013, reconnect and continue sending audio. On 1008, top up the balance first; reconnecting immediately will fail with 402.

Billing

Streaming is billed on audio seconds per channel, under the transcription rule, for the real length of the session: no minimum block, no rounding up to the minute. A 12-second session costs 12 seconds. Silence counts: a session that holds the connection open with no one talking is still audio the model processed.

The price is quoted per minute in the pricing table, as the market does, but the metering is per second. A stereo call costs twice a mono one of the same length. The duration array in Metadata is exactly what gets billed, and each session appears in the dashboard attributed to the project and API key that opened it.

Long sessions are billed in windows as they run, not only at the end, so the consumption shows up in the dashboard while the call is still in progress.

A working example

Streams a WAV file in real time and prints the transcription. Requires pip install websockets.

python

import asyncio
import json
import wave

import websockets

URL = "wss://api.sippulse.ai/v1/asr/listen"
API_KEY = "sp-..."
BLOCK_MS = 40


async def main(path: str) -> None:
    audio = wave.open(path, "rb")
    rate, channels = audio.getframerate(), audio.getnchannels()
    url = (
        f"{URL}?encoding=linear16&sample_rate={rate}&channels={channels}"
        f"&language=pt-BR&endpointing=700&model=pulse-stt-streaming-v1"
    )

    async with websockets.connect(url, additional_headers={"api-key": API_KEY}) as ws:
        async def receive() -> None:
            async for message in ws:
                frame = json.loads(message)
                if frame["type"] == "Results":
                    text = frame["channel"]["alternatives"][0]["transcript"]
                    if not text:
                        continue
                    mark = "final" if frame["is_final"] else "     "
                    print(f"{mark} [ch{frame['channel_index'][0]}] {text}")
                elif frame["type"] == "Metadata":
                    print(f"billed: {sum(frame['duration']):.1f}s")
                elif frame["type"] == "Error":
                    print(f"error: {frame['code']} - {frame['message']}")

        reader = asyncio.create_task(receive())
        block = int(rate * BLOCK_MS / 1000)
        while chunk := audio.readframes(block):
            await ws.send(chunk)
            await asyncio.sleep(BLOCK_MS / 1000)  # real time pacing

        await ws.send(json.dumps({"type": "CloseStream"}))
        await asyncio.wait_for(reader, timeout=10)


asyncio.run(main("call.wav"))

To transcribe a microphone instead of a file, replace the read loop with your audio capture library and keep everything else: the protocol does not care where the PCM came from.

Coming from Deepgram

The protocol is deliberately Deepgram-shaped, so a client already built on it needs very little. Both livekit-plugins-deepgram and the official Python SDK connect by pointing the base URL at this endpoint and using an API key as the credential; nothing else in the configuration changes.

python

from livekit.plugins import deepgram

stt = deepgram.STT(
    base_url="wss://api.sippulse.ai/v1/asr/listen",
    api_key="sp-...",
    model="pulse-stt-streaming-v1",
    language="pt-BR",
    endpointing_ms=700,
)

The official SDK appends /v1/listen to the host it is given, and that path is accepted as well: wss://api.sippulse.ai/v1/listen and wss://api.sippulse.ai/v1/asr/listen open the same session. The Authorization: Token <key> scheme that both clients use is accepted exactly like Bearer.

Endpointing

endpointing outside the supported range no longer closes the connection: it is normalized to the nearest bound of [560, 1500] milliseconds. The value in effect is what closes the turn, so a client that asks for 25, the plugin default, gets 560. endpointing=false, no or 0 (the plugin sends false when endpointing_ms is 0) means "no endpointing" in Deepgram and maps to the longest turn available, 1500.

The 1500 ceiling is not arbitrary: above it the turn closes too far from the end of speech and comes out without the terminal punctuation.

A value that is neither an integer nor false/no returns INVALID_PARAM (HTTP 400).

Parameters that are ignored

Parameters we do not recognize, including the ones the Deepgram clients send by default - punctuate, smart_format, diarize, no_delay, vad_events, filler_words, profanity_filter, numerals - are ignored in silence. They do not close the session and they do not appear in the results.

Messages we do not send

SpeechStarted and UtteranceEnd are not emitted. The LiveKit plugin works without them: it treats them as optional and keeps transcribing from the Results and Metadata frames alone.

Known limitations

One language at a time: language accepts pt-BR and pt, which are the same language. Any other value returns UNSUPPORTED_LANGUAGE (HTTP 400). Other languages are not available on this model.
endpointing out of range: integer values outside [560, 1500] are normalized to the nearest bound, not rejected. false, no or zero mean "no endpointing" and become the maximum turn, 1500. Only a value that is neither an integer nor one of those returns INVALID_PARAM (HTTP 400).

Choosing between streaming and REST

	Streaming	REST
Result arrives	while speaking	after the whole file
Input	live PCM	MP3, WAV, OGG, PCM up to 25 MB
Languages	pt-BR	multilingual
Diarization	by channel	by channel or by speaker
Anonymization, Audio Intelligence	no	yes

A common arrangement is both: streaming for the live experience, and a REST transcription of the recording afterwards for analytics, redaction and insights.

Agents

Configuration

Tools

Advanced

Deploying Agents

Settings

Real-Time Transcription (Streaming STT)

At a glance

Handshake

Query parameters

Authentication

Sending audio

Receiving results

Results

Metadata

Error

When the session is refused

When the session ends

Billing

A working example

Coming from Deepgram

Endpointing

Parameters that are ignored

Messages we do not send

Known limitations

Choosing between streaming and REST

Configuration

Tools

Advanced

Deploying Agents

Real-Time Transcription (Streaming STT) ​

At a glance ​

Handshake ​

Query parameters ​

Authentication ​

Sending audio ​

Receiving results ​

Results ​

Metadata ​

Error ​

When the session is refused ​

When the session ends ​

Billing ​

A working example ​

Coming from Deepgram ​

Endpointing ​

Parameters that are ignored ​

Messages we do not send ​

Known limitations ​

Choosing between streaming and REST ​

Real-Time Transcription (Streaming STT)

At a glance

Handshake

Query parameters

Authentication

Sending audio

Receiving results

Results

Metadata

Error

When the session is refused

When the session ends

Billing

A working example

Coming from Deepgram

Endpointing

Parameters that are ignored

Messages we do not send

Known limitations

Choosing between streaming and REST