Frame contract

The precise behavior of the raw-frame API: buffer ownership, backpressure and drop semantics, and the invariants the publish and receive paths guarantee. The developer docs cover usage; this document covers the guarantees.

Buffer ownership

Publish. write() copies every buffer synchronously before it returns. The caller may reuse or free y.data, u.data, v.data and pcm immediately. There is no borrow, and no lifetime obligation.

Receive. A delivered frame is an owned copy in JavaScript memory. The SDK keeps no reference to it after delivery, so a frame may be held across await, transferred to a worker_thread, or batched, with no restriction.

close() is an optional hint that releases the plane buffers immediately rather than waiting for garbage collection. It is not required for correctness. It is idempotent, and reading plane data (y/u/v, or pcm) after close() throws. Non-plane metadata stays readable.

When write() returns false

false always means the frame was not sent. It never means "buffer full, wait for drain" - that is the Node Writable convention, and tracks are deliberately not presented as Node streams.

Kind	false means
Video	libwebrtc's adapter rejected the frame: most often the encoder sink has not attached yet (a write before connect() resolves), and otherwise rate-limiting or a rejected resolution.
Audio	The write did not fit in the bounded publish queue. Nothing was buffered: a rejected write is never partially published.

Invalid input does not return false; it throws TypeError or RangeError. The distinction is deliberate: false is a runtime condition to react to, an exception is a programming error to fix.

Backpressure and drops

Every frame queue is bounded. Blocking the media engine and buffering without limit are both rejected: blocking corrupts reconnection and statistics, and at roughly 41 MB/s per 720p30 video track, unbounded buffering is an eventual out-of-memory failure. A full receive queue drops; a full audio publish queue rejects the write, because the publisher is a caller that can be told false while a decoder thread cannot.

Where frames are dropped

There are two queues in the receive path, and DeliveryStats.framesDropped is the sum across both, so it is the total the consumer never saw.

Queue	Location	Policy
Native transfer queue	Before the native-to-JS handoff	Bounded to the resolved maxQueue, drop-oldest. Bounding it here is what stops a latest consumer holding a deep queue of ~1.3 MB frames in native memory.
JS policy queue	After the handoff	The mode/maxQueue/drop policy given to frames().

Publish is different. Video publish is synchronous - write() hands the frame straight to the encoder - so there is no SDK-side send queue at all, and sendQueueDepth/maxQueue are always 0. Audio publish has a real queue, drained one 10 ms chunk at a time, so its depth and bound are meaningful.

The audio publish queue rejects rather than sheds. A write() is accepted only if it fits whole within source.maxQueue (10 ms chunks, default 50, so ~500 ms); otherwise nothing is buffered and write() returns false. A single write() larger than the bound can never fit and is always rejected, so size maxQueue to the largest burst the application intends to publish. Rejecting rather than shedding keeps false meaning "not sent" on both paths, and keeps WriteStats.framesWritten an exact count of what the SDK accepted.

Receive policy

The rest of this section is the receive path: the options frames() takes. The publish bound above is separate and is configured through source.maxQueue.

mode: 'latest' keeps only the newest frame. mode: 'queue' buffers up to maxQueue and then applies drop: 'oldest' sheds from the front, 'newest' sheds the arriving frame.

Defaults are media-aware, because the cost of loss differs by kind:

Kind	mode	maxQueue	Why
Video	latest	1	A stale frame has no value for live inference.
Audio	queue	10	A small buffer smooths receiver jitter; gaps degrade transcription.

These are counts of frames, not 10 ms chunks; the publish-side source.maxQueue is the one measured in chunks.

maxQueue is capped at 1024 so a misconfiguration cannot exhaust memory.

Ordering is preserved. In queue mode, frames are delivered in arrival order. Dropping removes frames from the sequence; it never reorders the ones that survive.

A frame handed directly to a waiting consumer is never queued, so no policy applies to it and it cannot be dropped.

Observability

Every drop is counted. Nothing is discarded silently anywhere in either path.

Signal	Direction	Shape
track.getStats()	Receive	DeliveryStats: framesDelivered, framesDropped, queueDepth, maxQueue, lastTimestamp?
track.getWriteStats()	Publish	WriteStats: framesWritten, framesDropped, sendQueueDepth, maxQueue, lastTimestamp?, timestampRegressions
frameDropped event	Receive	(count, sinceLastUs), coalesced over 500 ms

Receive stats remain readable after the receiver ends, reporting the final numbers, so a summary can be taken during teardown.

There is no separate underrun signal. An idle track is indistinguishable from a stalled one at this layer; use framesDelivered against wall-clock time, or the WebRTC statistics from Room.getStats(), to tell them apart.

Audio publish counters split by scope. framesWritten, framesDropped, lastTimestamp and timestampRegressions count one track's own write() calls. sendQueueDepth and maxQueue describe the audio device, which is shared by every local audio track in the process; source.maxQueue binds that same shared device, so the most recent value passed to createLocalAudioTrack applies to all of them. With one local audio track - the normal case - the distinction does not arise.

Timestamps

timestamp is a plain number of microseconds, in both directions. Microsecond resolution stays exact in a JS number for roughly 285 years, and the engine reports microseconds natively, so no wider integer type is needed.

Omitting timestamp on publish stamps the frame "now", which is correct for an application-paced live source.

The two publish paths use the supplied timestamp differently. Video carries it through to the encoded frame. Audio publish is strictly FIFO - the audio device emits queued samples on its own 10 ms cadence - so an audio timestamp is observability-only: it feeds WriteStats.lastTimestamp and WriteStats.timestampRegressions and does not alter what is sent or when.

Non-monotonic timestamps are handled per kind. On audio, a timestamp that does not advance past the previous one is accepted and increments WriteStats.timestampRegressions. Rejecting would break a legitimate producer that restarts, such as a looping file source; silently reordering or discarding would hide a real problem. A rising count on a live source means the supplied timestamps are wrong, which shows up downstream as jitter.

On video, libwebrtc's adapter can reject a frame whose timestamp does not advance. A rejected frame makes write() return false and is counted in WriteStats.framesDropped, not timestampRegressions, which counts only regressed frames the adapter accepted.

frameId is an SDK-generated monotonic per-track counter on received frames. Use it for gap and drop detection. There is no frameId on input: correlate a published frame by its timestamp.

Publish invariants

• Audio format is fixed: PCM S16LE, 48 kHz, mono. AudioFrameInput exposes no sample-rate or channel fields, because the engine's encoder path is wired for exactly this and anything else would be silently wrong rather than resampled. Resampling is the application's responsibility.
• Video format is fixed: I420. format is optional on input but rejected if present and not 'I420'.
• No dynamic resize. Where a track was created with a source, a frame whose dimensions disagree is rejected rather than rescaled. Without a source, the encoder follows what is written.
• Dimensions must be positive and even. I420 chroma is subsampled by two in each dimension, so an odd dimension has no well-defined chroma plane size.
• Strides must be at least the plane width. Padding beyond that is fine and is preserved through the copy.

Execution model and threading

Frames are delivered to the event loop as ready, application-owned objects. Delivery is serial and in order per track.

Decoded frames arrive on native decoder threads. The bulk plane copy out of libwebrtc happens there, off the event loop; the copy into JS buffers currently happens on the loop. Whether that second copy moves off-loop is an internal optimization that can land without an API change.

The SDK cannot stop an application blocking the loop with synchronous per-frame work. Heavy per-frame CPU work belongs in worker_threads or in an asynchronous service call; frames are structured to transfer to a worker without copying.

A track supports a single receiver. Calling frames() while a receiver is active throws. An application that needs several consumers fans out inside its own loop. The iterator ends on unsubscribe and on Room disconnect or dispose, so a for await loop exits on its own rather than hanging.

Data track

send() accepts a string or a Buffer. Messages larger than 64 KB (kMaxMessageSize) are rejected synchronously with a RangeError and never transmitted; the limit is counted in UTF-8 bytes, not characters.

send() returns a promise that always resolves, never rejects, so a fire-and-forget send cannot produce an unhandled rejection. It resolves to { ok, messageId, error? } once the engine has processed the send.

If the track is destroyed while a send is still in flight, the outstanding promise resolves with ok: false and an error naming the teardown, rather than being left pending.