Streaming Architecture

Concept of the skill

What it is: streaming-architecture is the discipline for designing one logical result as an ordered sequence of values over time, with explicit flow control, framing, error, resume, and termination semantics.

Mental model: Every streaming design has a producer, stream, consumer, backpressure path, and termination signal. Transports encode those primitives differently; they do not replace the design work.

Why it exists: Some results are too large, too slow, or too useful early to wait for a complete batch response. Streaming gives earlier value and bounded memory only when the contract is explicit.

What it is NOT: It is not browser freshness UX, one-shot API payload design, durable worker execution, model/tool transcript protocol design, event-contract design, or the page-level rendering taxonomy.

Adjacent concepts: real-time-updates owns browser freshness and live-dashboard UX; api-design owns one bounded request/response surface; background-jobs owns durable queued work; tool-call-flow owns model/tool message-history protocol; rendering-models owns CSR/SSR/SSG/RSC taxonomy; event-contract-design owns event payload and topic contracts.

One-line analogy: A streaming architecture is a conveyor belt: boxes move one at a time, the dock signals when it is full, and a final marker says the shipment is complete.

Common misconception: The transport is not the concept. The concept is the contract for ordered incremental delivery, backpressure, and termination.

Coverage

The discipline of designing systems that emit and consume sequences of values over time with explicit flow control. Covers the five primitives (producer, stream, consumer, backpressure, termination), the transport mechanisms (HTTP chunked transfer, SSE, WebSocket, HTTP/2 streams, gRPC streaming, WHATWG Streams, Node streams), the directionality and backpressure-model taxonomies, in-stream error semantics, delivery guarantees, the design contract between producer and consumer, and the failure modes that streaming systems exhibit at scale (slow consumer, abandoned consumer, mid-stream disconnect, head-of-line blocking, partial-result correctness).

Philosophy of the skill

Streaming is the response to a category of problem that batch request/response cannot solve: results that are too big to materialize, too slow to wait for, or too useful at the front to delay until the back arrives. The cost is a more demanding contract between producer and consumer — error semantics get harder, backpressure must be explicit, connections must be managed — but for the problem class it serves, batch is not an inferior streaming; batch is wrong.

The deeper philosophy is that streaming is a contract about time. The five primitives — producer, stream, consumer, backpressure, termination — are the same whether the transport is an SSE event source, a gRPC bidirectional RPC, a WHATWG ReadableStream piping into a TransformStream, an RSC chunked response, or an LLM emitting tokens. A practitioner who learns the contract once can move between transports at the cost of an encoding translation; a practitioner who learns only one transport's API conflates the contract with its encoding and treats every new streaming surface as a new concept.

The discipline of streaming architecture is to know when streaming is the right shape, to design the contract explicitly when it is, and to make backpressure and termination first-class in that contract rather than emergent properties of the transport.

The Five Primitives

Any streaming system can be analyzed as: who is the producer, what is the stream's framing, who is the consumer, how does backpressure travel upstream, and how is termination signaled. A streaming system that has no answer for any of these is incomplete and will fail under load.

Transport Comparison

Selection rule: pick directionality first (one-way → SSE or HTTP chunked; two-way → WebSocket or gRPC bidi), then framing needs (binary structured → gRPC or WebSocket; text events → SSE; opaque bytes → HTTP chunked), then infrastructure compatibility (HTTP/1.1 proxies often break WebSocket and SSE; HTTP/2 proxies are friendlier).

Server-Sent Events — The Streaming Default For Server→Client

SSE is the lowest-ceremony, highest-compatibility transport for server→client streaming. The HTML Living Standard EventSource API ships in every modern browser.

GET /stream HTTP/1.1
Accept: text/event-stream

HTTP/1.1 200 OK
Content-Type: text/event-stream
Cache-Control: no-store

event: token
data: {"id":1,"text":"Hello"}

event: token
data: {"id":2,"text":" world"}

event: done
data: {}

Properties:

• One-way (server→client). Client can only initiate the connection; once established, only the server sends.
• UTF-8 text events with event:, data:, and id: fields.
• Built-in reconnect via Last-Event-ID header on the resume request.
• Works through HTTP/1.1 proxies; no upgrade handshake.
• No backpressure beyond TCP-level flow control — application-level pacing must be designed in if the producer can outrun the consumer.

For LLM token streaming, progress bars, status feeds, dashboards, and any one-way live update channel, SSE is the right starting point. Move to WebSocket only if bidirectionality is required.

WebSocket — When Bidirectionality Is Required

WebSocket (RFC 6455) is a bidirectional, framed, binary-or-text protocol upgraded from HTTP. Both ends can send at any time.

GET /ws HTTP/1.1
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: ...
Sec-WebSocket-Version: 13

HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: ...

Properties:

• Full-duplex framed message protocol.
• Application-level backpressure required — neither end's send rate is automatically matched to the other end's receive rate.
• No built-in reconnect; the application must handle close codes and resume manually.
• Sensitive to HTTP/1.1 intermediaries that buffer or close idle connections.

For collaborative editing, multiplayer games, chat, and any interaction where both sides need to send unsolicited messages, WebSocket is the right transport. For one-way server-driven updates, it is more machinery than necessary and SSE is usually a better fit.

Backpressure In Detail

The slow-consumer failure mode is the most consequential streaming failure. A producer that emits at 1000 events/sec and a consumer that processes at 100/sec produces 900 events/sec of accumulation. After one minute, the buffer holds 54,000 events; after one hour, 3.24 million. Without backpressure, this exhausts memory.

For each new streaming endpoint, the answer to "what happens when the consumer is slower than the producer?" must be one of these strategies — explicitly, not by accident.

Termination And Resume

Termination is a distinct message, not the absence of new values. A consumer that interprets a 10-second silence as "the stream ended" will be wrong on any production network. Explicit termination signals:

Resume after disconnect is a separate concern. SSE has it built in (Last-Event-ID); WebSocket requires application-level resume tokens; gRPC offers reconnect but not exactly-once across reconnects. A streaming consumer must be designed for "the connection dropped at value 4,732; reconnect and continue at 4,733" if that semantic matters.

In-Stream Errors

The choice depends on whether the consumer can usefully proceed past an error. For LLM token streams, an error mid-generation is usually fatal — fail-fast. For a search-results stream, one row's permission error need not stop the others — in-band errors. For a long-lived telemetry stream, errors are out-of-band by convention.

Streaming In Modern Web Frameworks

Each of these is the same five-primitive contract dressed in a framework's API. The framework adds typing, suspense integration, error boundary handling, and ergonomic composition — but the underlying contract is the streaming-architecture primitive.

Verification

After applying this skill, verify:

• [ ] Every streaming endpoint has a named answer for: who is the producer, who is the consumer, how is the stream framed, how does backpressure travel upstream, how is termination signaled.
• [ ] No long-lived connection assumes silence means "done" — termination is always a distinct signal.
• [ ] No streaming consumer materializes the full stream into a collection unless the stream is known-bounded and small.
• [ ] Backpressure strategy is explicit, not emergent: pull, credit-based push, drop-on-overflow, or block-on-overflow are named choices.
• [ ] Mid-stream errors have a defined encoding (fail-fast, in-band, or out-of-band) — they are not left to "whatever the transport does."
• [ ] If reconnect/resume matters for correctness, the protocol carries enough state (last-event-id, resume token) for the consumer to resume without gaps or duplicates.
• [ ] SSE is used for one-way; WebSocket for bidirectional; gRPC streaming for typed inter-service streams — choices are justified by directionality and framing, not by familiarity.
• [ ] The streaming endpoint's behavior under a deliberately slow consumer has been tested, not assumed.

Do NOT Use When

Key Sources

• IETF. RFC 9112 — HTTP/1.1, § 7.1 Chunked Transfer Coding. The base mechanism for streaming an HTTP body of unknown length.
• IETF. RFC 9113 — HTTP/2, § 5 Streams and Multiplexing. HTTP/2's stream primitive and per-stream flow control (WINDOW_UPDATE frames).
• IETF. RFC 6455 — The WebSocket Protocol. The canonical specification for the bidirectional framed-message protocol.
• WHATWG. HTML Living Standard — Server-Sent Events. The EventSource API and the text/event-stream protocol.
• WHATWG. Streams Living Standard. ReadableStream, WritableStream, TransformStream — the in-browser streaming primitives.
• Node.js. Stream API documentation. Readable, Writable, Duplex, Transform; backpressure via highWaterMark and pipe().
• gRPC Authors. gRPC Concepts — RPC Lifecycle. Server-streaming, client-streaming, and bidirectional-streaming RPC modes.
• Reactive Streams. Reactive Streams Specification. The cross-language specification for asynchronous stream processing with non-blocking backpressure — the basis of Akka Streams, RxJava, Project Reactor.
• React. renderToPipeableStream. The server-rendering API that pipes React HTML to a Node.js stream and supports aborting unfinished rendering.
• Mozilla Developer Network. Using readable streams. Practical reference for browser-side stream consumption.

Skill Graph context

<!-- skill-graph-context:start (generated — do not edit by hand) -->

Classification

• Subject: backend-engineering
• Public: true
• Domain: engineering/realtime
• Scope: Teaching the portable architecture discipline for incremental value delivery over time: producer, stream, consumer, backpressure, framing, termination, reconnect/resume, in-stream error semantics, and transport trade-offs across HTTP chunked transfer, Server-Sent Events, WebSocket, HTTP/2/gRPC, WHATWG Streams, Node streams, and React server rendering streams. Applies when one logical result is delivered as many ordered chunks or messages and the system must reason about slow consumers, abandoned consumers, partial-result correctness, and resource bounds. Excludes browser freshness UX and live-dashboard transport selection (real-time-updates), single request/response payload design (api-design), durable worker execution and retries (background-jobs), model/tool transcript protocol design (tool-call-flow), event payload/domain-event contracts (event-contract-design), and page-level rendering-model taxonomy (rendering-models).

When to use

• model a producer, stream, consumer, backpressure, and termination contract for an SSE progress stream
• choose a pull, credit-based push, drop, block, or sampling backpressure strategy for a stream whose producer outruns its consumer
• compare HTTP chunked transfer, SSE, WebSocket, gRPC streaming, WHATWG Streams, and Node streams by directionality, framing, backpressure, and termination
• Triggers: streaming-architecture, how should this endpoint stream, producer stream consumer backpressure termination, what's the backpressure story, partial result delivery

Not for

• design the JSON shape and status codes for a single request-response API payload
• choose polling, SSE, or WebSocket for browser dashboard freshness UX
• move a slow CSV export into a background job and define retry policy

Related skills

• Verify with: client-server-boundary, performance-budgets, api-design, real-time-updates
• Related: event-contract-design, rendering-models, real-time-updates, api-design, background-jobs, client-server-boundary, performance-budgets, tool-call-flow

Concept

• Mental model: |
• Purpose: |
• Boundary: |
• Analogy: A streaming architecture is to data delivery what a conveyor belt is to a factory's order fulfillment — you do not wait for an entire shipment to be assembled before any piece leaves the warehouse; the belt moves boxes one at a time, the loading dock signals when it's full (backpressure), a final marker indicates the shipment is complete (termination), and the receiving truck can start unloading the first box while the last one is still being assembled. A conveyor with no full-dock signal flings boxes onto the floor; a conveyor with no end-marker keeps the truck driver waiting forever.
• Common misconception: |

Grounding

• Mode: universal
• Truth sources: https://www.rfc-editor.org/rfc/rfc9112#name-chunked-transfer-coding, https://www.rfc-editor.org/rfc/rfc9113#name-streams-and-multiplexing, https://www.rfc-editor.org/rfc/rfc6455, https://html.spec.whatwg.org/multipage/server-sent-events.html, https://streams.spec.whatwg.org/, https://nodejs.org/api/stream.html, https://grpc.io/docs/what-is-grpc/core-concepts/, https://www.reactive-streams.org/, https://react.dev/reference/react-dom/server/renderToPipeableStream, https://developer.mozilla.org/en-US/docs/Web/API/Streams_API/Using_readable_streams

Keywords

• streaming, stream, backpressure, SSE, server-sent events, chunked transfer, HTTP/2, WebSocket, WHATWG Streams, ReadableStream

<!-- skill-graph-context:end -->