WebSockets: The Full-Duplex Paradigm

Mechanics

The WebSocket protocol provides a persistent, full-duplex bidirectional communication channel over a single TCP connection. The communication process begins with an HTTP handshake, where the client sends a request with an Upgrade: websocket header. If the server accepts the connection, it responds with a 101 (Switching Protocols) status code, after which communication transitions from HTTP to the WebSocket protocol. This feature allows both parties—client and server—to independently send data at any time, which is a fundamental difference from the request-response model characteristic of HTTP.

Strengths

The main advantage of the WebSocket protocol is its capability for low-latency, bidirectional communication. This makes it the standard for highly interactive applications where immediate data exchange is crucial. Typical use cases include multiplayer online games, real-time collaboration tools (e.g., text editors like Google Docs), trading platforms, and traditional chat applications. Once the connection is established, the overhead for sending subsequent messages is minimal, as there is no need to resend HTTP headers for each interaction.

Inherent Challenges

• Head-of-Line (HOL) Blocking: Because WebSockets operate over a single, ordered TCP stream, the loss of one packet can block all subsequent messages, even if they are independent of each other. This is a significant drawback in applications that transmit a mix of data with different priorities (e.g., critical control data and less important status updates) over the same connection.⁹
• State Management Complexity: A WebSocket connection is inherently stateful, meaning the client is tied to a specific server instance for the duration of the session. This creates significant challenges for horizontal scaling. To ensure that messages can be sent between clients connected to different server instances, it is necessary to implement additional mechanisms such as sticky sessions and message brokers (e.g., Redis Pub/Sub).
• Reconnection Logic: The WebSocket protocol itself does not define a mechanism for automatic reconnection in case of a dropped connection. Such logic must be implemented on the client application side, which increases its complexity.

Server-Sent Events (SSE): The Unidirectional Streaming Standard

Mechanics

Server-Sent Events (SSE) is a much simpler, unidirectional protocol where a server can push data to a client over a standard, long-lived HTTP connection. The client initiates the connection using the browser's built-in EventSource API. The server, in response, sends a Content-Type: text/event-stream header and keeps the connection open to stream data in a simple text-based format.⁹ Each event is separated by a double newline and can include fields like id, event, and data.

Strengths

• Simplicity: SSE uses the standard HTTP protocol, making it much easier to implement and more compatible with existing network infrastructure, such as corporate firewalls and proxy servers, which may have issues with the non-standard WebSocket protocol.
• Built-in Features: The client-side EventSource API automatically handles key features like reconnection after a dropped connection and passing the ID of the last received event (Last-Event-ID). This allows for seamless stream resumption without data loss and significantly reduces the complexity of the client-side code.
• Precedent in LLM Response Streaming: SSE is the de facto standard used by leading language model platforms, including ChatGPT, for streaming responses token by token. This proves its excellent fit for this specific use case, which is central to the analyzed architecture.

Limitations

• Unidirectionality: The main limitation of SSE is that it only supports server-to-client communication. Messages from the client to the server must be sent using a separate, standard HTTP request, for example, a POST.
• Text-Only Data: The SSE protocol is designed for transmitting text data in UTF-8 format and does not natively support binary data, unlike WebSockets.
• Connection Limits: Historically, with the HTTP/1.1 protocol, browsers imposed a limit on the number of simultaneous SSE connections per domain (usually 6). This problem has been largely mitigated by connection multiplexing in the HTTP/2 protocol.

Protocol Comparison Matrix

The following table presents a condensed, direct comparison of the key technical features of the analyzed protocols, facilitating an informed architectural decision.

Adoption of Streaming Protocols by AI Models

The following table summarizes the findings from this section, presenting a market overview and cementing SSE's position as the industry standard for streaming text responses.

Recommendation for the Agent Runtime-Frontend Connection

Deconstruction of the Communication Pattern in the Presented Architecture

Analyzing the attached diagram, key communication patterns between the frontend and backend can be identified:

• Asymmetric Flow: The primary real-time need is to stream responses from the "Agent Runtime" (likely from an LLM agent) to the "Web User" client. This is a classic server-to-client "push" model, often in a one-to-many relationship.
• Client Input: User input, such as prompts or commands, are discrete, self-contained messages sent from the client to the server. They do not require a persistent, low-latency stream from the client.

Why SSE is the Architecturally Superior Choice

Given the above patterns and the technical analysis, SSE is the recommended protocol for the following reasons:

1. Protocol-to-Use-Case Fit: SSE is explicitly designed for unidirectional server-to-client streaming, which perfectly matches the application's dominant requirement. Using WebSockets in this case would be an over-engineered and more complex solution.
2. Superior Integration with GCP API Gateway: The ability to apply policies to individual events in an SSE stream using EventFlow is a key advantage in terms of security, observability, and mediation, which is lacking in the passthrough model for WebSockets.
3. Operational Simplicity and Resilience: SSE uses the standard HTTP protocol and has built-in client-side reconnection logic, which reduces complexity on both the server and client sides and makes the system more resilient to transient network issues and timeout misconfigurations.
4. Industry Precedent: Using SSE for streaming LLM responses aligns with the proven architectural patterns of major market players like OpenAI, further validating the soundness of this choice.

When to Reconsider: The Case for WebSockets

To provide a balanced report, it is necessary to define the narrow and specific conditions under which WebSockets would become the preferred choice. These would be situations where the application requires:

• True Low-Latency Bidirectional Streaming: If the client needed to stream data to the server with very low latency, for example, live audio from the user's microphone for immediate transcription or translation, or continuous telemetry data.
• Binary Data Requirement: If the primary data type being sent in both directions was binary data, which WebSockets support natively and SSE does not.

Based on the provided diagram and typical interactions with a chat agent, these conditions do not appear to be met, which reinforces the recommendation to use SSE.