websocket output buffers and slow consumers
a WSS connection can stay open while the data waiting to be sent keeps growing in memory.
calling send() does not mean the client received the message. it does not mean the client processed it either.
outgoing data passes through several layers: the application's queue, the websocket library, TLS and the kernel's TCP send buffer. the exact buffering depends on the implementation.
if the connection cannot drain data fast enough, pending bytes accumulate.
for a queue receiving r_in bytes/s and draining r_out bytes/s, while it is nonempty:
db/dt = r_in - r_out
if you produce 5 MiB/s and drain 3 MiB/s, the backlog grows by roughly 2 MiB every second. another 100 MiB of headroom buys around 50 seconds at that rate, excluding allocation and protocol overhead.
a larger buffer changes how long you can absorb the mismatch. it does not change the sustainable throughput.
there are several ways to get there.
a user switches to a slow mobile network. your server keeps producing the same updates, but that connection can carry less data. its output queue grows while other clients remain healthy.
a token suddenly becomes active. the stream jumps from 100 to 10,000 updates/s. a buffer can absorb a temporary burst if the connection later has enough spare capacity to clear the backlog. at exactly the ongoing production rate, it will never catch up.
a Node.js consumer blocks its event loop with parsing or synchronous computation. it reads its socket less frequently. as receive buffers fill, TCP advertises less receive capacity. pressure eventually reaches the sender, whose writes can no longer drain at the same rate.
a browser can keep receiving network data while its JavaScript falls behind. a client library can also keep reading into its own application queue. in both cases, the server's output buffer may stay small while the consumer's memory and processing delay grow.
TCP acknowledgements tell you about transport delivery to the peer, not completion of your application handler. application-level acknowledgements or processed sequence numbers are needed if that distinction matters.
fan-out adds another constraint.
a 100 KiB/s stream sent to 1,000 clients requires about 97.7 MiB/s of outgoing application data, before transport overhead. serialization might be shared, but the network still has to carry each delivery.
each connection can accumulate a different backlog. actual memory usage depends on whether the library shares payload storage or creates copies, as well as per-connection metadata and transport buffers.
large snapshots create a different kind of delay.
if a snapshot and live prices share one connection, updates can wait behind snapshot data already queued for transmission. TCP preserves byte order. a newer application message cannot jump ahead of bytes already committed to that stream.
splitting a snapshot into application messages and pacing their submission can let you schedule live updates between chunks. a separate connection avoids sharing that ordered stream, although both connections still compete for bandwidth.
Cloudflare or an L7 load balancer from your cloud provider adds another part of the path.
when a proxy terminates the connection, your origin sends to that proxy over one transport connection, and the proxy forwards to the client over another. each leg has its own flow control and buffering.
the origin can successfully drain bytes to the proxy while the client-facing leg is slower. once the intermediary's available buffering fills, it must apply backpressure or enforce its limits. the delay before pressure reaches your origin depends on the implementation.
a transparent forwarding load balancer does not necessarily split the TCP connection this way. check the actual network path and termination points.
also distinguish queue growth from connection timeouts. an idle timeout concerns inactivity; a connection continuously delivering stale updates may never be idle.
AWS documents the idle timeout for application load balancers here: https://docs.aws.amazon.com/elasticloadbalancing/latest/application/edit-load-balancer-attributes.html
Cloudflare also documents that infrastructure restarts can terminate websocket connections: https://developers.cloudflare.com/network/websockets/
reconnection therefore needs to be part of the protocol. reopening a socket alone does not restore missed events.
the right queue policy depends on the data.
for price updates, you may be able to keep only the latest pending state per token. coalesce in your own queue before handing messages to the websocket library. once bytes have entered the transport stream, you cannot selectively retract them.
that policy only works if consumers need the latest state. it does not preserve every intermediate price change, and it cannot be applied blindly to deltas that depend on previous messages.
for transactions or fills, intermediate events can matter. bound the queue, disconnect consumers that cannot keep up, and provide a replay mechanism with sequence numbers or cursors. if the cursor is older than your retention window, define a full resynchronization path.
measure pending bytes per connection and the age of the oldest queued message. also measure the rate at which the queue drains. library buffer metrics usually describe a particular layer, so do not assume one counter includes the kernel, proxy and receiving application.
with a steady drain rate r_out, b queued bytes represent approximately b / r_out seconds before newly queued data can get past that backlog, ignoring other delays.
10 MiB waiting behind a 2 MiB/s connection already means roughly five seconds. that can be acceptable for a file transfer and unusable for a live trading feed.
set both per-connection and aggregate memory budgets. many clients with moderate backlogs can exhaust the server even when none reaches its individual limit.
a connected socket tells you the connection still exists. the age of the data tells you whether the stream is still useful.