Skip to main content
Long Pollinglesson 2 of 3 · 2 min read

Timeouts and Reconnects

You do not choose the deadline

Every long poll needs a deadline, and you do not get to choose it freely.

Any hop between your client and your server that kills idle connections sets your ceiling. Cloud balancers commonly default to 60 seconds, proxies to the same, and some corporate ones give up at 30.

Hold a request for 90 seconds behind a balancer set to 60 and that balancer answers with an error at the one minute mark. Your client code then gets to demonstrate what its retry logic does.

Set your server's timeout comfortably below the shortest thing in the path, and 25 seconds is a common safe pick. Answer the timeout case with an empty success so your client simply loops.

The reconnect gap

Mind the reconnect gap, which is the subtle bug here. Between one response completing and the next request arriving there is a window, often 50 to 200 milliseconds across the public internet. Any event firing inside it has no parked request to ride home on.

Ask for whatever is new since now and those events vanish silently. Carry a cursor instead. Every response includes the identifier of the last event it delivered, and the next request sends it back so your server can hand over anything that landed in the gap.

Do that from day one. Retrofitting it means reproducing a race you can barely see.

Give failures their own rules. On a clean response, reconnect immediately. On an error or a dropped connection, back off with some randomness, because a server that just crashed does not want every client back inside the same 100 milliseconds.

Reuse your connections while you are at it. With keep-alive, a reconnect is one round trip on a warm connection instead of three on a cold one.

the shape of it
Response donelast id 41150 ms gapmsg 42 landsGET ?since=41Returns msg 42nothing lostcursor replay
step 1 of 3
A cursor in the reconnect request recovers events that land in the gap between polls.

Worked example

Marcus deploys a notifications long poll behind an ALB and holds requests for 90 seconds because a bigger number felt more efficient. Beta users start reporting occasional error toasts, and the ALB metrics show a steady drip of 504s, one per client every 90 seconds. He drops the hold to 25 seconds and the 504s disappear. Two weeks later a different report: chat messages occasionally never arrive until refresh. Logs show every missing message was created within 120 ms of a poll response completing, right inside the reconnect gap. He adds a since cursor to the request, replays the gap on reconnect, and writes an integration test that fires an event exactly between two polls. The test fails on the old code every single run, which is the most satisfying kind of regression test.