Most timeout advice is about picking a number. The more important question is which clock the number is attached to, because the common default measures something that a badly-behaved server will never trigger.
Idle timeouts and absolute deadlines are different things ¶
An idle timeout fires when nothing has arrived for N seconds. It is what most HTTP clients give you by default, and it is the correct control for a dead connection.
An absolute deadline fires when the whole operation has taken N seconds, regardless of activity.
The gap between them is a real hole. A server that sends one byte every few seconds never idles, so an idle timeout never fires — the request runs indefinitely while the client sits there believing it is making progress. That is the slow-loris shape, and it does not require malice: a database query returning results slowly, or a proxy trickling a response, produces the same thing.
Redirects multiply it. If a check follows up to ten hops and each hop gets its own idle timeout, the worst case is ten times the number you configured — and nobody who set that number was thinking in those terms.
This site now runs both: a per-socket idle timeout and a total wall-clock budget for the whole chain, checked before every redirect hop. Either alone leaves a gap — and the gap is what an alert that fires when nothing is wrong is usually hiding in.
Picking the number ¶
Three inputs, in order of how much they should influence you.
1. Measure your own cold start
The floor is not "how fast is it usually" but "how slow is it legitimately". A serverless function on a cold start, a JVM warming up, a game server loading a large modded world: all can take many times the median. Set the timeout above the legitimate worst case or you will page on every deploy and every restart.
This is the single most common mistake, and it produces alerts that fire on a schedule — which is the fastest way to train a team to ignore them.
2. Decide what "too slow" means to a user
A response that takes 30 seconds is a failure even with a 200. If your timeout is 60 seconds you are calling that success, and your availability figure will disagree with every user. Latency belongs in the definition of up, and a separate slow-but-alive state is more useful than a binary.
3. Fit it inside the budget above it
A check timeout must be shorter than the interval it runs on, or checks overlap and queue. A check inside a scheduled job must fit inside that job's own deadline, or the platform kills the whole run and you get no result at all rather than a failed one — which is much worse, because a killed job reports nothing and a failed check reports something.
Timeouts are not the alerting rule ¶
The most useful separation: a timeout decides when one check gives up; the alert rule decides how many failures constitute a problem. Conflating them is what makes people set timeouts too long.
With debounce — two consecutive failures to open, two consecutive successes to close — a shorter, honest timeout becomes safe. One slow response is recorded as slow and does not page anyone. Sustained slowness pages, which is correct, and it pages faster than a long timeout would have allowed.
A workable starting point ¶
| Control | Starting value | Why |
|---|---|---|
| Connect timeout | 5s | A TCP handshake that slow is already a failure |
| Idle timeout | 10s | Catches a dead connection quickly |
| Absolute deadline | 20-30s | The one that stops a trickling response, including across redirects |
| Slow threshold | p95 x 3 | Record as degraded, do not fail |
| Consecutive failures to alert | 2 | One blip is not an outage |
Then adjust from evidence rather than from feel: if an alert fired and nothing was wrong, find out whether the timeout was short or the debounce was absent before changing either.
The failure mode nobody plans for ¶
A timeout that is too long does not just delay detection — it can consume the thing doing the checking. Checks that hang hold connections, and enough of them will exhaust a pool while every individual check still looks like it is working. The monitoring stops reporting, which reads as everything being fine.
Bounding the total work — an absolute deadline, a cap on concurrent checks, a limit on how much of a response you will read — is what keeps a monitoring system from becoming its own outage. If you are picking which layer to bound first, what each probe actually proves is the place to start; if you want the vocabulary for slow-but-alive, see latency, availability and reliability.
Retries multiply every number you just chose ¶
A timeout is rarely the whole budget, because something above it retries. Three attempts at a 30-second timeout is a 90-second worst case, and if the caller above that also retries, the numbers multiply rather than add.
Two consequences worth planning for:
- The user-facing budget is the product, not the unit. If a page must respond in 10 seconds, and it calls a service with a 5-second timeout and two retries, the page has already lost before the last attempt starts.
- Retries against a struggling service make it worse. A service slow enough to time out is usually saturated, and three times the traffic is the opposite of what it needs. Retry with backoff and jitter, cap the total attempts, and stop retrying entirely when the failure rate says the dependency is down rather than flaky.
The pattern to avoid is a retry storm: every client times out at the same moment, retries at the same moment, and the synchronised second wave keeps the service down long after the original cause cleared.
Budget from the top down, not the bottom up ¶
Most timeout configurations are assembled bottom-up — each component picks a number that feels reasonable in isolation — and the total is whatever falls out. The result is a system where the outer request gives up before the inner one has finished, so work is done and thrown away.
Start from what the user will wait for, subtract, and pass the remaining budget down. If a call has 3 seconds left, it should not start an operation whose own timeout is 10. That is also the reasoning behind checking a deadline before following another redirect hop rather than only at the start: a chain that has already spent its allowance must not begin a fresh request.