AI Service Health Checks: Decide When to Pause and When to Restart

Your AI assistant cannot reach its model provider. A health check turns red, so the hosting platform restarts the application. The fresh instance reaches the same unavailable provider, fails the same check and restarts again.

In that hypothetical workflow, the monitoring has detected a real problem but chosen an action that cannot repair its cause. The business now has to understand both the dependency failure and the repeated restarts.

For an owner commissioning an AI service, “it has automatic health checks” is therefore an incomplete reassurance. Ask what each check measures, what action follows a failure and why that action should improve the situation.

Three questions that deserve different answers

The Kubernetes documentation distinguishes startup, liveness and readiness probes. A startup probe allows initialisation to finish before liveness and readiness checks run. Liveness failures can trigger a container restart after the configured tolerance. Readiness determines whether a container is ready to accept traffic; a failed readiness check does not itself restart the container.

In business language, separate these questions: has the application finished getting started; is it stuck in a state a restart may fix; and should it receive new work right now? They are related, but they do not require identical tests or responses.

You do not need Kubernetes to ask those questions. If a supplier uses another hosting system, ask them to explain its equivalent behaviour. A diagram of your own service is more useful than assuming a named platform supplies the right checks automatically.

A dependency failure is not necessarily a broken worker

Imagine an independent publisher with a service that accepts article requests, stores their status and sends drafts for review. The AI drafting step calls an external model. If that model is unavailable, the publisher might still want staff to view existing requests and correct instructions.

Under that proposed operating policy, the drafting component should stop accepting work it cannot complete, or route it into an explicitly managed waiting state. Restarting every part of the service merely because the model cannot be reached would need a separate justification.

Amazon’s Implementing health checks, published in December 2019, discusses this wider trade-off. A dependency check can detect trouble outside a server as well as local trouble. Automatically removing servers that could still perform useful work can worsen availability, particularly when a shared dependency affects many of them together.

The editorial implication is to match the response to the component and the cause. An alert about the model provider, a pause on drafting and a restart of a stuck local process should not be treated as interchangeable recovery actions.

Define what should remain available

Start with the work customers and staff actually need. In the publisher example, list viewing a saved request, submitting a new one, generating a draft and approving an existing draft separately. For each, decide whether it can operate safely without the model provider.

Do not mark an operation available merely because its web page loads. If submitting a request only produces an error after the customer has entered everything, the page being reachable is a poor description of the customer experience. Decide whether the action should be offered, delayed or clearly declined.

If you offer a waiting state, say what it means. Has the request been stored durably? Who can see that it is waiting? What will resume processing, and who handles it if the dependency stays unavailable? These are design requirements for this hypothetical service, rather than promises supplied by a health endpoint.

MaryChuks.com’s article on keeping a busy AI feature from blocking essential work examines the related isolation question. Health checks add a different decision: once a problem is detected, which automated action should follow?

Give startup a fair test of its own

A service that is still loading should not be confused with one that has stopped making progress. Ask the developer to demonstrate a normal cold start and explain how the checks allow for its expected initialisation.

Kubernetes provides startup probes for this purpose. Its documentation also warns that poorly implemented liveness probes can contribute to cascading failures by restarting containers under load and increasing pressure on the remaining ones.

That is a reason to ask for evidence about the configured checks, rather than a reason to remove them. The owner should hear a concrete explanation of which failure a restart can resolve and how the team avoids interpreting ordinary startup or temporary overload as that failure.

Request a demonstration with distinct failure cases

Use a test environment and disposable work. First, ask the team to make the model dependency unavailable while keeping the local application healthy. Observe which functions remain accessible, whether new drafting work is accepted and whether the system repeatedly restarts.

Second, ask the team to simulate a local fault for which a restart is the agreed recovery. Check that this produces the intended response. The two demonstrations should make the distinction between dependency trouble and local failure visible.

Third, restore the dependency. Watch what happens to waiting work and to requests already in progress. The expected result should follow the documented workflow, with a clear record of completion, delay or failure. Do not infer successful recovery solely from a green status indicator.

Finally, include a slow-start demonstration. If you only test a fully running system, you have not checked the conditions under which the application first becomes ready. Record the intended outcomes so the same scenarios can be revisited after a deployment change.

Give each signal an owner and an action

Ask for a short operational note that pairs every health signal with its meaning and response. Include the action taken automatically, the notification sent to a person and the point at which that person must intervene.

Avoid a single unexplained “healthy” badge as the owner’s only view. In the publisher example, “request viewing available; drafting paused because the model dependency is unavailable” describes the operating state more usefully. It also gives support staff something accurate to tell a customer.

Health checks are part of a decision system. Their value depends on the work they protect and the recovery they trigger. A check that correctly notices trouble still needs a response that fits the trouble it noticed.

Sources

Primary references: Kubernetes: Liveness, Readiness, and Startup Probes (living documentation, checked 8 November 2026); Amazon: Implementing health checks (5 December 2019). The publishing-service scenario, operating policy and demonstration sequence are illustrative recommendations, not a report of an outage.