AI Service Restarts: Give Unfinished Work a Clear Exit

A deployment finishes successfully. The new version opens, the health indicator is green and the owner returns to work. Meanwhile, a customer’s report is still marked “generating”. The old process disappeared while preparing it, and nobody has established whether the job finished, can resume or needs attention. The website has restarted; the customer’s work has not necessarily survived.

For a small AI business, a release review should therefore include an exit question: what happens to work already under way when the service stops? This applies to a planned deployment as well as a deliberate restart. Starting the replacement is only one part of the handover.

A shutdown signal gives an opportunity

Kubernetes’s maintained Pod Lifecycle documentation describes a graceful termination process with a limited grace period. Typically, the runtime sends a termination signal before forcefully stopping remaining processes when the period expires. The default termination grace period is 30 seconds. A configured pre-stop hook runs within that shutdown allowance, rather than providing an unlimited extra window.

That mechanism offers an application time to respond. It does not establish that a customer’s report has been saved or that a background job can recover. Those are application behaviours that an owner should ask the team to demonstrate. A platform can provide an opportunity for orderly closure while the application still lacks a useful closing routine.

The details are platform-specific. Amazon ECS’s StopTask reference likewise describes a stop signal, normally SIGTERM, and a default timeout before SIGKILL. It also documents different handling for Windows containers. These examples explain the principle; they are not instructions to copy one timeout into every hosting service.

Separate the front door from the workbench

Imagine a hypothetical service that creates product descriptions. A short web request accepts the customer’s brief, and a separate worker prepares the output. Closing the web server to new traffic would not, by itself, tell the worker to stop collecting jobs. Equally, stopping the worker from collecting jobs would not explain what happens to the description it is already preparing.

Ask for both paths to be described. Which component stops accepting new requests? Which component stops claiming queued work? What happens to requests that have already been accepted? The answer should name the actual components in the service, including any worker running separately from the website.

This complements the question in AI Service Health Checks: Decide When to Pause and When to Restart. A readiness check concerns whether a component should receive work. An exit plan must also account for the work it already holds. One green replacement instance is incomplete evidence of a successful handover.

Choose an outcome for each unfinished job

For the product-description example, ask the team to choose a clear outcome for each stage. A nearly finished draft might be allowed to complete within the available window. A longer job might save enough durable progress for recovery. Another job might return to a queue for a controlled retry. These are proposed design choices, not automatic guarantees of either platform.

The choice depends on what can actually be recovered. If the application has no supported way to resume a model request halfway through, calling its status “paused” would mislead the customer. A more honest state might explain that the attempt was interrupted and a new attempt is required. The interface should follow the implementation’s evidence.

Define completion precisely. A result existing only in the departing process is different from a result saved where its replacement can retrieve it. In the acceptance brief, identify the durable record that proves completion and the point at which the customer can access the finished output.

Do not make waiting longer the whole plan

A larger shutdown allowance can be part of a design, but I would not accept it as the complete answer. Ask what the application does during the allowance, what it does when the allowance runs out and which job remains recoverable afterwards. The same questions apply even when the usual work takes very little time.

Consider a hypothetical report that normally finishes quickly but waits on an external service during the test. If the exit routine simply waits indefinitely, the platform’s eventual forced stop still needs a recovery path. If it declares success immediately, the record needs evidence that the work really finished. Neither a long wait nor a quick exit settles the business question.

Include uncertainty about external actions. Suppose the service requested an export from another provider just before stopping. The replacement needs a defined way to establish what happened before repeating the action. Ask the team to document which operations can safely be retried and which require reconciliation. Avoid promising that restarting the worker alone prevents duplication.

Rehearse a stop while useful work is running

A meaningful acceptance exercise uses a disposable test environment and recognisable sample jobs. Start work, initiate the normal deployment or stop procedure, and inspect the outcome from the customer’s side. Keep the job references so the team can connect the visible result with the saved status and operational records.

Repeat the exercise at different stages: before a job starts, while it is working, after its result is saved and before the customer sees completion. Also test a case that outlasts the permitted shutdown window. Write the expected outcome for each stage before running the exercise, so a reassuring log message cannot substitute for the intended behaviour.

The owner does not need to interpret every operating-system signal. Request a short record showing which sample jobs completed, which were recovered, which need intervention and whether any were duplicated or disappeared. An unexplained job is a finding to resolve, even if the deployment itself is marked successful.

Make the handover visible to people

Assign responsibility for unresolved work. If an interrupted job requires manual review, identify who receives it and what evidence they need. Give customer support a way to locate the job without asking the customer to submit the same material repeatedly. Agree the wording for waiting, retrying and intervention states.

A practical release criterion is that every accepted sample job has an explained outcome after the old process stops. That does not promise protection against every possible failure. It gives the business a concrete standard for routine changes: the replacement service starts, and the work entrusted to its predecessor remains accounted for.

Sources

Primary references: Kubernetes, Pod Lifecycle: termination of Pods; Amazon Web Services, Amazon ECS StopTask API reference. These are maintained technical documents, checked on 8 November 2026, rather than dated news reports. The product-description and report scenarios are hypothetical; the owner review process is editorial guidance.