Imagine a small studio that uses an AI workflow to categorise requests for changes to print orders. During the day, the workflow adds a category to each request and sends an acknowledgement. At the end of the week, it counts the categories for the studio manager. After improving the categories, the manager wants to rebuild a month of reports. A developer reruns the whole workflow, and customers receive fresh acknowledgements for orders they collected weeks ago.
The manager asked for a better account of the past. The workflow treated the old requests as new work. If you run a business that depends on AI to classify, summarise or route records, the distinction matters. You may want to recalculate what a record means without repeating what the business did in response.
Say what you are rebuilding
A backfill processes an earlier period, often to fill missing data or recalculate a result. In its backfill documentation, Apache Airflow distinguishes creating runs for dates with no run, rerunning dates with failures and rerunning completed dates. Your system may use a different scheduler, but that distinction is useful even when a small script does the work. Filling a gap and replacing an existing result are separate instructions.
For the studio, define the deliverable as a comparison report: take the accepted change requests from a named month and apply the new labels in a separate working table. Keep the old labels and the current customer records as they are. If the business eventually adopts the rebuilt report, readers should be able to tell which label rules produced it. Otherwise a drop in one category might be mistaken for a real change in customer behaviour when only the classification changed.
Mark the beginning and end of the period in the same time zone the original report used. For the studio, a request received just before midnight on the last day should not disappear from both months or appear in both. Include one record on either side of the boundary in the trial so the team can see exactly where the rebuilt month starts and ends.
Also ask whether the goal is to reproduce the original report or reinterpret the month. Reproduction requires the original inputs and settings to the extent they were kept. Reinterpretation can use the current rules, but it should be described that way. If a later edit replaced the original request text and nobody kept a copy, the rebuilt report can only use the text available now. Record that gap instead of claiming a faithful reconstruction.
Give historical runs a different route
In his 2005 note on event sourcing, software designer Martin Fowler describes a related problem: replayed records can trigger messages to external systems that cannot tell a rebuild from live work. He describes putting that decision at the boundary through which updates leave the application. For a small studio, the practical equivalent is to ensure the rebuild can write its comparison report but cannot send email or update the live order system.
Do this outside the prompt. Telling an AI assistant to avoid sending messages is weaker than running the rebuild without a live sending connection. Check what happens after the first write, too. A separate report table is useful only if adding a row there does not trigger a live automation. Walk through the destinations with whoever owns the workflow: customer messages, order status, staff tasks and anything that another service watches.
The exceptions deserve their own decision. Suppose the rebuild exposes one old request that was never acknowledged. Sending a message today may be appropriate, but a person should first check the order’s current state and decide whether a reply is still useful. Our guide to recovering failed AI jobs covers that kind of individual follow-up. A historical report can bring the case to someone’s attention without reopening every old request.
Preview the dates, then test the effects
A preview answers the question it was designed to answer. Airflow’s backfill dry run lists the dates it would consider. That list alone cannot show whether your workflow would send an email, use the wrong data or count a request twice. Test a small slice with representative records and observe both the report and the destinations the rebuild must leave alone.
- Choose a completed period with a known request count. Include one ordinary request, one whose label should change and one that was already acknowledged.
- Check the comparison report against those records. Account for excluded rows explicitly and record the model or rules used to produce the new labels.
- Verify that the rebuild made no changes to the live order table, sent no customer messages and created no staff tasks. Use logs or test destinations where available.
- Repeat the same slice to check that the comparison report does not accumulate duplicate rows. If the classification can vary, keep the outputs from each attempt separate and compare them.
Before expanding the date range, decide what would make you stop. An unexplained jump in the number of requests, a report that includes newer orders, or any attempted send to a customer should pause the run. Schedule the larger rebuild so that it does not crowd out the live work your team still needs to do. Keep the original report until someone has reviewed the new one and can explain its differences.
A backfill can tell you something new about old work. It should leave you with a report you can explain and a separate list of any actions that still deserve attention. If the only thing you wanted to change was the report, that should be the only thing the run changes.
Sources
Discover more from Marychuks.com AI, Psychology, Business & CreativeVerse
Subscribe to get the latest posts sent to your email.
