A print studio asks an AI assistant to write a script that reads a batch of orders, checks the quantities and prepares a packing list. The assistant splits the order stream into two readers. The first checks every row; the second starts work only if the first finds no problems. The script behaves well on a sample of ten orders. Then the studio runs it on a much larger file and memory use climbs during the check, before a single line of the packing list has been produced.
The split was meant to avoid reading the source twice. It did that by creating a new obligation: every order already seen by the first reader had to remain available to the second. The useful question for a reviewer is how much of the batch might have to wait in memory, and what the studio actually needs to keep while it waits.
What Python has to keep for the slower reader
Python’s documentation for itertools.tee describes how it creates separate iterators from one input. An iterator yields items in sequence as you ask for them. If the first reader asks for an item before the second does, the split has to retain enough information for the second to receive the same item later. It does not rerun the source to recreate that item. If one reader goes through most of the input before the other starts, the documentation suggests considering a list instead.
Here is a deliberately small example. It uses quantities in place of full order records, so the gap is easy to see:
from itertools import tee
source = iter([4, 2, 7])
for_checks, for_packing = tee(source)
assert next(for_checks) == 4
assert next(for_checks) == 2
assert next(for_packing) == 4
assert list(for_packing) == [2, 7]
assert list(for_checks) == [7]By the time the packing reader starts, the checking reader has moved past two values. Both still reach the end in order. Now imagine that the checking reader consumes the entire run before packing starts. While the second reader is held back, the gap can grow to the size of the batch. The precise memory cost depends on the objects and implementation; the key point is that the buffering need grows when the readers drift apart.
There is also one source behind the two readers. After the split, use the returned iterators rather than reading directly from the original alongside them. The Python documentation also warns that iterators from the same split are not safe to use simultaneously in multiple threads. A second reader is a way to get the same sequence at a different pace. It does not give you a second independent source or a separate copy of each order record.
Choose what really needs a second pass
For the print studio, the checks and packing calculations might be simple enough to perform together. As each order arrives, validate its quantity and prepare the corresponding line. If the studio needs the whole batch to pass before anyone packs it, write to a staging file and release it only after the last check succeeds. You can read the input once while keeping a partly prepared file out of the recipient’s hands.
A small batch may be easier to hold in an ordinary list, where the cost is obvious and the code can inspect records several times. For larger files, another option is to read the same saved input twice, provided it cannot change between passes. If the source is a changing feed, a second read may see a different set of orders. In that case, use a stable snapshot if you need the two passes to agree. Choose based on how much data there is, what must happen before release and whether the input can be replayed.
Pay attention to what happens when one reader has finished its part of the job. A preview that needs only a handful of orders should release its reader when the preview is complete, rather than keeping it around while the main job runs. Better still, decide whether the preview can be produced from the main reader at the start. That makes the small amount of retained data explicit instead of letting an idle reader create a growing backlog.
Test the gap, not just the totals
- Draw the order of work. Mark when each reader first starts and whether both advance together. A full first pass followed by a second pass is the case to inspect most closely.
- Use a batch that resembles production. Include large records and realistic row counts. Watch peak memory while the slower reader waits, as well as whether the final file is correct.
- Check what an error leaves behind. Put an invalid record late in a practice batch. Verify that the previous successful packing file stays identifiable and the partial replacement does not go out.
- Ask whether the second reader needs every field. If it only needs the order ID and quantity, consider retaining or staging just those fields. Note what you would need to keep to investigate a rejected order later.
MaryChuks.com’s guide to why counting a stream can leave nothing to total explains the problem that often prompts a split in the first place. Splitting can solve that problem when both readers progress at a similar pace. If the second must wait until the first finishes, choose where the data should wait and make that choice visible in the design. The sample run should show more than the right answer: it should show why the same approach will still work when the sample becomes a day’s orders.
Source
- Python 3.12 documentation,
itertools.tee.
