The source vanished from the answer, not the database

A convincing absence

In an assistant that answers questions with document citations, retrieval happens before the answer is written. The reader sees the final text, not whether every source was searched successfully. The answer offered a caveat that sounded careful: the topic was not in the retrieved passages. That is a useful sentence when search works and finds no evidence. Here, the document was still in the database. What had vanished was the path to it.

The change appeared after an index refresh. The primary source stopped appearing in review questions while other sources still did. The assistant did not need to invent anything to mislead. It only needed to present an empty result as a conclusion about the content.

An empty list looks like a simple fact. In a retrieval system, it can hide two very different stories: I searched the source and found no passage; or I could not search the source at all.

The new file and the old reference

The refresh recreated the index. Meanwhile, processes still serving questions held an open reference to the old table. That reference pointed at files that no longer existed. The query failed while reading the old data, even though the new table was available.

Clearing the cache in the process that performed the refresh helped there, but could not reach the other running processes. Nor could the system count on their restarting soon. As long as they stayed alive, the old reference stayed old.

The less visible part was error handling. Retrieval converted the failure to an empty list, and context assembly translated that list into a gap in the source. The answer mechanism received a statement of absence instead of a signal that retrieval was unavailable.

A second attempt for a specific reason

The repair had to start before answer generation. When the error indicated a reference to deleted files, the process dropped that reference, opened the current table, and tried the query once more. Once was enough: if the fresh read also failed, an endless loop would only hide a real failure for longer.

There was a wrinkle in retrieval. It tried a fuller column selection and could fall back to a basic selection for an older index. A stale-reference error must not disappear into that fallback. It had to reach the routine that knew how to reopen the table.

A second failure also had to be visible to operators. The code began logging it as an error so it could be monitored. After that log, though, it still returned an empty list to context assembly. Operations gained a diagnostic signal; the answer shown to a reader could still carry the ambiguity.

What review can establish

The incident record describes running processes with old references after the refresh and the missing primary source in review questions. The code retries this error once and records an explicit failure when the priority source is empty. Tests cover the observed storage error, reopening, the one-retry limit, and propagation of unrelated failures.

Those checks do not show that every future answer will distinguish missing content from unavailable retrieval. In the implementation examined, operator visibility improves, but the interface still receives an empty list if the second attempt fails. The next boundary is carrying an unavailable state through to the answer, so readers do not get a conclusion about a document the system could not consult.