Last week one of our own meetings went through Loquelle, our live meeting copilot. The call was about an AI search marketplace, outbound to a competitor, a conference, a follow-up email. The recap at the end was accurate to the line. The live suggestions during the call were a different story. Four of them, delivered with complete confidence, were about energy efficiency upgrades to reduce heating bills.
Nobody had said the word heating. Nobody had said energy, upgrades, or bills. A fifth card named a bank that was never mentioned and a tenfold cost saving that nobody had claimed. If the person in that meeting had read one of those cards aloud, the tool built to make them look sharp would have made them look absent.
This post is about where those cards came from and what we changed. It is written for anyone building or buying a live AI assistant, because the failure is not specific to us.
Where does a hallucinated suggestion actually come from?
It is tempting to blame the model, and the model is part of it. But when we traced the heating cards back, the source was closer to home. The user had opened the meeting with a feature we call Continue: it loads the notes of a previous meeting as background so the copilot remembers what was agreed last time. That earlier meeting had been about something else entirely.
In the first minute of a call there is almost no transcript. The model has a rich block of context from last week and a few words of audio from today, and it does what language models do with an imbalance like that: it answers from the richer source. The heating cards were not invented from nothing. They were faithfully derived from the wrong meeting.
The invented bank name was a different mechanism. The transcript mentioned a conference and a hardware vendor. The model, asked to be useful, reached for a plausible customer to complete the thought. That is the classic hallucination: a fluent continuation with no anchor in the source.
So there were two distinct problems wearing the same symptom. Stale context being treated as live, and fluent completion filling gaps the transcript left open.
Why is a prompt not enough to stop it?
Our prompt already said, in several places, never to invent a subject, never to introduce a name that was not said, and to stay silent rather than repeat. It had said so for months. The model followed those instructions most of the time, which is exactly the problem with instructions: most of the time is not a guarantee, and a live call is judged on its worst card, not its average one.
We had learned this once already with numbers. Long before the heating incident, Loquelle had produced “we cut latency by 30%” in an interview where no such figure existed. The fix then was not a better sentence in the prompt. It was a validator that checks every number in a spoken line against the transcript, the brief and the attached document, and removes the line if the number is not there. That validator has been quietly deleting invented figures ever since.
The lesson we should have generalised sooner: anything that must never happen needs a check in code, not a request in prose.
What are the three checks that now run on every card?
Every live suggestion now passes through three deterministic gates before it can reach the screen. Each one is conservative on purpose, and each drops only the offending line, never the whole reply.
- Numbers. Every digit-bearing claim must appear in the source material. “Thirty percent” in speech counts as a source for “30%” on a card. This is the original validator, unchanged.
- Names. A capitalised word that is not at the start of a sentence, is not an ordinary capitalised word like a weekday or a language, and appears nowhere in the sources is treated as an invented name and the line is dropped. Acronyms are exempt, because GPU or SEO are generic terms, not entities.
- Topics. A line with five or more content words, of which fewer than a third appear anywhere in what has been said, is about something nobody said. We measured this on the heating meeting: the invented cards scored between zero and 29 percent overlap; the genuine ones scored between 62 and 83. A threshold of one third sits comfortably between them.
Two exemptions matter. Technical mode, where the copilot answers a coding or design question, legitimately uses vocabulary nobody has spoken yet, so the topic gate does not apply there. And an explicit ask from the user, typed into the panel, is the user requesting exactly that kind of fresh answer, so it is exempt too.
How do we stop last week's meeting leaking into this one?
The stale context problem needed its own rule, separate from the model. For the name and topic checks, the source material is now defined as the live transcript, the attached document, the user's own note, and the brief with any previous-meeting section removed. Last week's notes are still handed to the model as background, because that is what the user asked for. But a card cannot be grounded by them. Until this call's transcript touches a subject, that subject cannot appear on a card.
That one change would have stopped all four heating cards on its own. The topic check is the belt to its braces.
What about the repeats and the contradictions?
The same meeting showed two smaller faults that are just as damaging to trust. The same suggestion about confirming a conference arrived three times in slightly different words. And the question of whether to send a follow-up email got three answers over ten minutes: send it now, send it tomorrow, wait a couple of days.
Repeats already had a filter, but it was bypassed whenever the user pressed the button that asks for words right now, on the theory that an explicit request should never come back empty. True, but three presses were producing three identical sentences. Now a near-repeat of something already on screen is answered with a short note saying so, instead of a fourth copy.
Contradictions were new territory. The fix is simple to state: once a question has been answered in a meeting, a later reply built on the same question is dropped whole. Consistency beats a second opinion nobody asked for. The prompt also now says so explicitly, but as above, the code is the guarantee and the prompt is the courtesy.
What does the result look like?
We replayed the original transcript through the new gates with the original cards. All four heating cards were dropped as off-topic. The card naming the bank was dropped for an unknown name. Every genuine card survived: the conference, the introduction email, the prototype, the question about the link. One deliberately fresh-vocabulary answer, the kind you would give in a technical interview, was dropped in normal mode and kept in Technical mode, which is the intended behaviour.
The recap, which was already right, is untouched by any of this. It summarises what was said, and it had never been the problem.
None of this makes the live suggestions perfect. A gate that drops a line can drop a good one, and we would rather show silence than a confident mistake, so the trade is deliberate. But it does mean the specific class of failure that would have embarrassed a user in front of a client can no longer reach the screen, whatever the model decides to say.
Questions this raises
Does grounding make the suggestions slower?
No. All three checks are string operations on text already in memory, and they run in well under a millisecond after the model has replied. The latency you see is the model's, not the validator's.
Can a correct suggestion be dropped by mistake?
Yes, and we accept that. A short direct answer with fresh vocabulary can fail the topic check in normal meeting mode. We exempt short lines, Technical mode and explicit asks to keep that rare, and we log every drop with its reason so we can see when the gate is wrong.
Why not just remove the Continue feature?
Because people use it for recurring calls and it works well once the conversation is under way. The problem was only ever the first minute, before the transcript had anything to say. Restricting what can ground a card fixes that without taking the feature away.