Spent a chunk of a weekend debugging something that turned out not to be a bug I could fix — it was a known limitation I had to route around instead. Writing it down because “the AI can’t find anything in the knowledge base” is exactly the kind of vague symptom that’s easy to misdiagnose as something wrong with your setup.
The symptom
Kristen asked her assistant a simple question: what equipment is in her garage gym. She’d already uploaded a fitness profile into a Knowledge base and connected it to her assistant in Model Settings. The assistant came back with nothing. Not a wrong answer — no retrieval at all, like the knowledge base wasn’t there.
First instinct was the usual: restart the container, assume the index got corrupted when the assistant was rebuilt, maybe re-upload the document. None of that was actually the problem.
The real cause
Instead of guessing further, went looking for whether this was a known issue rather than something specific to this setup. It is: knowledge bases attached to a model in Model Settings don’t reliably auto-trigger RAG in Open WebUI. The connection exists, the document is indexed, but the automatic retrieval step that’s supposed to happen on every relevant question just doesn’t fire consistently.
The documented workaround is to reference the knowledge base explicitly in the chat itself, using #:
#Kristen Fitness Profile What equipment do I have in my garage gym?
That worked — confirming the knowledge base and the document inside it were both fine. The auto-attachment was the broken link, not the data.
Why the workaround wasn’t good enough
Explicit # tagging works, but it means the person asking has to already know which knowledge base to invoke and remember to type it correctly every time. That’s fine for a power user testing things. It’s not fine for an assistant that’s supposed to just work when someone asks it a normal question.
So during a later cleanup pass — rebuilding the consolidated Assistant from scratch — the fix wasn’t to keep fighting the auto-RAG behavior. It was to stop relying on it for anything that actually needs to be reliably remembered. The fitness profile content that lived in a Knowledge base got copied directly into Memory instead — as several sanitized, factual entries, the same style used for hardware specs and project details — in addition to the knowledge base still existing for reference.
Why Memory turned out more dependable
Memory in Open WebUI works differently from Knowledge/RAG under the hood: it’s injected into context automatically on every relevant turn, not retrieved conditionally based on a similarity search that may or may not decide the query matches. There’s no indexing step to go stale, no auto-attachment toggle that silently doesn’t fire, no need to remember special chat syntax. If it’s in Memory, the assistant has it. Every time.
The tradeoff is real and worth naming honestly: Memory doesn’t scale to large documents the way a Knowledge base can, and it’s manual — you’re writing or copying the facts in yourself instead of just dropping a file and letting RAG chunk it. For a big reference document (a manual, a long spec sheet), Knowledge is still the right tool, #-tagged explicitly when it matters. But for the kind of compact, durable facts an assistant should just know — what gear is in the gym, what hardware is in a given machine, what a project’s current status is — typing it straight into Memory turned out to be more reliable than trusting RAG to find it on its own.
The actual lesson
“The AI can’t find it in the knowledge base” isn’t always a data problem or a you-configured-it-wrong problem. Sometimes it’s the retrieval layer itself being unreliable, and the fix isn’t a better prompt or a cleaner document — it’s picking the storage mechanism that actually guarantees delivery for the kind of information that matters. Knowledge bases for bulk reference material you’re willing to explicitly tag. Memory for the small, load-bearing facts you need the assistant to have every single time, no exceptions.