Rivet PCB indexes the whole KiCad library at startup — 22,744 symbols and 15,124 footprints. Until this week, typing 0805 resistor into the chooser returned exactly zero results.
Not a bad result. Not a slow result. Zero rows, from a library that contains several hundred 0805 resistors. The same was true of resistor 0805, 10k resistor and 0603 capacitor. Every one of them: nothing.
And every gate in the repository was green.
What was actually wrong
Two separate faults, stacked, which is why nobody caught it by squinting at the code.
The first: the query was matched as one string, not as a set of terms. A part whose name and keywords contain "resistor" and separately contain "0805" does not contain the substring "0805 resistor" anywhere, so it scored nothing. The more precisely you described the part you wanted, the less likely you were to get it — which is the exact opposite of how a search is supposed to behave, and it means the users most punished were the ones being most specific.
The second showed up when you typed a single word. resistor returned 168 hits, and the row at the top was Device:Heater. A heater. Its keyword list is heater R resistor — a genuine match — and with nothing to break the tie beyond alphabetical order, "Heater" sorts before "R". The correct answer was in the list. It was just underneath a heating element.
The fix, and the part that matters
The ranking is now one module. It had been three: the symbol chooser, the footprint chooser and the agent's own candidate lookup each carried a hand-copied version of the same twenty lines, which had drifted. Queries are split into whole tokens, every token has to land, and exact-token matches on a part's name outrank incidental matches in a keyword list — which is what puts the resistor above the heater.
| Measure | Before | After |
|---|---|---|
| Top-1 correct, 23 labelled queries | 10 / 23 | 23 / 23 |
| Top-5 correct | 11 / 23 | 23 / 23 |
| Time per query | 2.9 ms | 2.7 ms |
The 23 cases are hand-labelled: a query, and the part a person actually meant by it. They run against the real index — the same 22,744 symbols and 15,124 footprints the app builds at startup — not against a fixture. A fixture would have made this test pass on the broken code, because the bug only exists at a scale where near-misses can outrank the answer.
Why the test refuses to pass quietly
A green gate is worth nothing if it would also be green with the thing it guards removed. This one was written to fail in four specific ways, because the previous arrangement was green throughout the entire period the search was returning nothing.
- It fails if the old ranking does not score worse. The pre-fix scoring function is frozen into the test and run alongside the new one. If a future edit makes them score the same, that is reported as a failure — because it means the test is no longer sensitive to the thing it exists to measure.
- It has a negative control that must come back red. The heater is still in there as a case that is required to be reported as wrong. If the checker ever calls it correct, the checker is broken and says so.
- It refuses to run on a corpus that is too small. If the index has fewer than 20,000 symbols or 14,000 footprints, the test hard-skips rather than passing. An empty corpus produces zero failures, and zero failures is the most dangerous green there is.
- It bounds the time per query, so the ranking cannot be made accurate by becoming slow enough to be unusable.
The lesson is not "we had a bug". It is that a suite of green checks told us nothing at all about whether the search worked, because not one of them ever asked it for a part by description.
That is the failure worth writing down. The code was reviewed, the tests passed, and the feature was completely non-functional for anyone who typed the way a person types. The only thing that would have caught it is the thing that eventually did: asking the running product for a part the way a user would, and looking at what came back.
How the index is built, and what is in it, is on the libraries page. The way the agent's own scoring is measured — including the rows it does badly on — is on the measurement page.