When a website’s search returns poor results, replacing the search tool feels like the obvious solution.

Sometimes it is the right one. Default website search can be extremely limited, and there are much better ways to interpret a query, rank results and help people narrow down a large collection.

But a new search engine cannot make sense of information that the organisation itself has never made consistent. It cannot reliably distinguish two things that have been given the same name, connect two things that have different names, or fill in important information that was never recorded.

Often, the search is not the real problem. It is simply the place where problems with the content become visible.

Search can only work with what it has

Imagine a collection containing several thousand technical records, accumulated over a number of years and maintained by different people.

Some records use a formal name, while others use the name most people actually know. A subject might be recorded as a category on one record, mentioned in the description on another and omitted completely from a third. Dates appear in different formats. Two departments use different words for the same thing. Uploaded documents contain useful detail that never makes it into the website’s visible fields.

None of this is particularly unusual. It is what happens when information grows around the needs of an organisation rather than being created all at once for a search system.

A basic keyword search will expose every inconsistency. A more advanced search may disguise some of them, but it cannot safely invent the missing structure.

Before changing the technology, we need to understand the material it is being asked to search.

A search result starts with the record

Good search depends on records being sufficiently clear and consistent.

That does not mean every item needs dozens of painstakingly completed fields. It means each kind of record should carry the information people genuinely need in order to find it, recognise it and decide whether it is useful.

For a resource library, that might include:

• a clear title;
• a short description;
• subject and intended audience;
• format and language;
• publication and review dates;
• ownership or provenance; and
• alternative terms someone might use to describe it.

For a directory of people, it might instead include skills, experience, location, availability, preferences and the status of any required checks or approvals.

The right fields follow from the decisions users need to make. They should not be added simply because the content management system makes it easy to add another field.

This distinction matters. More metadata does not automatically produce better search. Unclear, duplicated or inconsistently completed metadata can create a more elaborate version of the same problem.

Your users may not speak your language

Organisations develop their own vocabulary. It is efficient internally because everyone gradually learns what the abbreviations, programme names and formal classifications mean.

People outside the organisation have not had that training.

They may search for an outdated name, a common misnomer, a regional variation, an abbreviation or a description of the problem rather than the official name of its solution. Experts and occasional users may use entirely different language for the same subject.

That does not make the user’s query wrong.

We have solved this problem for clients with large collections of technical information intended to help non-specialists make informed decisions. In one case, the source material was necessarily detailed and used precise industry terminology. The eventual users did not always know that terminology and should not have needed to learn it before they could find useful information.

Alternative names, misnomers and related terms therefore became part of the search design. The authoritative records remained authoritative, but the route into them reflected the language people actually used.

This can often be handled without rewriting every record. A controlled set of synonyms and alternative terms can connect user language to organisational language while preserving both.

Categories help until they don’t

Categorisation is another common source of false confidence.

A tidy list of categories looks like structure. Over time, though, people add near-duplicates, create a category for a single item or disagree about where something belongs. Internal management categories are exposed as public filters even though they mean very little to users.

The result is a system that is technically organised but practically difficult to search.

On another project, a client needed a skills-matching service that could help people find volunteers with appropriate experience. Too little structure would make useful matching impossible. Too much unrestricted structure would leave administrators managing duplicates and volunteers choosing between categories whose differences were not clear.

The answer was not simply to add more categories. It was to understand which distinctions mattered, control how they were introduced and supplement explicit categories with other searchable evidence where appropriate.

A useful taxonomy is a maintained organisational asset. It needs ownership, rules for change and a way to accommodate the words users bring with them.

Some content problems can be corrected automatically

Finding problems in the source information does not mean every record must be corrected by hand.

Well-defined normalisation can do a great deal of useful work. It can make dates and locations consistent, remove accidental formatting differences, connect known abbreviations to full terms and bring data from several sources into a shared structure.

Uploaded documents can be processed to extract searchable text. Existing classifications can be mapped to an agreed vocabulary. Duplicate candidates can be flagged for review. Suggested tags can be generated from the content of a record.

These processes increase what we sometimes call the record’s search footprint: the reliable information a search can use to decide that a result may be relevant.

The important word is reliable.

Automatically extracted or inferred information should not quietly become authoritative fact. The system should retain the distinction between what the organisation knows, what it has normalised and what a tool has suggested. The higher the consequence of an error, the more visible that distinction and its review process need to be.

AI gives us more ways to extract, classify and connect information, but it does not remove this responsibility. A confident suggestion is still a suggestion.

Search analytics reveal content problems

One of the simplest ways to improve onsite search is to pay attention to how it fails.

Anonymous search reporting can show:

• queries that return no results;
• very broad queries that return too many;
• results that are displayed but never selected;
• repeated searches in which the user changes their wording; and
• frequent searches for information the organisation does not currently publish.

This is useful content research based on real behaviour.

If people repeatedly search for something you already have but use a different name, the answer may be a new synonym. If they find the record but never select it, its title or description may not make its relevance clear. If a popular query has no suitable answer, that may identify a genuine gap in the content.

Search should therefore be treated as something to observe and improve, not a component that is installed and considered finished.

What to inspect before replacing your search

There are good reasons to replace or extend a website’s search technology. It may not index the fields you need, understand synonyms, respect complex permissions, support several languages or rank results in a useful way.

Before procuring the replacement, it is worth answering a few questions:

1. What kinds of record are being searched?
2. What does a user need to know in order to choose a result?
3. Is that information recorded consistently?
4. Which terms do users employ that do not appear in the records?
5. Are internal categories being mistaken for useful public filters?
6. What useful information is trapped inside uploaded documents or disconnected systems?
7. Which inconsistencies can be normalised safely?
8. Who will review failed searches and improve the system after launch?

The answers will help define what the search technology actually needs to do. They will also identify the content and governance work without which even an excellent search product will disappoint.

The search box is where a user asks the organisation a question. If the answer is poor, inspect the machinery—but inspect the organisation’s information too.