HNSW

What is it

In the context of modern Artificial Intelligence, HNSW (Hierarchical Navigable Small World) is the gold standard for proximity information retrieval. When dealing with RAG (Retrieval-Augmented Generation) systems, we don't look for exact keywords (as in old Google), but for meaning. For this search not to take minutes analyzing millions of documents, we need an ultra-efficient data structure. HNSW is that structure.

Imagine you have a library with a million books. If you want to find books about 'electric fleet management', a linear search would require reading the synopsis of every single book. HNSW creates a network of links between these books, allowing the system to quickly 'jump' between generic topics until reaching a specific niche in milliseconds. It is the technology underpinning most Vector Databases used by SMEs to create internal knowledge assistants.

How it works

HNSW is based on two fundamental concepts: hierarchy (Hierarchical) and 'small world' networks (Small World).

1. The Layered Structure (Hierarchy): The algorithm creates several data layers. The top layer has very few points (the most generic 'summaries'). As we go down, the density of points increases. It is comparable to a map: on the top layer, you only see the main highways; on the bottom layer, you see the dirt paths of your village.

2. Navigation: The search starts at the top. The algorithm identifies the closest point to your question in that simplified layer and 'jumps' to the next layer at that same location. It repeats this process, refining precision with each descent, until it finds the exact result in the database.

3. Small World Networks: Within each layer, data is linked so that any point can reach another with few 'jumps'. This prevents the system from getting lost in infinite calculations.

Technically, HNSW is an Approximate Nearest Neighbor (ANN) search algorithm. It sacrifices a minimal fraction of absolute precision (it might not find the 1st best result, but finds the 2nd or 3rd best) in exchange for speeds that are orders of magnitude higher than an exact search.

When to use

For an SME, HNSW is not something to be programmed from scratch, but a feature to be activated or configured in the database supporting the AI. It should be the choice when:

• Scale: When your knowledge base (PDFs, meeting transcripts, ticket history) exceeds a few thousand entries. Below that, simple methods may suffice, but HNSW is what ensures the system doesn't slow down as the company grows.
• Low Latency: If your customer support chatbot needs to respond in less than 2 seconds, HNSW is mandatory so that the 'context search' phase takes only milliseconds.
• RAG Systems: In any implementation where an LLM needs to consult private company data before generating a response.

Common errors

1. Static configuration: HNSW has parameters like 'M' (number of links) and 'efConstruction' (effort in index creation). Many companies leave the default values, which can result in a slow database or one that consumes too much RAM unnecessarily.
2. Ignoring memory cost: Unlike traditional indexes that sit on disk, HNSW ideally lives in RAM to be fast. Trying to run huge HNSW indexes on servers with little RAM will result in critical system failures.
3. Constant reindexing: Inserting new data into an HNSW index is fast, but changing the structure of how data is compared requires rebuilding the index from scratch, which can be computationally expensive.
4. Confusing it with a database: HNSW is the algorithm, not the place where you store data. You must ensure you choose a tool (such as Qdrant, Pinecone, or Weaviate) that implements HNSW robustly.

Practical example for an SME

A distribution company for electrical materials has a catalog with 150,000 technical references, installation manuals, and compatibility tables. They want to create an AI assistant for counter sales staff.

Without HNSW, when the salesperson asks 'Which circuit breaker is compatible with system X?', the system would have to probabilistically compare that question with each of the 150,000 manual paragraphs. This would take about 10 to 15 seconds per response.

With HNSW configured in the company's vector database:

1. The system receives the question.
2. In the top level of HNSW, it immediately realizes the question is about 'Protection Components' and not 'Lighting'.
3. It 'jumps' to the Circuit Breakers cluster.
4. In less than 50 milliseconds, it identifies the exact 3 paragraphs explaining compatibility.
5. The salesperson receives the answer on the screen almost instantly, allowing for a fluid service.

Frequently Asked Questions

Q: Is HNSW better than BM25? A: They are different tools. HNSW focuses on semantics (concepts), while BM25 is focused on exact keywords. Currently, the best SMEs use a 'Hybrid Search' that combines both.

Q: Can I use HNSW in traditional SQL databases? A: Some modern databases (like PostgreSQL with the pgvector extension) already support HNSW indexing, allowing you to keep company data and AI search capability in the same place.

Q: What is the difference between HNSW and Linear search? A: Linear search analyzes everything (very slow, 100% precision). HNSW analyzes only an intelligent fraction of the data (very fast, ~98-99% precision).

Q: Does infrastructure cost increase much with HNSW? A: The primary cost is RAM. Since the algorithm needs to keep the links between data in memory, you will need to invest in a server with more RAM than a typical document database.

• 01Instant retrieval of technical manuals in a database with 200,000 documents.
• 02Semantic search in a historical correspondence archive for a law firm.
• 03Similar product recommendation engine for a specialized retail e-commerce.
• 04Context retrieval phase for an HR chatbot consulting internal regulations.

Want to use HNSW in your company?

30 minutes, free, no commitment. We map where it fits.

Free AI diagnosis