Qdrant is attempting to reduce the architectural complexity of hybrid search by eliminating the need for manual sparse vector pipelines. With the release of Qdrant 1.20, the vector search engine now enables keyword ranking directly on stored text using the BM25 algorithm. This development allows engineering teams to combine semantic and keyword search results in a single query without generating separate keyword embeddings or manually maintaining data statistics. By automating these processes, the company aims to simplify how enterprises manage retrieval-augmented generation and complex search workloads.
Automating Hybrid Search via BM25
The Qdrant 1.20 update introduces the ability to rank keyword matches directly on the full-text index, a capability that previously only allowed for filtering. By utilizing BM25—the industry standard for keyword search engines—the platform can now derive necessary statistics directly from the stored data. Previously, achieving hybrid search required an additional inference step to convert text into embeddings and necessitated that teams manually supply and update data statistics. This manual process often led to outdated information as datasets evolved. The new release positions keyword ranking as a primitive that can be combined with dense retrieval in a single query. This change is designed to assist teams transitioning from traditional keyword search engines while removing the burden of maintaining a separate sparse encoding pipeline.
Enhancing Retrieval Precision and Infrastructure
Beyond search mechanics, Qdrant 1.20 addresses specific failure modes in semantic search, such as the inability to match exact product codes, error numbers, or legal clauses. By ranking keyword matches alongside semantic results, the engine aims to prevent AI assistants from retrieving "lookalike" documents when exact matches are required. The update also improves software code search by recognizing variations like "user name," "getUserName," and "get_user_name." On the infrastructure side, the release introduces TurboQuant compression, which the company claims can reduce vector storage by half or more with minimal accuracy loss. Additional technical improvements include faster filtering on nested data, reduced memory usage for large collections, and expanded backup support for Google Cloud Storage and Azure Blob Storage.
Key Takeaways
- Qdrant 1.20 implements BM25 keyword ranking directly on the full-text index, removing the requirement for separate sparse encoding pipelines.
- The update automates the derivation of ranking statistics from stored data, ensuring they remain current as documents are added or modified.
- New TurboQuant compression options are available to potentially reduce vector storage requirements by 50% or more.
TechInsyte's Take
In our view, Qdrant is making a calculated move to lower the barrier to entry for enterprise-grade hybrid search. By integrating BM25 directly into the vector engine, they are addressing a significant pain point: the operational overhead of managing dual pipelines for dense and sparse vectors. This shift suggests that the industry is moving away from "pure" semantic search toward a more pragmatic, multi-modal retrieval model where exact keyword matching remains a critical safeguard against the hallucinations or "blurring" inherent in vector embeddings. For CIOs, this represents a potential reduction in both engineering complexity and the specialized compute required for constant embedding generation.
Questions & Answers
How does the new BM25 integration affect the engineering workload for hybrid search?
It removes the need for engineers to build and maintain a separate sparse encoding pipeline. Because the system now derives statistics from the data itself, teams no longer need to manually update statistics as their datasets change.
Can existing sparse vector workflows be maintained in Qdrant 1.20?
Yes. The company stated that teams utilizing specialized AI models for keyword matching can continue to use the earlier method involving sparse vectors.
What specific technical problem does this solve for AI-driven assistants?
It addresses the tendency of semantic search to return "lookalike" results instead of exact matches. By ranking keyword matches alongside semantic ones, the engine can prioritize exact terms like product codes or contract clauses.
What are the primary infrastructure benefits of the TurboQuant compression?
TurboQuant is designed to cut vector storage by half or more with little to no loss in accuracy, while also supporting lower memory usage for very large collections.
Source: Qdrant
