用Spring AI＋Qdrant实现混合检索服务：Dense、Sparse、RRF与权限过滤
文章摘要单一Dense向量检索擅长语义理解却容易漏掉产品型号、合同编号和专业缩写纯关键词检索能够精确命中术语却难以理解自然语言。本文使用Spring Boot、Spring AI和Qdrant设计一个可复用的企业混合检索服务完成Dense与Sparse双路召回、租户权限过滤、RRF融合、去重、可观测日志和统一响应结构并给出后续接入Cross-Encoder重排的扩展点。一、我们要实现什么目标调用HybridSearchResponseresponsehybridSearchService.search(principal,五码身份如何支持白酒渠道控盘,10);内部链路用户问题 → 构造权限过滤 ├─ Dense向量召回Top50 └─ Sparse关键词召回Top50 → RRF融合 → 去重 → 返回Top10本篇重点是检索层不包含最终大模型生成。二、项目依赖pom.xml示意dependencyManagementdependenciesdependencygroupIdorg.springframework.ai/groupIdartifactIdspring-ai-bom/artifactIdversion2.0.0/versiontypepom/typescopeimport/scope/dependency/dependencies/dependencyManagementdependenciesdependencygroupIdorg.springframework.boot/groupIdartifactIdspring-boot-starter-web/artifactId/dependencydependencygroupIdorg.springframework.ai/groupIdartifactIdspring-ai-starter-model-deepseek/artifactId/dependencydependencygroupIdorg.springframework.ai/groupIdartifactIdspring-ai-starter-vector-store-qdrant/artifactId/dependencydependencygroupIdio.qdrant/groupIdartifactIdclient/artifactId/dependency/dependencies具体版本和Starter名称应以当前Spring AI与Qdrant Java SDK文档为准。三、为什么需要直接使用Qdrant ClientSpring AIVectorStore提供统一相似度检索接口适合普通Dense RAG。混合检索需要Named Dense VectorSparse VectorPrefetchRRF或DBSF FusionFormula Query多阶段查询。这些数据库特有能力不一定全部通过统一VectorStore接口暴露。因此可以采用Spring AI管理Embedding和上层RAG Qdrant Client实现高级检索业务层仍然通过自己的统一接口隔离数据库细节。四、定义检索主体publicrecordRetrievalPrincipal(StringuserId,StringtenantId,SetStringdepartmentIds,intsecurityLevel,StringpermissionVersion){}五、定义统一检索结果publicrecordHybridSearchHit(StringdocumentId,StringchunkId,Stringcontent,doublescore,intrank,SetStringmatchedRetrievers,MapString,Objectmetadata){}响应publicrecordHybridSearchResponse(StringrequestId,Stringquery,ListHybridSearchHithits,longdurationMs){}六、定义检索接口publicinterfaceHybridSearchService{HybridSearchResponsesearch(RetrievalPrincipalprincipal,Stringquery,intlimit);}七、Dense Embedding接口业务层不要直接依赖具体Provider。publicinterfaceQueryEmbeddingService{float[]embed(Stringquery);}Spring AI实现ServicepublicclassSpringAiQueryEmbeddingServiceimplementsQueryEmbeddingService{privatefinalEmbeddingModelembeddingModel;publicSpringAiQueryEmbeddingService(EmbeddingModelembeddingModel){this.embeddingModelembeddingModel;}Overridepublicfloat[]embed(Stringquery){EmbeddingResponseresponseembeddingModel.embedForResponse(List.of(query));returnresponse.getResults().getFirst().getOutput();}}不同Spring AI小版本的返回类型可能略有差异应以当前API为准。八、Sparse查询向量Sparse向量可以来自BM25服务SPLADEQdrant Inference自建关键词模型第三方Sparse Embedding API。统一接口publicrecordSparseVector(ListIntegerindices,ListFloatvalues){}publicinterfaceSparseEmbeddingService{SparseVectorembed(Stringquery);}如果暂时没有Sparse模型也可以先使用外部全文检索引擎随后在应用层RRF融合。九、构造权限过滤器ComponentpublicclassQdrantPermissionFilterFactory{publicFiltercreate(RetrievalPrincipalprincipal){// 伪代码具体Builder以Qdrant Java SDK为准returnFilter.newBuilder().addMust(match(tenant_id,principal.tenantId())).addMust(rangeLessOrEqual(security_level,principal.securityLevel())).addMust(match(status,EFFECTIVE)).addShould(matchAny(department_ids,principal.departmentIds())).addShould(match(visibility,PUBLIC_TENANT)).build();}}生产实现必须正确处理must should must_not minimum_should_match null字段 空部门集合十、双路检索服务ServicepublicclassQdrantHybridSearchServiceimplementsHybridSearchService{privatefinalQdrantClientqdrantClient;privatefinalQueryEmbeddingServicedenseService;privatefinalSparseEmbeddingServicesparseService;privatefinalQdrantPermissionFilterFactoryfilterFactory;publicQdrantHybridSearchService(QdrantClientqdrantClient,QueryEmbeddingServicedenseService,SparseEmbeddingServicesparseService,QdrantPermissionFilterFactoryfilterFactory){this.qdrantClientqdrantClient;this.denseServicedenseService;this.sparseServicesparseService;this.filterFactoryfilterFactory;}OverridepublicHybridSearchResponsesearch(RetrievalPrincipalprincipal,Stringquery,intlimit){longstartSystem.nanoTime();StringrequestIdUUID.randomUUID().toString();float[]densedenseService.embed(query);SparseVectorsparsesparseService.embed(query);FilterfilterfilterFactory.create(principal);ListRawSearchHitdenseHitssearchDense(dense,filter,50);ListRawSearchHitsparseHitssearchSparse(sparse,filter,50);ListHybridSearchHitfusedrrfFuse(denseHits,sparseHits,limit);longdurationMs(System.nanoTime()-start)/1_000_000;returnnewHybridSearchResponse(requestId,query,fused,durationMs);}}Qdrant本身支持Universal Query和服务端融合。如果Java SDK已经暴露对应能力应优先在数据库端完成双路Prefetch和Fusion减少网络往返。十一、RawSearchHitpublicrecordRawSearchHit(StringpointId,StringdocumentId,StringchunkId,Stringcontent,doublerawScore,intrank,Stringretriever,MapString,Objectmetadata){}十二、RRF融合实现privateListHybridSearchHitrrfFuse(ListRawSearchHitdenseHits,ListRawSearchHitsparseHits,intlimit){intk60;MapString,MutableFusionHitmergednewHashMap();addRrfScores(merged,denseHits,k);addRrfScores(merged,sparseHits,k);returnmerged.values().stream().sorted(Comparator.comparingDouble(MutableFusionHit::score).reversed()).limit(limit).map(MutableFusionHit::toResult).toList();}累积分数privatevoidaddRrfScores(MapString,MutableFusionHitmerged,ListRawSearchHithits,intk){for(RawSearchHithit:hits){doublescore1.0/(khit.rank());merged.compute(hit.chunkId(),(key,current)-{MutableFusionHittargetcurrentnull?MutableFusionHit.from(hit):current;target.addScore(score);target.addRetriever(hit.retriever());returntarget;});}}十三、为什么按chunk_id去重Dense和Sparse可能返回同一个Chunk。如果不去重同一段内容进入上下文两次 → 浪费Token → 放大某个证据权重 → 结果缺乏多样性去重Key可以是chunk_id但还要处理近重复Chunk例如同一段出现在不同版本文档中。可以进一步使用content_hash source_version十四、服务端RRF更推荐如果数据库支持原生FusionDense Prefetch Top50 Sparse Prefetch Top50 → RRF Query → Top10优势一次网络请求数据库内部执行少传输候选结果更容易扩展多阶段查询统一监控。应用层RRF适合组合不同搜索系统数据库能力不一致需要自定义逻辑快速验证。十五、接入Cross-Encoder定义publicinterfaceReranker{ListHybridSearchHitrerank(Stringquery,ListHybridSearchHitcandidates,intlimit);}调用RRF Top30 → Reranker Top10不要把50或100个长Chunk全部发给第三方API先控制长度和候选数。十六、查询类型动态路由精确编号HT-2026-001 SKU-A892Sparse更重要。概念问题渠道费用为什么无法形成闭环Dense更重要。可以分类publicenumQueryType{EXACT,SEMANTIC,MIXED}策略EXACTSparse Top80Dense Top20 SEMANTICDense Top80Sparse Top20 MIXED两路Top50如果使用RRF不同路权重可通过Weighted RRF或候选数量间接调整。十七、可观测性记录request_id query_type tenant_id dense_duration_ms sparse_duration_ms fusion_duration_ms dense_candidate_count sparse_candidate_count overlap_count final_count permission_version不要在普通日志中记录完整文档正文。十八、错误处理Dense失败可降级到Sparse但要标记retrieval_mode SPARSE_ONLYSparse失败可降级到Dense。权限服务失败必须Fail Closed不能取消过滤继续搜索。两路都失败返回稳定错误码RETRIEVAL_UNAVAILABLE十九、测试数据至少包含产品型号合同编号中文同义词缩写错别字新旧版本跨租户同名文档表格Chunk长问题。评测Dense Recall10 Sparse Recall10 Hybrid Recall10 MRR nDCG P95延迟二十、生产优化Dense和Sparse并行执行Embedding查询缓存限制最大查询长度批量Embedding服务端FusionPayload字段索引候选数量动态调整Reranker批处理超时和熔断权限过滤自动化测试。总结混合检索服务的核心链路是可信权限上下文 → Dense与Sparse召回 → RRF融合 → 去重 → 可选重排 → 统一结果Spring AI负责模型与RAG上层集成Qdrant高级查询能力负责检索业务层再通过统一接口隔离底层实现。这样既能获得混合检索质量也能保留后续更换数据库和重排器的空间。
)
)
)
)
