Learn→Natural Language Processing Systems→Semantic Search (Dense Embeddings, ANN)→4 of 6

Natural Language Processing Systems • Semantic Search (Dense Embeddings, ANN)Hard⏱️ ~3 min

Semantic vs Keyword Search: When to Use Each and Hybrid Approaches

Semantic vs Keyword Search
Semantic search excels at understanding intent but struggles with exact matches. A search for "error code 404" might return general articles about web errors because the embedding captures "error concepts" rather than the specific code. Keyword search finds that exact string immediately. The reverse happens with conceptual queries: "why is my website broken" finds nothing with keywords but works perfectly with semantic search.
Most production systems use hybrid search: semantic results merged with keyword results via reciprocal rank fusion or re-ranking. The hybrid approach captures both exact matches and conceptual relevance.
Cost vs Quality
Better embedding models cost more. At scale, embedding costs can exceed storage and compute costs. Start with the best model, measure quality on your data, then test cheaper alternatives. If quality drops less than 10% while cost drops 80%, the cheaper model wins.
⚠️ Key Trade-off: Storage scales with dimensions times documents. 10M documents at 1536 dimensions uses 60GB. Reducing to 384 dims cuts to 15GB but may reduce search quality. Run A/B tests to quantify impact.
Latency vs Recall
ANN parameter tuning controls latency-recall. HNSW's ef_search determines candidates to explore: higher values find more neighbors but take longer. A system at 10ms might achieve 90% recall; 50ms might push to 98%. Decide based on result criticality - 90% recall means 1 in 10 relevant articles is missed.
When NOT to Use
Semantic search adds complexity. For structured queries (filter by date, category), traditional indices are faster. For exact lookups (SKU, order ID), keyword search is superior. Use semantic when users express intent in natural language.

💡 Key Takeaways

✓Semantic search struggles with exact matches ('error 404') while keyword search fails on conceptual queries ('why is my site broken') - hybrid combines both

✓Storage formula: 10M docs at 1536 dims = 60GB vectors; reducing to 384 dims cuts to 15GB but may reduce quality

✓HNSW ef_search parameter controls recall-latency: 10ms queries might achieve 90% recall, 50ms might reach 98%

✓Skip semantic search for structured queries (date filters) and exact lookups (SKU, order IDs) - traditional indices are better

📌 Interview Tips

1Give the hybrid search example: 'error code 404' needs keyword match, 'why is my website broken' needs semantic. Production systems use both.

2Calculate storage: dimensions times docs times 4 bytes. 1536 dims at 10M docs = 60GB. Show you can estimate infrastructure needs.

3Frame the when-not-to-use: semantic search adds cost and complexity. For SKU lookup or date filters, traditional DB is better.

← Back to Semantic Search (Dense Embeddings, ANN) Overview