HAYSTACK

Haystack deepset Python RAG production framework.

Définition

Haystack 2.x (current stable 2024) key concepts : (1) Components : modular building blocks (Embedders OpenAI/SentenceTransformers/Cohere, Retrievers vector + BM25 + hybrid, Generators OpenAI/Anthropic/Llama, Readers QA extraction, Classifiers, Routers, Filters). (2) Pipelines : compose Components via add_component + connect, define dataflow Components graph. (3) Document Stores : abstractions over vector DBs (InMemoryDocumentStore for testing, ElasticsearchDocumentStore, OpenSearchDocumentStore, QdrantDocumentStore, PineconeDocumentStore, WeaviateDocumentStore, ChromaDocumentStore, Astra DB, OpenSearch, etc.), unified interface. (4) Document model : Document class (content + metadata + embedding + score), supports text + images + tables. (5) RAG patterns built-in : Indexing Pipeline (preprocess + embed + index documents), Query Pipeline (retrieve + generate), Hybrid retrieval (vector + BM25 + reranker), Extractive QA (highlight answer span source document), Generative QA (LLM generates synthesized answer). (6) Evaluation : built-in evaluators (Exact Match, F1, SAS Semantic Answer Similarity, faithfulness, answer relevance). deepset Cloud SaaS managed Haystack 2.x : visual pipeline builder + deployment + observability + evaluation. Customers : Airbus + Klarna + Spotify + Siemens + KraftHeinz + Etsy.

Origine

deepset fondee 2018 a Berlin par Milos Rusic + Malte Pietsch + Timo Moller ; Haystack v0.1 OSS release 2019 ; Series B $30M juin 2023 (Atomico lead) ; Haystack 2.0 release fevrier 2024 (major API redesign).

Exemple en contexte

Klarna enterprise customer support AI uses Haystack 2.x : Indexing Pipeline (~10M customer support ticket history + product documentation + policies indexed Elasticsearch + dense vectors via OpenAI embeddings) ; Query Pipeline (user question -> Hybrid Retrieval BM25 + dense + Cohere Reranker top-5 + GPT-4 generates answer with citations) ; deepset Cloud production deployment.

Termes liés

• LlamaIndex — alternative RAG-focused.