feat: Add feast rag retriver functionality #5405
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fiona-Waters
changed the title
Fiona-Waters
force-pushed
the
ragretriever
branch
4 times, most recently
from
0abf934 to
fc99dd0
Compare
Fiona-Waters
force-pushed
the
ragretriever
branch
14 times, most recently
from
069fe93 to
6873c84
Compare
Fiona-Waters
force-pushed
the
ragretriever
branch
5 times, most recently
from
d986e6b to
ec5ba98
Compare
Signed-off-by: Fiona Waters <[email protected]>
…ation Signed-off-by: Fiona Waters <[email protected]>
Fiona-Waters
force-pushed
the
ragretriever
branch
from
ec5ba98 to
40dfde0
Compare
ntkathole
reviewed
Jun 6, 2025
| @@ -163,7 +168,10 @@ ci = [ | |||
| "types-setuptools", | |||
| "types-tabulate", | |||
| "virtualenv<20.24.2", | |||
| "feast[aws, azure, cassandra, clickhouse, couchbase, delta, docling, duckdb, elasticsearch, faiss, gcp, ge, go, grpcio, hazelcast, hbase, ibis, ikv, k8s, mcp, milvus, mssql, mysql, opentelemetry, spark, trino, postgres, pytorch, qdrant, redis, singlestore, snowflake, sqlite_vec]" | |||
| "transformers>=4.36.0", | |||
There was a problem hiding this comment.
This can be removed from here, adding rag to feast[..] pulls those dependencies.
ntkathole
reviewed
Jun 6, 2025
| @@ -107,6 +107,11 @@ postgres = ["psycopg[binary,pool]==3.2.5"] | |||
| postgres-c = ["psycopg[c,pool]==3.2.5"] | |||
| pytorch = ["torch==2.2.2", "torchvision>=0.17.2"] | |||
| qdrant = ["qdrant-client>=1.12.0"] | |||
| rag = [ | |||
There was a problem hiding this comment.
rag can also be included in nlp group https://github.com/feast-dev/feast/blob/master/pyproject.toml#L168
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
This PR adds a FeastRagRetriever that inherits from the HuggingFace Transformers RagRetriever class. It allows for integration with feast functionality allowing RAG to be performed more easily using the Feature Store.
Low Level Design created by @jyejare
The following has been added:
rag_retriever.py (New file)
This module implements a custom RAG retriever by combining Feast feature store capabilities with transformer-based models. The implementation provides:
vector_store.py (New file)
This module implements an abstract VectorStore class, and a FeastVectorStore class that inherits from it. The FeastVectorStore takes in a FeatureStore, FeatureView and features using these to query the store via the existing retrieve_online_documents_v2 function.
setup.py (Additions to existing file)
Adding entries for RAG dependencies allowing for installation with
pip install feast[rag].init.py (Additions to existing file)
Adding entries for new classes added in rag_retriever.py and vector_store.py
pyproject.toml (Additions to existing file)
The dependencies required for the RAG implementation have been added here.
The relevant dependencies have also been added to the relevant requirements.txt files.
test_rag.py (New file)
Unit tests to cover the added functionality.
Which issue(s) this PR fixes:
Related to #5391 but get_top_docs has not been implemented here as the FeastIndex is a dummy index - the retrieval functionality exists in the FeastRagRetriever via the FeastVector store which uses
retrieve_online_documents_v2.Misc