Metadata Filtering

Alternative Names

• Self Query Retriever

Required Graph Shape

No specific Graph Shape required. Requires a graph that contains properties or relationships that can be filtered on.

Description

Metadata information that is present in the underlying database is retrieved from the user question. The user question is embedded using the same embedder that has been used before. A vector similarity search is executed only on the embeddings of nodes that meet the filtering requirements.

Usage

It is always useful to have structured data that can help filtering the dataset. If applied correctly, metadata filtering will lead to less hallucinations within the answers. However, if we only have unstructured data without metadata, the metadata extraction first needs to be executed, e.g. by an LLM. This can lead to inconsistencies and relevant documents being excluded from the search.
Metadata filtering can happen before or after the vector search. Filtering can also happen pattern-based where nodes are not only filtered based on given values but also on existing or non-existing patterns.

Further reading

• Implementing advanced RAG strategies with Neo4j (November 2023)
• Graph-based Metadata Filtering to Improve Vector Search in RAG Applications (Tomaz Bratanic, April 2024)

Existing Implementations

• Langchain: Neo4jVector: Metadata Filtering
• Langchain Retrievers: Self-querying