ParentDocumentRetriever Class — langchain Architecture

Architecture documentation for the ParentDocumentRetriever class in parent_document_retriever.py from the langchain codebase.

Class python

Entity Profile

Dependency Diagram

graph TD
  eb3f3ce7_75a2_f5aa_7781_56a2f86f1973["ParentDocumentRetriever"]
  c7ceffe2_56b9_b96d_5de5_8e679b778e9a["MultiVectorRetriever"]
  eb3f3ce7_75a2_f5aa_7781_56a2f86f1973 -->|extends| c7ceffe2_56b9_b96d_5de5_8e679b778e9a
  1307a6c2_d463_f8f0_774b_171b2dce9e8d["parent_document_retriever.py"]
  eb3f3ce7_75a2_f5aa_7781_56a2f86f1973 -->|defined in| 1307a6c2_d463_f8f0_774b_171b2dce9e8d
  913129df_8767_b41b_518c_6c6d98c8bd9d["_split_docs_for_adding()"]
  eb3f3ce7_75a2_f5aa_7781_56a2f86f1973 -->|method| 913129df_8767_b41b_518c_6c6d98c8bd9d
  92bed5c9_a76d_c063_a022_68683b1b371d["add_documents()"]
  eb3f3ce7_75a2_f5aa_7781_56a2f86f1973 -->|method| 92bed5c9_a76d_c063_a022_68683b1b371d
  c068c021_6d63_9a46_4f44_2d7d7141278d["aadd_documents()"]
  eb3f3ce7_75a2_f5aa_7781_56a2f86f1973 -->|method| c068c021_6d63_9a46_4f44_2d7d7141278d

Relationship Graph

Source Code

libs/langchain/langchain_classic/retrievers/parent_document_retriever.py lines 11–176

class ParentDocumentRetriever(MultiVectorRetriever):
    """Retrieve small chunks then retrieve their parent documents.

    When splitting documents for retrieval, there are often conflicting desires:

    1. You may want to have small documents, so that their embeddings can most
        accurately reflect their meaning. If too long, then the embeddings can
        lose meaning.
    2. You want to have long enough documents that the context of each chunk is
        retained.

    The ParentDocumentRetriever strikes that balance by splitting and storing
    small chunks of data. During retrieval, it first fetches the small chunks
    but then looks up the parent IDs for those chunks and returns those larger
    documents.

    Note that "parent document" refers to the document that a small chunk
    originated from. This can either be the whole raw document OR a larger
    chunk.

    Examples:
        ```python
        from langchain_chroma import Chroma
        from langchain_community.embeddings import OpenAIEmbeddings
        from langchain_text_splitters import RecursiveCharacterTextSplitter
        from langchain_classic.storage import InMemoryStore

        # This text splitter is used to create the parent documents
        parent_splitter = RecursiveCharacterTextSplitter(
            chunk_size=2000, add_start_index=True
        )
        # This text splitter is used to create the child documents
        # It should create documents smaller than the parent
        child_splitter = RecursiveCharacterTextSplitter(
            chunk_size=400, add_start_index=True
        )
        # The VectorStore to use to index the child chunks
        vectorstore = Chroma(embedding_function=OpenAIEmbeddings())
        # The storage layer for the parent documents
        store = InMemoryStore()

        # Initialize the retriever
        retriever = ParentDocumentRetriever(
            vectorstore=vectorstore,
            docstore=store,
            child_splitter=child_splitter,
            parent_splitter=parent_splitter,
        )
        ```
    """

    child_splitter: TextSplitter
    """The text splitter to use to create child documents."""

    """The key to use to track the parent id. This will be stored in the
    metadata of child documents."""
    parent_splitter: TextSplitter | None = None
    """The text splitter to use to create parent documents.
    If none, then the parent documents will be the raw documents passed in."""

    child_metadata_fields: Sequence[str] | None = None
    """Metadata fields to leave in child documents. If `None`, leave all parent document
        metadata.
    """

    def _split_docs_for_adding(
        self,
        documents: list[Document],
        ids: list[str] | None = None,
        *,
        add_to_docstore: bool = True,
    ) -> tuple[list[Document], list[tuple[str, Document]]]:
        if self.parent_splitter is not None:
            documents = self.parent_splitter.split_documents(documents)
        if ids is None:
            doc_ids = [str(uuid.uuid4()) for _ in documents]
            if not add_to_docstore:
                msg = "If IDs are not passed in, `add_to_docstore` MUST be True"
                raise ValueError(msg)
        else:
            if len(documents) != len(ids):