load_and_split() — langchain Function Reference

Architecture documentation for the load_and_split() function in base.py from the langchain codebase.

Function python DocumentProcessing DataLoaders calls 1

Entity Profile

DocumentProcessing→ DataLoaders→ load_and_split() — langchain Function Reference

Dependency Diagram

graph TD
  8fbd76b5_602e_6d85_e949_58422d1e40b5["load_and_split()"]
  a03328f5_8313_3e81_e0e5_6b8aa55a53b8["BaseLoader"]
  8fbd76b5_602e_6d85_e949_58422d1e40b5 -->|defined in| a03328f5_8313_3e81_e0e5_6b8aa55a53b8
  e0a3f14c_d483_262a_6986_006241b8be81["load()"]
  8fbd76b5_602e_6d85_e949_58422d1e40b5 -->|calls| e0a3f14c_d483_262a_6986_006241b8be81
  style 8fbd76b5_602e_6d85_e949_58422d1e40b5 fill:#6366f1,stroke:#818cf8,color:#fff

Relationship Graph

Source Code

libs/core/langchain_core/document_loaders/base.py lines 53–87

    def load_and_split(
        self, text_splitter: TextSplitter | None = None
    ) -> list[Document]:
        """Load `Document` and split into chunks. Chunks are returned as `Document`.

        !!! danger

            Do not override this method. It should be considered to be deprecated!

        Args:
            text_splitter: `TextSplitter` instance to use for splitting documents.

                Defaults to `RecursiveCharacterTextSplitter`.

        Raises:
            ImportError: If `langchain-text-splitters` is not installed and no
                `text_splitter` is provided.

        Returns:
            List of `Document` objects.
        """
        if text_splitter is None:
            if not _HAS_TEXT_SPLITTERS:
                msg = (
                    "Unable to import from langchain_text_splitters. Please specify "
                    "text_splitter or install langchain_text_splitters with "
                    "`pip install -U langchain-text-splitters`."
                )
                raise ImportError(msg)

            text_splitter_: TextSplitter = RecursiveCharacterTextSplitter()
        else:
            text_splitter_ = text_splitter
        docs = self.load()
        return text_splitter_.split_documents(docs)