langchain Architecture Docs
Home / File/ api.py — langchain Source File

api.py — langchain Source File

Architecture documentation for api.py, a python file in the langchain codebase. 12 imports, 0 dependents.

File python CoreAbstractions Serialization 12 imports 16 functions 3 classes

Entity Profile

Dependency Diagram

graph LR
  203188c0_72d6_6932_bc21_edf25c4c00ef["api.py"]
  ca3eea8c_ddf5_4ba7_a40c_5ed2287c91fa["hashlib"]
  203188c0_72d6_6932_bc21_edf25c4c00ef --> ca3eea8c_ddf5_4ba7_a40c_5ed2287c91fa
  7025b240_fdc3_cf68_b72f_f41dac94566b["json"]
  203188c0_72d6_6932_bc21_edf25c4c00ef --> 7025b240_fdc3_cf68_b72f_f41dac94566b
  8dfa0cac_d802_3ccd_f710_43a5e70da3a5["uuid"]
  203188c0_72d6_6932_bc21_edf25c4c00ef --> 8dfa0cac_d802_3ccd_f710_43a5e70da3a5
  0c635125_6987_b8b3_7ff7_d60249aecde7["warnings"]
  203188c0_72d6_6932_bc21_edf25c4c00ef --> 0c635125_6987_b8b3_7ff7_d60249aecde7
  436f77bc_653d_0edb_555c_c2679d5a59ac["itertools"]
  203188c0_72d6_6932_bc21_edf25c4c00ef --> 436f77bc_653d_0edb_555c_c2679d5a59ac
  8e2034b7_ceb8_963f_29fc_2ea6b50ef9b3["typing"]
  203188c0_72d6_6932_bc21_edf25c4c00ef --> 8e2034b7_ceb8_963f_29fc_2ea6b50ef9b3
  bc0c97a0_93eb_cfbc_6044_05825b687ba4["langchain_core.document_loaders.base"]
  203188c0_72d6_6932_bc21_edf25c4c00ef --> bc0c97a0_93eb_cfbc_6044_05825b687ba4
  c554676d_b731_47b2_a98f_c1c2d537c0aa["langchain_core.documents"]
  203188c0_72d6_6932_bc21_edf25c4c00ef --> c554676d_b731_47b2_a98f_c1c2d537c0aa
  75137834_4ba7_dc43_7ec5_182c05eceedf["langchain_core.exceptions"]
  203188c0_72d6_6932_bc21_edf25c4c00ef --> 75137834_4ba7_dc43_7ec5_182c05eceedf
  c1c8dd57_1ebb_bad3_6e65_32e0aa3099ae["langchain_core.indexing.base"]
  203188c0_72d6_6932_bc21_edf25c4c00ef --> c1c8dd57_1ebb_bad3_6e65_32e0aa3099ae
  d55af636_303c_0eb6_faee_20d89bd952d5["langchain_core.vectorstores"]
  203188c0_72d6_6932_bc21_edf25c4c00ef --> d55af636_303c_0eb6_faee_20d89bd952d5
  cfe2bde5_180e_e3b0_df2b_55b3ebaca8e7["collections.abc"]
  203188c0_72d6_6932_bc21_edf25c4c00ef --> cfe2bde5_180e_e3b0_df2b_55b3ebaca8e7
  style 203188c0_72d6_6932_bc21_edf25c4c00ef fill:#6366f1,stroke:#818cf8,color:#fff

Relationship Graph

Source Code

"""Module contains logic for indexing documents into vector stores."""

from __future__ import annotations

import hashlib
import json
import uuid
import warnings
from itertools import islice
from typing import (
    TYPE_CHECKING,
    Any,
    Literal,
    TypedDict,
    TypeVar,
    cast,
)

from langchain_core.document_loaders.base import BaseLoader
from langchain_core.documents import Document
from langchain_core.exceptions import LangChainException
from langchain_core.indexing.base import DocumentIndex, RecordManager
from langchain_core.vectorstores import VectorStore

if TYPE_CHECKING:
    from collections.abc import (
        AsyncIterable,
        AsyncIterator,
        Callable,
        Iterable,
        Iterator,
        Sequence,
    )

# Magic UUID to use as a namespace for hashing.
# Used to try and generate a unique UUID for each document
# from hashing the document content and metadata.
NAMESPACE_UUID = uuid.UUID(int=1984)


T = TypeVar("T")


def _hash_string_to_uuid(input_string: str) -> str:
    """Hashes a string and returns the corresponding UUID."""
    hash_value = hashlib.sha1(
        input_string.encode("utf-8"), usedforsecurity=False
    ).hexdigest()
    return str(uuid.uuid5(NAMESPACE_UUID, hash_value))


_WARNED_ABOUT_SHA1: bool = False


def _warn_about_sha1() -> None:
    """Emit a one-time warning about SHA-1 collision weaknesses."""
    # Global variable OK in this case
    global _WARNED_ABOUT_SHA1  # noqa: PLW0603
    if not _WARNED_ABOUT_SHA1:
        warnings.warn(
// ... (889 more lines)

Domain

Subdomains

Functions

Classes

Dependencies

  • collections.abc
  • hashlib
  • itertools
  • json
  • langchain_core.document_loaders.base
  • langchain_core.documents
  • langchain_core.exceptions
  • langchain_core.indexing.base
  • langchain_core.vectorstores
  • typing
  • uuid
  • warnings

Source

Frequently Asked Questions

What does api.py do?
api.py is a source file in the langchain codebase, written in python. It belongs to the CoreAbstractions domain, Serialization subdomain.
What functions are defined in api.py?
api.py defines 16 function(s): _abatch, _adelete, _batch, _calculate_hash, _deduplicate_in_order, _delete, _get_document_with_hash, _get_source_id_assigner, _hash_nested_dict, _hash_string, and 6 more.
What does api.py depend on?
api.py imports 12 module(s): collections.abc, hashlib, itertools, json, langchain_core.document_loaders.base, langchain_core.documents, langchain_core.exceptions, langchain_core.indexing.base, and 4 more.
Where is api.py in the architecture?
api.py is located at libs/core/langchain_core/indexing/api.py (domain: CoreAbstractions, subdomain: Serialization, directory: libs/core/langchain_core/indexing).

Analyze Your Own Codebase

Get architecture documentation, dependency graphs, and domain analysis for your codebase in minutes.

Try Supermodel Free