_generate_batches() — langchain Function Reference

Architecture documentation for the _generate_batches() function in qdrant.py from the langchain codebase.

Function python LangChainCore MessageInterface calls 2 called by 1

Entity Profile

LangChainCore→ MessageInterface→ _generate_batches() — langchain Function Reference

Dependency Diagram

graph TD
  e5f5df10_5f00_b3ee_ed57_3bff5ff2fd2d["_generate_batches()"]
  671b47a0_cdd3_a89d_e90f_0631a4bd67d3["QdrantVectorStore"]
  e5f5df10_5f00_b3ee_ed57_3bff5ff2fd2d -->|defined in| 671b47a0_cdd3_a89d_e90f_0631a4bd67d3
  c81b5f54_4d7b_ff69_0cd1_397bf595fdc2["add_texts()"]
  c81b5f54_4d7b_ff69_0cd1_397bf595fdc2 -->|calls| e5f5df10_5f00_b3ee_ed57_3bff5ff2fd2d
  b6a0e093_c39e_398d_bfa1_46505881abd6["_build_vectors()"]
  e5f5df10_5f00_b3ee_ed57_3bff5ff2fd2d -->|calls| b6a0e093_c39e_398d_bfa1_46505881abd6
  52a54952_99f4_c33d_5753_3404574980e8["_build_payloads()"]
  e5f5df10_5f00_b3ee_ed57_3bff5ff2fd2d -->|calls| 52a54952_99f4_c33d_5753_3404574980e8
  style e5f5df10_5f00_b3ee_ed57_3bff5ff2fd2d fill:#6366f1,stroke:#818cf8,color:#fff

Relationship Graph

Source Code

libs/partners/qdrant/langchain_qdrant/qdrant.py lines 1038–1071

    def _generate_batches(
        self,
        texts: Iterable[str],
        metadatas: list[dict] | None = None,
        ids: Sequence[str | int] | None = None,
        batch_size: int = 64,
    ) -> Generator[tuple[list[str | int], list[models.PointStruct]], Any, None]:
        texts_iterator = iter(texts)
        metadatas_iterator = iter(metadatas or [])
        ids_iterator = iter(ids or [uuid.uuid4().hex for _ in iter(texts)])

        while batch_texts := list(islice(texts_iterator, batch_size)):
            batch_metadatas = list(islice(metadatas_iterator, batch_size)) or None
            batch_ids = list(islice(ids_iterator, batch_size))
            points = [
                models.PointStruct(
                    id=point_id,
                    vector=vector,
                    payload=payload,
                )
                for point_id, vector, payload in zip(
                    batch_ids,
                    self._build_vectors(batch_texts),
                    self._build_payloads(
                        batch_texts,
                        batch_metadatas,
                        self.content_payload_key,
                        self.metadata_payload_key,
                    ),
                    strict=False,
                )
            ]

            yield batch_ids, points