_agenerate_rest_batches() — langchain Function Reference

Architecture documentation for the _agenerate_rest_batches() function in vectorstores.py from the langchain codebase.

Function python CoreAbstractions RunnableInterface calls 2 called by 1

Entity Profile

CoreAbstractions→ RunnableInterface→ _agenerate_rest_batches() — langchain Function Reference

Dependency Diagram

graph TD
  7268a9dd_664a_8cff_f4f9_a345b28dafd3["_agenerate_rest_batches()"]
  bf62db79_4217_463c_798f_6f8528ed0d6e["Qdrant"]
  7268a9dd_664a_8cff_f4f9_a345b28dafd3 -->|defined in| bf62db79_4217_463c_798f_6f8528ed0d6e
  675dc134_0c09_bf23_f892_4a4af82a4549["aadd_texts()"]
  675dc134_0c09_bf23_f892_4a4af82a4549 -->|calls| 7268a9dd_664a_8cff_f4f9_a345b28dafd3
  b81b33c0_c789_80eb_7445_6fda6aa905e0["_aembed_texts()"]
  7268a9dd_664a_8cff_f4f9_a345b28dafd3 -->|calls| b81b33c0_c789_80eb_7445_6fda6aa905e0
  818b1334_70e9_80aa_1f55_1c72ba7d484c["_build_payloads()"]
  7268a9dd_664a_8cff_f4f9_a345b28dafd3 -->|calls| 818b1334_70e9_80aa_1f55_1c72ba7d484c
  style 7268a9dd_664a_8cff_f4f9_a345b28dafd3 fill:#6366f1,stroke:#818cf8,color:#fff

Relationship Graph

Source Code

libs/partners/qdrant/langchain_qdrant/vectorstores.py lines 2238–2277

    async def _agenerate_rest_batches(
        self,
        texts: Iterable[str],
        metadatas: list[dict] | None = None,
        ids: Sequence[str] | None = None,
        batch_size: int = 64,
    ) -> AsyncGenerator[tuple[list[str], list[models.PointStruct]], None]:
        texts_iterator = iter(texts)
        metadatas_iterator = iter(metadatas or [])
        ids_iterator = iter(ids or [uuid.uuid4().hex for _ in iter(texts)])
        while batch_texts := list(islice(texts_iterator, batch_size)):
            # Take the corresponding metadata and id for each text in a batch
            batch_metadatas = list(islice(metadatas_iterator, batch_size)) or None
            batch_ids = list(islice(ids_iterator, batch_size))

            # Generate the embeddings for all the texts in a batch
            batch_embeddings = await self._aembed_texts(batch_texts)

            points = [
                models.PointStruct(
                    id=point_id,
                    vector=vector  # type: ignore[arg-type]
                    if self.vector_name is None
                    else {self.vector_name: vector},
                    payload=payload,
                )
                for point_id, vector, payload in zip(
                    batch_ids,
                    batch_embeddings,
                    self._build_payloads(
                        batch_texts,
                        batch_metadatas,
                        self.content_payload_key,
                        self.metadata_payload_key,
                    ),
                    strict=False,
                )
            ]

            yield batch_ids, points