_astream() — langchain Function Reference

Architecture documentation for the _astream() function in base.py from the langchain codebase.

Function python LangChainCore LanguageModelBase calls 2 called by 1

Entity Profile

LangChainCore→ LanguageModelBase→ _astream() — langchain Function Reference

Dependency Diagram

graph TD
  58b8669b_d421_b6e5_b179_03394129fd02["_astream()"]
  6bee45b2_b649_e251_1fdc_dcf49f8bb331["BaseOpenAI"]
  58b8669b_d421_b6e5_b179_03394129fd02 -->|defined in| 6bee45b2_b649_e251_1fdc_dcf49f8bb331
  2d321de2_cda4_2af7_7550_f3f179b1dddb["_agenerate()"]
  2d321de2_cda4_2af7_7550_f3f179b1dddb -->|calls| 58b8669b_d421_b6e5_b179_03394129fd02
  11f618dd_a369_1622_0130_0cc8a952e5cd["get_sub_prompts()"]
  58b8669b_d421_b6e5_b179_03394129fd02 -->|calls| 11f618dd_a369_1622_0130_0cc8a952e5cd
  15d3c1cb_a282_c6f2_5db8_1d781bb37535["_stream_response_to_generation_chunk()"]
  58b8669b_d421_b6e5_b179_03394129fd02 -->|calls| 15d3c1cb_a282_c6f2_5db8_1d781bb37535
  style 58b8669b_d421_b6e5_b179_03394129fd02 fill:#6366f1,stroke:#818cf8,color:#fff

Relationship Graph

Source Code

libs/partners/openai/langchain_openai/llms/base.py lines 400–427

    async def _astream(
        self,
        prompt: str,
        stop: list[str] | None = None,
        run_manager: AsyncCallbackManagerForLLMRun | None = None,
        **kwargs: Any,
    ) -> AsyncIterator[GenerationChunk]:
        params = {**self._invocation_params, **kwargs, "stream": True}
        self.get_sub_prompts(params, [prompt], stop)  # this mutates params
        async for stream_resp in await self.async_client.create(
            prompt=prompt, **params
        ):
            if not isinstance(stream_resp, dict):
                stream_resp = stream_resp.model_dump()
            chunk = _stream_response_to_generation_chunk(stream_resp)

            if run_manager:
                await run_manager.on_llm_new_token(
                    chunk.text,
                    chunk=chunk,
                    verbose=self.verbose,
                    logprobs=(
                        chunk.generation_info["logprobs"]
                        if chunk.generation_info
                        else None
                    ),
                )
            yield chunk