_astream() — langchain Function Reference

Architecture documentation for the _astream() function in llms.py from the langchain codebase.

Function python LangChainCore LanguageModelBase called by 1

Entity Profile

LangChainCore→ LanguageModelBase→ _astream() — langchain Function Reference

Dependency Diagram

graph TD
  67bb9e7b_f91d_5598_fccd_d291f76257e7["_astream()"]
  ce4aa464_3868_179e_5d99_df48bc307c5f["BaseLLM"]
  67bb9e7b_f91d_5598_fccd_d291f76257e7 -->|defined in| ce4aa464_3868_179e_5d99_df48bc307c5f
  6d1a7389_71e1_42ae_986c_fd865f245d2b["astream()"]
  6d1a7389_71e1_42ae_986c_fd865f245d2b -->|calls| 67bb9e7b_f91d_5598_fccd_d291f76257e7
  style 67bb9e7b_f91d_5598_fccd_d291f76257e7 fill:#6366f1,stroke:#818cf8,color:#fff

Relationship Graph

Source Code

libs/core/langchain_core/language_models/llms.py lines 733–778

    async def _astream(
        self,
        prompt: str,
        stop: list[str] | None = None,
        run_manager: AsyncCallbackManagerForLLMRun | None = None,
        **kwargs: Any,
    ) -> AsyncIterator[GenerationChunk]:
        """An async version of the _stream method.

        The default implementation uses the synchronous _stream method and wraps it in
        an async iterator. Subclasses that need to provide a true async implementation
        should override this method.

        Args:
            prompt: The prompt to generate from.
            stop: Stop words to use when generating.

                Model output is cut off at the first occurrence of any of these
                substrings.
            run_manager: Callback manager for the run.
            **kwargs: Arbitrary additional keyword arguments.

                These are usually passed to the model provider API call.

        Yields:
            Generation chunks.
        """
        iterator = await run_in_executor(
            None,
            self._stream,
            prompt,
            stop,
            run_manager.get_sync() if run_manager else None,
            **kwargs,
        )
        done = object()
        while True:
            item = await run_in_executor(
                None,
                next,
                iterator,
                done,
            )
            if item is done:
                break
            yield item  # type: ignore[misc]