on_llm_new_token() — langchain Function Reference

Architecture documentation for the on_llm_new_token() function in base.py from the langchain codebase.

Function python Observability TelemetryUtilities calls 1 called by 1

Entity Profile

Observability→ TelemetryUtilities→ on_llm_new_token() — langchain Function Reference

Dependency Diagram

graph TD
  726c93cd_c9bb_47da_6e63_25438b2f6fff["on_llm_new_token()"]
  64513429_d001_87b1_5f19_cc17a3f3577b["AsyncCallbackHandler"]
  726c93cd_c9bb_47da_6e63_25438b2f6fff -->|defined in| 64513429_d001_87b1_5f19_cc17a3f3577b
  e13a7e6b_ef2e_5cb6_65a5_ceb4a608b981["on_llm_new_token()"]
  e13a7e6b_ef2e_5cb6_65a5_ceb4a608b981 -->|calls| 726c93cd_c9bb_47da_6e63_25438b2f6fff
  e13a7e6b_ef2e_5cb6_65a5_ceb4a608b981["on_llm_new_token()"]
  726c93cd_c9bb_47da_6e63_25438b2f6fff -->|calls| e13a7e6b_ef2e_5cb6_65a5_ceb4a608b981
  style 726c93cd_c9bb_47da_6e63_25438b2f6fff fill:#6366f1,stroke:#818cf8,color:#fff

Relationship Graph

Source Code

libs/core/langchain_core/callbacks/base.py lines 551–572

    async def on_llm_new_token(
        self,
        token: str,
        *,
        chunk: GenerationChunk | ChatGenerationChunk | None = None,
        run_id: UUID,
        parent_run_id: UUID | None = None,
        tags: list[str] | None = None,
        **kwargs: Any,
    ) -> None:
        """Run on new output token. Only available when streaming is enabled.

        For both chat models and non-chat models (legacy text completion LLMs).

        Args:
            token: The new token.
            chunk: The new generated chunk, containing content and other information.
            run_id: The ID of the current run.
            parent_run_id: The ID of the parent run.
            tags: The tags.
            **kwargs: Additional keyword arguments.
        """