on_llm_new_token() — langchain Function Reference

Architecture documentation for the on_llm_new_token() function in base.py from the langchain codebase.

Function python Observability TelemetryUtilities calls 1 called by 1

Entity Profile

Observability→ TelemetryUtilities→ on_llm_new_token() — langchain Function Reference

Dependency Diagram

graph TD
  e13a7e6b_ef2e_5cb6_65a5_ceb4a608b981["on_llm_new_token()"]
  8d0067ae_0de0_c143_63e7_e97f7fd9f614["LLMManagerMixin"]
  e13a7e6b_ef2e_5cb6_65a5_ceb4a608b981 -->|defined in| 8d0067ae_0de0_c143_63e7_e97f7fd9f614
  726c93cd_c9bb_47da_6e63_25438b2f6fff["on_llm_new_token()"]
  726c93cd_c9bb_47da_6e63_25438b2f6fff -->|calls| e13a7e6b_ef2e_5cb6_65a5_ceb4a608b981
  726c93cd_c9bb_47da_6e63_25438b2f6fff["on_llm_new_token()"]
  e13a7e6b_ef2e_5cb6_65a5_ceb4a608b981 -->|calls| 726c93cd_c9bb_47da_6e63_25438b2f6fff
  style e13a7e6b_ef2e_5cb6_65a5_ceb4a608b981 fill:#6366f1,stroke:#818cf8,color:#fff

Relationship Graph

Source Code

libs/core/langchain_core/callbacks/base.py lines 64–87

    def on_llm_new_token(
        self,
        token: str,
        *,
        chunk: GenerationChunk | ChatGenerationChunk | None = None,
        run_id: UUID,
        parent_run_id: UUID | None = None,
        tags: list[str] | None = None,
        **kwargs: Any,
    ) -> Any:
        """Run on new output token.

        Only available when streaming is enabled.

        For both chat models and non-chat models (legacy text completion LLMs).

        Args:
            token: The new token.
            chunk: The new generated chunk, containing content and other information.
            run_id: The ID of the current run.
            parent_run_id: The ID of the parent run.
            tags: The tags.
            **kwargs: Additional keyword arguments.
        """