on_llm_new_token() — langchain Function Reference

Architecture documentation for the on_llm_new_token() function in manager.py from the langchain codebase.

Function python Observability CallbackManager calls 2 called by 1

Entity Profile

Observability→ CallbackManager→ on_llm_new_token() — langchain Function Reference

Dependency Diagram

graph TD
  c817e64b_0cff_aea1_66cf_1dde9e8eda41["on_llm_new_token()"]
  a2d4d2d4_52ab_0e90_145b_14769a61a7d9["CallbackManagerForLLMRun"]
  c817e64b_0cff_aea1_66cf_1dde9e8eda41 -->|defined in| a2d4d2d4_52ab_0e90_145b_14769a61a7d9
  00662245_87d9_b092_6f7f_ae1aca037dce["on_llm_new_token()"]
  00662245_87d9_b092_6f7f_ae1aca037dce -->|calls| c817e64b_0cff_aea1_66cf_1dde9e8eda41
  00662245_87d9_b092_6f7f_ae1aca037dce["on_llm_new_token()"]
  c817e64b_0cff_aea1_66cf_1dde9e8eda41 -->|calls| 00662245_87d9_b092_6f7f_ae1aca037dce
  8964b4e9_440e_18a7_2a15_a54a0b2b733c["handle_event()"]
  c817e64b_0cff_aea1_66cf_1dde9e8eda41 -->|calls| 8964b4e9_440e_18a7_2a15_a54a0b2b733c
  style c817e64b_0cff_aea1_66cf_1dde9e8eda41 fill:#6366f1,stroke:#818cf8,color:#fff

Relationship Graph

Source Code

libs/core/langchain_core/callbacks/manager.py lines 675–702

    def on_llm_new_token(
        self,
        token: str,
        *,
        chunk: GenerationChunk | ChatGenerationChunk | None = None,
        **kwargs: Any,
    ) -> None:
        """Run when LLM generates a new token.

        Args:
            token: The new token.
            chunk: The chunk.
            **kwargs: Additional keyword arguments.

        """
        if not self.handlers:
            return
        handle_event(
            self.handlers,
            "on_llm_new_token",
            "ignore_llm",
            token=token,
            run_id=self.run_id,
            parent_run_id=self.parent_run_id,
            tags=self.tags,
            chunk=chunk,
            **kwargs,
        )