on_llm_new_token() — langchain Function Reference

Architecture documentation for the on_llm_new_token() function in manager.py from the langchain codebase.

Function python Observability CallbackManager calls 2 called by 1

Entity Profile

Observability→ CallbackManager→ on_llm_new_token() — langchain Function Reference

Dependency Diagram

graph TD
  00662245_87d9_b092_6f7f_ae1aca037dce["on_llm_new_token()"]
  edb9d036_ba6e_cc5d_0607_e2486edb64dc["AsyncCallbackManagerForLLMRun"]
  00662245_87d9_b092_6f7f_ae1aca037dce -->|defined in| edb9d036_ba6e_cc5d_0607_e2486edb64dc
  c817e64b_0cff_aea1_66cf_1dde9e8eda41["on_llm_new_token()"]
  c817e64b_0cff_aea1_66cf_1dde9e8eda41 -->|calls| 00662245_87d9_b092_6f7f_ae1aca037dce
  c817e64b_0cff_aea1_66cf_1dde9e8eda41["on_llm_new_token()"]
  00662245_87d9_b092_6f7f_ae1aca037dce -->|calls| c817e64b_0cff_aea1_66cf_1dde9e8eda41
  3a243d21_72d1_dca9_f7be_acbb482b9f86["ahandle_event()"]
  00662245_87d9_b092_6f7f_ae1aca037dce -->|calls| 3a243d21_72d1_dca9_f7be_acbb482b9f86
  style 00662245_87d9_b092_6f7f_ae1aca037dce fill:#6366f1,stroke:#818cf8,color:#fff

Relationship Graph

Source Code

libs/core/langchain_core/callbacks/manager.py lines 774–801

    async def on_llm_new_token(
        self,
        token: str,
        *,
        chunk: GenerationChunk | ChatGenerationChunk | None = None,
        **kwargs: Any,
    ) -> None:
        """Run when LLM generates a new token.

        Args:
            token: The new token.
            chunk: The chunk.
            **kwargs: Additional keyword arguments.

        """
        if not self.handlers:
            return
        await ahandle_event(
            self.handlers,
            "on_llm_new_token",
            "ignore_llm",
            token,
            chunk=chunk,
            run_id=self.run_id,
            parent_run_id=self.parent_run_id,
            tags=self.tags,
            **kwargs,
        )