on_llm_new_token() — langchain Function Reference

Architecture documentation for the on_llm_new_token() function in base.py from the langchain codebase.

Function python Observability Tracers calls 2 called by 1

Entity Profile

Observability→ Tracers→ on_llm_new_token() — langchain Function Reference

Dependency Diagram

graph TD
  bdbf79bf_1c51_df7e_a101_2d5bec39a8b4["on_llm_new_token()"]
  0f6b3261_31fa_c34e_ca33_cb141bdf78ff["BaseTracer"]
  bdbf79bf_1c51_df7e_a101_2d5bec39a8b4 -->|defined in| 0f6b3261_31fa_c34e_ca33_cb141bdf78ff
  5f880efa_8d4b_5819_bceb_8fbd0e5df60f["on_llm_new_token()"]
  5f880efa_8d4b_5819_bceb_8fbd0e5df60f -->|calls| bdbf79bf_1c51_df7e_a101_2d5bec39a8b4
  a6b72ebe_4ebd_9d29_9fe1_f9e27948c7ff["_on_llm_new_token()"]
  bdbf79bf_1c51_df7e_a101_2d5bec39a8b4 -->|calls| a6b72ebe_4ebd_9d29_9fe1_f9e27948c7ff
  5f880efa_8d4b_5819_bceb_8fbd0e5df60f["on_llm_new_token()"]
  bdbf79bf_1c51_df7e_a101_2d5bec39a8b4 -->|calls| 5f880efa_8d4b_5819_bceb_8fbd0e5df60f
  style bdbf79bf_1c51_df7e_a101_2d5bec39a8b4 fill:#6366f1,stroke:#818cf8,color:#fff

Relationship Graph

Source Code

libs/core/langchain_core/tracers/base.py lines 135–167

    def on_llm_new_token(
        self,
        token: str,
        *,
        chunk: GenerationChunk | ChatGenerationChunk | None = None,
        run_id: UUID,
        parent_run_id: UUID | None = None,
        **kwargs: Any,
    ) -> Run:
        """Run on new LLM token.

        Only available when streaming is enabled.

        Args:
            token: The token.
            chunk: The chunk.
            run_id: The run ID.
            parent_run_id: The parent run ID.
            **kwargs: Additional arguments.

        Returns:
            The run.
        """
        # "chat_model" is only used for the experimental new streaming_events format.
        # This change should not affect any existing tracers.
        llm_run = self._llm_run_with_token_event(
            token=token,
            run_id=run_id,
            chunk=chunk,
            parent_run_id=parent_run_id,
        )
        self._on_llm_new_token(llm_run, token, chunk)
        return llm_run