_on_llm_new_token() — langchain Function Reference

Architecture documentation for the _on_llm_new_token() function in log_stream.py from the langchain codebase.

Function python Observability Tracers calls 1

Entity Profile

Observability→ Tracers→ _on_llm_new_token() — langchain Function Reference

Dependency Diagram

graph TD
  8261c1a6_4dc8_1ce7_5560_5ac4b395ac72["_on_llm_new_token()"]
  78fc514f_5439_00c6_7e00_021af9ce4e07["LogStreamCallbackHandler"]
  8261c1a6_4dc8_1ce7_5560_5ac4b395ac72 -->|defined in| 78fc514f_5439_00c6_7e00_021af9ce4e07
  c5f1fbd2_23b6_75cb_1f29_b9e028c332a0["send()"]
  8261c1a6_4dc8_1ce7_5560_5ac4b395ac72 -->|calls| c5f1fbd2_23b6_75cb_1f29_b9e028c332a0
  style 8261c1a6_4dc8_1ce7_5560_5ac4b395ac72 fill:#6366f1,stroke:#818cf8,color:#fff

Relationship Graph

Source Code

libs/core/langchain_core/tracers/log_stream.py lines 536–561

    def _on_llm_new_token(
        self,
        run: Run,
        token: str,
        chunk: GenerationChunk | ChatGenerationChunk | None,
    ) -> None:
        """Process new LLM token."""
        index = self._key_map_by_run_id.get(run.id)

        if index is None:
            return

        self.send(
            {
                "op": "add",
                "path": f"/logs/{index}/streamed_output_str/-",
                "value": token,
            },
            {
                "op": "add",
                "path": f"/logs/{index}/streamed_output/-",
                "value": chunk.message
                if isinstance(chunk, ChatGenerationChunk)
                else token,
            },
        )