langchain Architecture Docs
Home / File/ trajectory_eval_chain.py — langchain Source File

trajectory_eval_chain.py — langchain Source File

Architecture documentation for trajectory_eval_chain.py, a python file in the langchain codebase. 16 imports, 0 dependents.

File python CoreAbstractions Serialization 16 imports 3 classes

Entity Profile

Dependency Diagram

graph LR
  c646e46b_908e_bc3a_44e1_fafa62570a54["trajectory_eval_chain.py"]
  67ec3255_645e_8b6e_1eff_1eb3c648ed95["re"]
  c646e46b_908e_bc3a_44e1_fafa62570a54 --> 67ec3255_645e_8b6e_1eff_1eb3c648ed95
  cfe2bde5_180e_e3b0_df2b_55b3ebaca8e7["collections.abc"]
  c646e46b_908e_bc3a_44e1_fafa62570a54 --> cfe2bde5_180e_e3b0_df2b_55b3ebaca8e7
  8e2034b7_ceb8_963f_29fc_2ea6b50ef9b3["typing"]
  c646e46b_908e_bc3a_44e1_fafa62570a54 --> 8e2034b7_ceb8_963f_29fc_2ea6b50ef9b3
  80d582c5_7cc3_ac96_2742_3dbe1cbd4e2b["langchain_core.agents"]
  c646e46b_908e_bc3a_44e1_fafa62570a54 --> 80d582c5_7cc3_ac96_2742_3dbe1cbd4e2b
  f3bc7443_c889_119d_0744_aacc3620d8d2["langchain_core.callbacks"]
  c646e46b_908e_bc3a_44e1_fafa62570a54 --> f3bc7443_c889_119d_0744_aacc3620d8d2
  e8ec017e_6c91_4b34_675f_2a96c5aa9be6["langchain_core.callbacks.manager"]
  c646e46b_908e_bc3a_44e1_fafa62570a54 --> e8ec017e_6c91_4b34_675f_2a96c5aa9be6
  75137834_4ba7_dc43_7ec5_182c05eceedf["langchain_core.exceptions"]
  c646e46b_908e_bc3a_44e1_fafa62570a54 --> 75137834_4ba7_dc43_7ec5_182c05eceedf
  ba43b74d_3099_7e1c_aac3_cf594720469e["langchain_core.language_models"]
  c646e46b_908e_bc3a_44e1_fafa62570a54 --> ba43b74d_3099_7e1c_aac3_cf594720469e
  2312f229_c199_ac88_c29f_62e2a2958404["langchain_core.language_models.chat_models"]
  c646e46b_908e_bc3a_44e1_fafa62570a54 --> 2312f229_c199_ac88_c29f_62e2a2958404
  83d7c7fd_1989_762c_9cf3_cecb50ada22b["langchain_core.output_parsers"]
  c646e46b_908e_bc3a_44e1_fafa62570a54 --> 83d7c7fd_1989_762c_9cf3_cecb50ada22b
  43d88577_548b_2248_b01b_7987bae85dcc["langchain_core.tools"]
  c646e46b_908e_bc3a_44e1_fafa62570a54 --> 43d88577_548b_2248_b01b_7987bae85dcc
  6e58aaea_f08e_c099_3cc7_f9567bfb1ae7["pydantic"]
  c646e46b_908e_bc3a_44e1_fafa62570a54 --> 6e58aaea_f08e_c099_3cc7_f9567bfb1ae7
  91721f45_4909_e489_8c1f_084f8bd87145["typing_extensions"]
  c646e46b_908e_bc3a_44e1_fafa62570a54 --> 91721f45_4909_e489_8c1f_084f8bd87145
  31974615_0d58_bd26_13f1_776e0a9d1413["langchain_classic.chains.llm"]
  c646e46b_908e_bc3a_44e1_fafa62570a54 --> 31974615_0d58_bd26_13f1_776e0a9d1413
  style c646e46b_908e_bc3a_44e1_fafa62570a54 fill:#6366f1,stroke:#818cf8,color:#fff

Relationship Graph

Source Code

"""A chain for evaluating ReAct style agents.

This chain is used to evaluate ReAct style agents by reasoning about
the sequence of actions taken and their outcomes. It uses a language model
chain (LLMChain) to generate the reasoning and scores.
"""

import re
from collections.abc import Sequence
from typing import (
    Any,
    TypedDict,
    cast,
)

from langchain_core.agents import AgentAction
from langchain_core.callbacks import Callbacks
from langchain_core.callbacks.manager import (
    AsyncCallbackManagerForChainRun,
    CallbackManagerForChainRun,
)
from langchain_core.exceptions import OutputParserException
from langchain_core.language_models import BaseLanguageModel
from langchain_core.language_models.chat_models import BaseChatModel
from langchain_core.output_parsers import BaseOutputParser
from langchain_core.tools import BaseTool
from pydantic import ConfigDict, Field
from typing_extensions import override

from langchain_classic.chains.llm import LLMChain
from langchain_classic.evaluation.agents.trajectory_eval_prompt import (
    EVAL_CHAT_PROMPT,
    TOOL_FREE_EVAL_CHAT_PROMPT,
)
from langchain_classic.evaluation.schema import AgentTrajectoryEvaluator, LLMEvalChain

_MAX_SCORE = 5


class TrajectoryEval(TypedDict):
    """A named tuple containing the score and reasoning for a trajectory."""

    score: float
    """The score for the trajectory, normalized from 0 to 1."""
    reasoning: str
    """The reasoning for the score."""


class TrajectoryOutputParser(BaseOutputParser):
    """Trajectory output parser."""

    @property
    def _type(self) -> str:
        return "agent_trajectory"

    def parse(self, text: str) -> TrajectoryEval:
        """Parse the output text and extract the score and reasoning.

        Args:
            text: The output text to parse.
// ... (359 more lines)

Domain

Subdomains

Classes

Dependencies

  • collections.abc
  • langchain_classic.chains.llm
  • langchain_classic.evaluation.agents.trajectory_eval_prompt
  • langchain_classic.evaluation.schema
  • langchain_core.agents
  • langchain_core.callbacks
  • langchain_core.callbacks.manager
  • langchain_core.exceptions
  • langchain_core.language_models
  • langchain_core.language_models.chat_models
  • langchain_core.output_parsers
  • langchain_core.tools
  • pydantic
  • re
  • typing
  • typing_extensions

Source

Frequently Asked Questions

What does trajectory_eval_chain.py do?
trajectory_eval_chain.py is a source file in the langchain codebase, written in python. It belongs to the CoreAbstractions domain, Serialization subdomain.
What does trajectory_eval_chain.py depend on?
trajectory_eval_chain.py imports 16 module(s): collections.abc, langchain_classic.chains.llm, langchain_classic.evaluation.agents.trajectory_eval_prompt, langchain_classic.evaluation.schema, langchain_core.agents, langchain_core.callbacks, langchain_core.callbacks.manager, langchain_core.exceptions, and 8 more.
Where is trajectory_eval_chain.py in the architecture?
trajectory_eval_chain.py is located at libs/langchain/langchain_classic/evaluation/agents/trajectory_eval_chain.py (domain: CoreAbstractions, subdomain: Serialization, directory: libs/langchain/langchain_classic/evaluation/agents).

Analyze Your Own Codebase

Get architecture documentation, dependency graphs, and domain analysis for your codebase in minutes.

Try Supermodel Free