TrajectoryOutputParser Class — langchain Architecture

Architecture documentation for the TrajectoryOutputParser class in trajectory_eval_chain.py from the langchain codebase.

Class python

Entity Profile

Dependency Diagram

graph TD
  fc2ee41e_371b_eb40_7fbe_50ed7b88b273["TrajectoryOutputParser"]
  e155cd04_a39d_263a_c7f3_32e5830d204b["BaseOutputParser"]
  fc2ee41e_371b_eb40_7fbe_50ed7b88b273 -->|extends| e155cd04_a39d_263a_c7f3_32e5830d204b
  c646e46b_908e_bc3a_44e1_fafa62570a54["trajectory_eval_chain.py"]
  fc2ee41e_371b_eb40_7fbe_50ed7b88b273 -->|defined in| c646e46b_908e_bc3a_44e1_fafa62570a54
  97cd8467_ecb0_f0e6_a6de_079e61b3d911["_type()"]
  fc2ee41e_371b_eb40_7fbe_50ed7b88b273 -->|method| 97cd8467_ecb0_f0e6_a6de_079e61b3d911
  aaf43a39_34d2_dbaa_14b5_253722ce5c40["parse()"]
  fc2ee41e_371b_eb40_7fbe_50ed7b88b273 -->|method| aaf43a39_34d2_dbaa_14b5_253722ce5c40

Relationship Graph

Source Code

libs/langchain/langchain_classic/evaluation/agents/trajectory_eval_chain.py lines 49–93

class TrajectoryOutputParser(BaseOutputParser):
    """Trajectory output parser."""

    @property
    def _type(self) -> str:
        return "agent_trajectory"

    def parse(self, text: str) -> TrajectoryEval:
        """Parse the output text and extract the score and reasoning.

        Args:
            text: The output text to parse.

        Returns:
            A named tuple containing the normalized score and reasoning.

        Raises:
            If the score is not found in the output text or if the LLM's score is not a
            digit in the range 1-5.
        """
        if "Score:" not in text:
            msg = f"Could not find score in model eval output: {text}"
            raise OutputParserException(msg)

        reasoning, score_str = text.split("Score: ", maxsplit=1)

        reasoning, score_str = reasoning.strip(), score_str.strip()

        # Use regex to extract the score.
        # This will get the number in the string, even if it is a float or more than 10.
        # E.g. "Score: 1" will return 1, "Score: 3.5" will return 3.5, and
        # "Score: 10" will return 10.
        # The score should be an integer digit in the range 1-5.
        _score = re.search(r"(\d+(\.\d+)?)", score_str)
        # If the score is not found or is a float, raise an exception.
        if _score is None or "." in _score.group(1):
            msg = f"Score is not an integer digit in the range 1-5: {text}"
            raise OutputParserException(msg)
        score = int(_score.group(1))
        # If the score is not in the range 1-5, raise an exception.
        if not 1 <= score <= _MAX_SCORE:
            msg = f"Score is not a digit in the range 1-5: {text}"
            raise OutputParserException(msg)
        normalized_score = (score - 1) / (_MAX_SCORE - 1)
        return TrajectoryEval(score=normalized_score, reasoning=reasoning)