_evaluate_strings() — langchain Function Reference

Architecture documentation for the _evaluate_strings() function in base.py from the langchain codebase.

Function python LangChainCore LanguageModelBase calls 2 called by 1

Entity Profile

LangChainCore→ LanguageModelBase→ _evaluate_strings() — langchain Function Reference

Dependency Diagram

graph TD
  4021fe41_65d3_6fa7_5d31_3efed7bb0c71["_evaluate_strings()"]
  909e1aca_7c35_c9d5_2f9b_81064d86d9e6["JsonEqualityEvaluator"]
  4021fe41_65d3_6fa7_5d31_3efed7bb0c71 -->|defined in| 909e1aca_7c35_c9d5_2f9b_81064d86d9e6
  5524730f_9608_c667_bf87_2f343457eefb["_evaluate_strings()"]
  5524730f_9608_c667_bf87_2f343457eefb -->|calls| 4021fe41_65d3_6fa7_5d31_3efed7bb0c71
  7793dc6c_9c60_a7a0_ca86_d11624937249["_parse_json()"]
  4021fe41_65d3_6fa7_5d31_3efed7bb0c71 -->|calls| 7793dc6c_9c60_a7a0_ca86_d11624937249
  5524730f_9608_c667_bf87_2f343457eefb["_evaluate_strings()"]
  4021fe41_65d3_6fa7_5d31_3efed7bb0c71 -->|calls| 5524730f_9608_c667_bf87_2f343457eefb
  style 4021fe41_65d3_6fa7_5d31_3efed7bb0c71 fill:#6366f1,stroke:#818cf8,color:#fff

Relationship Graph

Source Code

libs/langchain/langchain_classic/evaluation/parsing/base.py lines 158–181

    def _evaluate_strings(
        self,
        prediction: str,
        reference: str | None = None,
        **kwargs: Any,
    ) -> dict:
        """Evaluate the prediction string.

        Args:
            prediction: The prediction string to evaluate.
            reference: The reference string to compare against.
            **kwargs: Additional keyword arguments (not used).

        Returns:
            `dict` containing the evaluation score.
        """
        parsed = self._parse_json(prediction)
        label = self._parse_json(cast("str", reference))
        if isinstance(label, list):
            if not isinstance(parsed, list):
                return {"score": 0}
            parsed = sorted(parsed, key=str)
            label = sorted(label, key=str)
        return {"score": self.operator(parsed, label)}