_evaluate_strings() — langchain Function Reference

Architecture documentation for the _evaluate_strings() function in base.py from the langchain codebase.

Function python LangChainCore LanguageModelBase calls 1 called by 1

Entity Profile

LangChainCore→ LanguageModelBase→ _evaluate_strings() — langchain Function Reference

Dependency Diagram

graph TD
  5524730f_9608_c667_bf87_2f343457eefb["_evaluate_strings()"]
  ca6449c4_5552_5059_7a40_009b5af57841["JsonValidityEvaluator"]
  5524730f_9608_c667_bf87_2f343457eefb -->|defined in| ca6449c4_5552_5059_7a40_009b5af57841
  4021fe41_65d3_6fa7_5d31_3efed7bb0c71["_evaluate_strings()"]
  4021fe41_65d3_6fa7_5d31_3efed7bb0c71 -->|calls| 5524730f_9608_c667_bf87_2f343457eefb
  4021fe41_65d3_6fa7_5d31_3efed7bb0c71["_evaluate_strings()"]
  5524730f_9608_c667_bf87_2f343457eefb -->|calls| 4021fe41_65d3_6fa7_5d31_3efed7bb0c71
  style 5524730f_9608_c667_bf87_2f343457eefb fill:#6366f1,stroke:#818cf8,color:#fff

Relationship Graph

Source Code

libs/langchain/langchain_classic/evaluation/parsing/base.py lines 62–88

    def _evaluate_strings(
        self,
        prediction: str,
        **kwargs: Any,
    ) -> dict:
        """Evaluate the prediction string.

        Args:
            prediction: The prediction string to evaluate.
            **kwargs: Additional keyword arguments (not used).

        Returns:
            `dict` containing the evaluation score. The score is `1` if
                the prediction is valid JSON, and `0` otherwise.

                If the prediction is not valid JSON, the dictionary also contains
                a `reasoning` field with the error message.

        """
        try:
            parse_json_markdown(prediction, parser=json.loads)
        except json.JSONDecodeError as e:
            return {"score": 0, "reasoning": str(e)}
        except Exception as e:
            _logger.exception("Passing JSON failed with unexpected error.")
            return {"score": 0, "reasoning": str(e)}
        return {"score": 1}