JsonEditDistanceEvaluator Class — langchain Architecture

Architecture documentation for the JsonEditDistanceEvaluator class in json_distance.py from the langchain codebase.

Class python

Entity Profile

Dependency Diagram

graph TD
  481593b0_ea9f_1093_bbdc_d3bd06c33d40["JsonEditDistanceEvaluator"]
  3bfa222e_74bd_6c80_db4a_a3d3a09b5e7b["StringEvaluator"]
  481593b0_ea9f_1093_bbdc_d3bd06c33d40 -->|extends| 3bfa222e_74bd_6c80_db4a_a3d3a09b5e7b
  b8988b93_4a10_f796_860e_c764b7dcad46["json_distance.py"]
  481593b0_ea9f_1093_bbdc_d3bd06c33d40 -->|defined in| b8988b93_4a10_f796_860e_c764b7dcad46
  d0762f64_a7cc_0f12_772c_aac695ded0ec["__init__()"]
  481593b0_ea9f_1093_bbdc_d3bd06c33d40 -->|method| d0762f64_a7cc_0f12_772c_aac695ded0ec
  c5932fe6_1913_187b_8946_42699891911e["requires_input()"]
  481593b0_ea9f_1093_bbdc_d3bd06c33d40 -->|method| c5932fe6_1913_187b_8946_42699891911e
  29833b54_701b_842d_d966_b5a9024b04ce["requires_reference()"]
  481593b0_ea9f_1093_bbdc_d3bd06c33d40 -->|method| 29833b54_701b_842d_d966_b5a9024b04ce
  186b4262_cd78_68b9_dc16_93ab6118dc38["evaluation_name()"]
  481593b0_ea9f_1093_bbdc_d3bd06c33d40 -->|method| 186b4262_cd78_68b9_dc16_93ab6118dc38
  ee287886_d81c_e9e6_156a_557c97ddb4b1["_parse_json()"]
  481593b0_ea9f_1093_bbdc_d3bd06c33d40 -->|method| ee287886_d81c_e9e6_156a_557c97ddb4b1
  e77d63b1_8ba5_e39d_d541_78866bbc0b02["_evaluate_strings()"]
  481593b0_ea9f_1093_bbdc_d3bd06c33d40 -->|method| e77d63b1_8ba5_e39d_d541_78866bbc0b02

Relationship Graph

Source Code

libs/langchain/langchain_classic/evaluation/parsing/json_distance.py lines 11–109

class JsonEditDistanceEvaluator(StringEvaluator):
    """An evaluator that calculates the edit distance between JSON strings.

    This evaluator computes a normalized Damerau-Levenshtein distance between two JSON strings
    after parsing them and converting them to a canonical format (i.e., whitespace and key order are normalized).
    It can be customized with alternative distance and canonicalization functions.

    Attributes:
        _string_distance (Callable[[str, str], float]): The internal distance computation function.
        _canonicalize (Callable[[Any], Any]): The internal canonicalization function.

    Examples:
        >>> evaluator = JsonEditDistanceEvaluator()
        >>> result = evaluator.evaluate_strings(
        ...     prediction='{"a": 1, "b": 2}', reference='{"a": 1, "b": 3}'
        ... )
        >>> assert result["score"] is not None

    Raises:
        ImportError: If `rapidfuzz` is not installed and no alternative `string_distance` function is provided.

    """  # noqa: E501

    def __init__(
        self,
        string_distance: Callable[[str, str], float] | None = None,
        canonicalize: Callable[[Any], Any] | None = None,
        **_: Any,
    ) -> None:
        """Initialize the JsonEditDistanceEvaluator.

        Args:
            string_distance: A callable that computes the distance between two strings.
                If not provided, a Damerau-Levenshtein distance from the `rapidfuzz`
                package will be used.
            canonicalize: A callable that converts a parsed JSON object into its
                canonical string form.
                If not provided, the default behavior is to serialize the JSON with
                sorted keys and no extra whitespace.

        Raises:
            ImportError: If the `rapidfuzz` package is not installed and no
                `string_distance` function is provided.
        """
        super().__init__()
        if string_distance is not None:
            self._string_distance = string_distance
        else:
            try:
                from rapidfuzz import distance as rfd
            except ImportError as e:
                msg = (
                    "The default string_distance operator for the "
                    " JsonEditDistanceEvaluator requires installation of "
                    "the rapidfuzz package. "
                    "Please install it with `pip install rapidfuzz`."
                )
                raise ImportError(msg) from e
            self._string_distance = rfd.DamerauLevenshtein.normalized_distance
        if canonicalize is not None:
            self._canonicalize = canonicalize
        else:
            self._canonicalize = lambda x: json.dumps(
                x,
                separators=(",", ":"),
                sort_keys=True,  # eliminate whitespace
            )

    @property
    @override
    def requires_input(self) -> bool:
        return False

    @property
    @override
    def requires_reference(self) -> bool:
        return True

    @property
    @override
    def evaluation_name(self) -> str: