ExactMatchStringEvaluator Class — langchain Architecture

Architecture documentation for the ExactMatchStringEvaluator class in base.py from the langchain codebase.

Class python

Entity Profile

Dependency Diagram

graph TD
  d210f142_16a2_0da3_73f7_079815481de7["ExactMatchStringEvaluator"]
  42f35457_68a1_961e_1ac4_cbaa4a2b48b3["StringEvaluator"]
  d210f142_16a2_0da3_73f7_079815481de7 -->|extends| 42f35457_68a1_961e_1ac4_cbaa4a2b48b3
  0b2ef852_a8bd_0357_97fb_8efda447e910["base.py"]
  d210f142_16a2_0da3_73f7_079815481de7 -->|defined in| 0b2ef852_a8bd_0357_97fb_8efda447e910
  530f3312_ec48_d537_4a05_79e731aacb5e["__init__()"]
  d210f142_16a2_0da3_73f7_079815481de7 -->|method| 530f3312_ec48_d537_4a05_79e731aacb5e
  208ce5dc_3341_817e_a767_7321dd236041["requires_input()"]
  d210f142_16a2_0da3_73f7_079815481de7 -->|method| 208ce5dc_3341_817e_a767_7321dd236041
  88a3ef90_0171_1a54_2e2e_ed92685cc91b["requires_reference()"]
  d210f142_16a2_0da3_73f7_079815481de7 -->|method| 88a3ef90_0171_1a54_2e2e_ed92685cc91b
  840f26fb_9b36_8a1a_067a_4eb3fbb6b8b5["input_keys()"]
  d210f142_16a2_0da3_73f7_079815481de7 -->|method| 840f26fb_9b36_8a1a_067a_4eb3fbb6b8b5
  2f6665be_0bb5_dc24_664f_9090b1c11ced["evaluation_name()"]
  d210f142_16a2_0da3_73f7_079815481de7 -->|method| 2f6665be_0bb5_dc24_664f_9090b1c11ced
  267d0a09_5cc6_f8c6_7f1b_9c32a09a316f["_evaluate_strings()"]
  d210f142_16a2_0da3_73f7_079815481de7 -->|method| 267d0a09_5cc6_f8c6_7f1b_9c32a09a316f

Relationship Graph

Source Code

libs/langchain/langchain_classic/evaluation/exact_match/base.py lines 9–101

class ExactMatchStringEvaluator(StringEvaluator):
    """Compute an exact match between the prediction and the reference.

    Examples:
    ----------
    >>> evaluator = ExactMatchChain()
    >>> evaluator.evaluate_strings(
            prediction="Mindy is the CTO",
            reference="Mindy is the CTO",
        )  # This will return {'score': 1.0}

    >>> evaluator.evaluate_strings(
            prediction="Mindy is the CTO",
            reference="Mindy is the CEO",
        )  # This will return {'score': 0.0}
    """

    def __init__(
        self,
        *,
        ignore_case: bool = False,
        ignore_punctuation: bool = False,
        ignore_numbers: bool = False,
        **_: Any,
    ):
        """Initialize the `ExactMatchStringEvaluator`.

        Args:
            ignore_case: Whether to ignore case when comparing strings.
            ignore_punctuation: Whether to ignore punctuation when comparing strings.
            ignore_numbers: Whether to ignore numbers when comparing strings.
        """
        super().__init__()
        self.ignore_case = ignore_case
        self.ignore_punctuation = ignore_punctuation
        self.ignore_numbers = ignore_numbers

    @property
    def requires_input(self) -> bool:
        """This evaluator does not require input."""
        return False

    @property
    def requires_reference(self) -> bool:
        """This evaluator requires a reference."""
        return True

    @property
    def input_keys(self) -> list[str]:
        """Get the input keys.

        Returns:
            The input keys.
        """
        return ["reference", "prediction"]

    @property
    def evaluation_name(self) -> str:
        """Get the evaluation name.

        Returns:
            The evaluation name.
        """
        return "exact_match"

    @override
    def _evaluate_strings(  # type: ignore[override]
        self,
        *,
        prediction: str,
        reference: str,
        **kwargs: Any,
    ) -> dict:
        """Evaluate the exact match between the prediction and the reference.

        Args:
            prediction: The prediction string.
            reference: The reference string.
            **kwargs: Additional keyword arguments (not used).

        Returns: