RegexMatchStringEvaluator Class — langchain Architecture

Architecture documentation for the RegexMatchStringEvaluator class in base.py from the langchain codebase.

Class python

Entity Profile

Dependency Diagram

graph TD
  12a7d7c3_a442_0a1c_99c0_b2002afd1515["RegexMatchStringEvaluator"]
  3bfa222e_74bd_6c80_db4a_a3d3a09b5e7b["StringEvaluator"]
  12a7d7c3_a442_0a1c_99c0_b2002afd1515 -->|extends| 3bfa222e_74bd_6c80_db4a_a3d3a09b5e7b
  6e19ef03_b3fe_9631_cb2c_38dd63c7086d["base.py"]
  12a7d7c3_a442_0a1c_99c0_b2002afd1515 -->|defined in| 6e19ef03_b3fe_9631_cb2c_38dd63c7086d
  6e636b98_7539_a4c9_83ba_eb0af2cd6d55["__init__()"]
  12a7d7c3_a442_0a1c_99c0_b2002afd1515 -->|method| 6e636b98_7539_a4c9_83ba_eb0af2cd6d55
  fa4cdd9f_770b_3325_1775_e2952c5007ad["requires_input()"]
  12a7d7c3_a442_0a1c_99c0_b2002afd1515 -->|method| fa4cdd9f_770b_3325_1775_e2952c5007ad
  f48889b4_3a1f_a928_6a13_0f8024860c35["requires_reference()"]
  12a7d7c3_a442_0a1c_99c0_b2002afd1515 -->|method| f48889b4_3a1f_a928_6a13_0f8024860c35
  f71b6173_772e_1368_dc94_190761443715["input_keys()"]
  12a7d7c3_a442_0a1c_99c0_b2002afd1515 -->|method| f71b6173_772e_1368_dc94_190761443715
  523001f8_775b_90a7_1aaf_78ba6abc2b41["evaluation_name()"]
  12a7d7c3_a442_0a1c_99c0_b2002afd1515 -->|method| 523001f8_775b_90a7_1aaf_78ba6abc2b41
  3493c43b_b018_2a91_76fb_e9999735516a["_evaluate_strings()"]
  12a7d7c3_a442_0a1c_99c0_b2002afd1515 -->|method| 3493c43b_b018_2a91_76fb_e9999735516a

Relationship Graph

Source Code

libs/langchain/langchain_classic/evaluation/regex_match/base.py lines 9–88

class RegexMatchStringEvaluator(StringEvaluator):
    """Compute a regex match between the prediction and the reference.

    Examples:
    ----------
    >>> evaluator = RegexMatchStringEvaluator(flags=re.IGNORECASE)
    >>> evaluator.evaluate_strings(
            prediction="Mindy is the CTO",
            reference="^mindy.*cto$",
        )  # This will return {'score': 1.0} due to the IGNORECASE flag

    >>> evaluator = RegexMatchStringEvaluator()
    >>> evaluator.evaluate_strings(
            prediction="Mindy is the CTO",
            reference="^Mike.*CEO$",
        )  # This will return {'score': 0.0}

    >>> evaluator.evaluate_strings(
            prediction="Mindy is the CTO",
            reference="^Mike.*CEO$|^Mindy.*CTO$",
        )  # This will return {'score': 1.0} as the prediction matches the second pattern in the union
    """  # noqa: E501

    def __init__(self, *, flags: int = 0, **_: Any):  # Default is no flags
        """Initialize the RegexMatchStringEvaluator.

        Args:
            flags: Flags to use for the regex match. Defaults to no flags.
        """
        super().__init__()
        self.flags = flags

    @property
    def requires_input(self) -> bool:
        """This evaluator does not require input."""
        return False

    @property
    def requires_reference(self) -> bool:
        """This evaluator requires a reference."""
        return True

    @property
    def input_keys(self) -> list[str]:
        """Get the input keys.

        Returns:
            The input keys.
        """
        return ["reference", "prediction"]

    @property
    def evaluation_name(self) -> str:
        """Get the evaluation name.

        Returns:
            The evaluation name.
        """
        return "regex_match"

    @override
    def _evaluate_strings(  # type: ignore[override]
        self,
        *,
        prediction: str,
        reference: str,
        **kwargs: Any,
    ) -> dict:
        """Evaluate the regex match between the prediction and the reference.

        Args:
            prediction: The prediction string.
            reference: The reference regex pattern.
            **kwargs: Additional keyword arguments (not used).

        Returns:
            The evaluation results containing the score.
        """
        match = re.match(reference, prediction, flags=self.flags)
        return {"score": int(bool(match))}