_get_score() — langchain Function Reference

Architecture documentation for the _get_score() function in eval_chain.py from the langchain codebase.

Function python LangChainCore ApiManagement called by 1

Entity Profile

LangChainCore→ ApiManagement→ _get_score() — langchain Function Reference

Dependency Diagram

graph TD
  21a82aa0_b2c0_86fc_987b_e439fa1cc378["_get_score()"]
  514ad815_2197_8c98_58ac_640cc349a6ab["eval_chain.py"]
  21a82aa0_b2c0_86fc_987b_e439fa1cc378 -->|defined in| 514ad815_2197_8c98_58ac_640cc349a6ab
  b91be59a_7ef5_038f_5ebf_96d2d077aa5f["_parse_string_eval_output()"]
  b91be59a_7ef5_038f_5ebf_96d2d077aa5f -->|calls| 21a82aa0_b2c0_86fc_987b_e439fa1cc378
  style 21a82aa0_b2c0_86fc_987b_e439fa1cc378 fill:#6366f1,stroke:#818cf8,color:#fff

Relationship Graph

Source Code

libs/langchain/langchain_classic/evaluation/qa/eval_chain.py lines 26–52

def _get_score(text: str) -> tuple[str, int] | None:
    match = re.search(r"grade:\s*(correct|incorrect)", text.strip(), re.IGNORECASE)
    if match:
        if match.group(1).upper() == "CORRECT":
            return "CORRECT", 1
        if match.group(1).upper() == "INCORRECT":
            return "INCORRECT", 0
    try:
        first_word = (
            text.strip().split()[0].translate(str.maketrans("", "", string.punctuation))
        )
        if first_word.upper() == "CORRECT":
            return "CORRECT", 1
        if first_word.upper() == "INCORRECT":
            return "INCORRECT", 0
        last_word = (
            text.strip()
            .split()[-1]
            .translate(str.maketrans("", "", string.punctuation))
        )
        if last_word.upper() == "CORRECT":
            return "CORRECT", 1
        if last_word.upper() == "INCORRECT":
            return "INCORRECT", 0
    except IndexError:
        pass
    return None