from_llm() — langchain Function Reference

Architecture documentation for the from_llm() function in eval_chain.py from the langchain codebase.

Function python LangChainCore Runnables calls 2 called by 1

Entity Profile

LangChainCore→ Runnables→ from_llm() — langchain Function Reference

Dependency Diagram

graph TD
  3987e2f0_a8de_74dd_9ceb_d96cade3aefa["from_llm()"]
  df005def_3ccc_8e5a_d8f1_4757105de3a2["LabeledScoreStringEvalChain"]
  3987e2f0_a8de_74dd_9ceb_d96cade3aefa -->|defined in| df005def_3ccc_8e5a_d8f1_4757105de3a2
  3c6dc9b9_bf18_3520_a593_9147d40d2f2a["from_llm()"]
  3c6dc9b9_bf18_3520_a593_9147d40d2f2a -->|calls| 3987e2f0_a8de_74dd_9ceb_d96cade3aefa
  3c6dc9b9_bf18_3520_a593_9147d40d2f2a["from_llm()"]
  3987e2f0_a8de_74dd_9ceb_d96cade3aefa -->|calls| 3c6dc9b9_bf18_3520_a593_9147d40d2f2a
  351b62b7_d2ea_9d3f_cc78_2712a988a68f["resolve_criteria()"]
  3987e2f0_a8de_74dd_9ceb_d96cade3aefa -->|calls| 351b62b7_d2ea_9d3f_cc78_2712a988a68f
  style 3987e2f0_a8de_74dd_9ceb_d96cade3aefa fill:#6366f1,stroke:#818cf8,color:#fff

Relationship Graph

Source Code

libs/langchain/langchain_classic/evaluation/scoring/eval_chain.py lines 433–484

    def from_llm(
        cls,
        llm: BaseLanguageModel,
        *,
        prompt: PromptTemplate | None = None,
        criteria: CRITERIA_TYPE | str | None = None,
        normalize_by: float | None = None,
        **kwargs: Any,
    ) -> LabeledScoreStringEvalChain:
        """Initialize the LabeledScoreStringEvalChain from an LLM.

        Args:
            llm: The LLM to use.
            prompt: The prompt to use.
            criteria: The criteria to use.
            normalize_by: The value to normalize the score by.
            **kwargs: Additional keyword arguments.

        Returns:
            The initialized LabeledScoreStringEvalChain.

        Raises:
            ValueError: If the input variables are not as expected.

        """
        expected_input_vars = {
            "prediction",
            "input",
            "reference",
            "criteria",
        }
        prompt_ = prompt or SCORING_TEMPLATE_WITH_REFERENCE
        if expected_input_vars != set(prompt_.input_variables):
            msg = (
                f"Input variables should be {expected_input_vars}, "
                f"but got {prompt_.input_variables}"
            )
            raise ValueError(msg)
        criteria_ = resolve_criteria(criteria)
        criteria_str = "\n".join(f"{k}: {v}" for k, v in criteria_.items()).strip()
        criteria_str = (
            CRITERIA_INSTRUCTIONS + f"{criteria_str}\n"
            if criteria_str
            else DEFAULT_CRITERIA
        )
        return cls(
            llm=llm,
            prompt=prompt_.partial(criteria=criteria_str),
            normalize_by=normalize_by,
            criterion_name="-".join(criteria_),
            **kwargs,
        )