from_llm() — langchain Function Reference

Architecture documentation for the from_llm() function in eval_chain.py from the langchain codebase.

Function python LangChainCore ApiManagement calls 2 called by 1

Entity Profile

LangChainCore→ ApiManagement→ from_llm() — langchain Function Reference

Dependency Diagram

graph TD
  69672bbd_1b9c_053c_d740_7c250335ad18["from_llm()"]
  fa921808_fa38_53ce_81bc_ca579d9b8656["LabeledPairwiseStringEvalChain"]
  69672bbd_1b9c_053c_d740_7c250335ad18 -->|defined in| fa921808_fa38_53ce_81bc_ca579d9b8656
  7f35a36c_35ea_7f92_9706_72edf0eab7e3["from_llm()"]
  7f35a36c_35ea_7f92_9706_72edf0eab7e3 -->|calls| 69672bbd_1b9c_053c_d740_7c250335ad18
  7f35a36c_35ea_7f92_9706_72edf0eab7e3["from_llm()"]
  69672bbd_1b9c_053c_d740_7c250335ad18 -->|calls| 7f35a36c_35ea_7f92_9706_72edf0eab7e3
  4995eef9_93cd_17ab_915e_ed4ae1cf159f["resolve_pairwise_criteria()"]
  69672bbd_1b9c_053c_d740_7c250335ad18 -->|calls| 4995eef9_93cd_17ab_915e_ed4ae1cf159f
  style 69672bbd_1b9c_053c_d740_7c250335ad18 fill:#6366f1,stroke:#818cf8,color:#fff

Relationship Graph

Source Code

libs/langchain/langchain_classic/evaluation/comparison/eval_chain.py lines 434–474

    def from_llm(
        cls,
        llm: BaseLanguageModel,
        *,
        prompt: PromptTemplate | None = None,
        criteria: CRITERIA_TYPE | str | None = None,
        **kwargs: Any,
    ) -> PairwiseStringEvalChain:
        """Initialize the LabeledPairwiseStringEvalChain from an LLM.

        Args:
            llm: The LLM to use.
            prompt: The prompt to use.
            criteria: The criteria to use.
            **kwargs: Additional keyword arguments.

        Returns:
            The initialized `LabeledPairwiseStringEvalChain`.

        Raises:
            ValueError: If the input variables are not as expected.

        """
        expected_input_vars = {
            "prediction",
            "prediction_b",
            "input",
            "reference",
            "criteria",
        }
        prompt_ = prompt or COMPARISON_TEMPLATE_WITH_REFERENCE
        if expected_input_vars != set(prompt_.input_variables):
            msg = (
                f"Input variables should be {expected_input_vars}, "
                f"but got {prompt_.input_variables}"
            )
            raise ValueError(msg)
        criteria_ = resolve_pairwise_criteria(criteria)
        criteria_str = "\n".join(f"{k}: {v}" for k, v in criteria_.items())
        criteria_str = CRITERIA_INSTRUCTIONS + criteria_str if criteria_str else ""
        return cls(llm=llm, prompt=prompt_.partial(criteria=criteria_str), **kwargs)