load_evaluators() — langchain Function Reference

Architecture documentation for the load_evaluators() function in loading.py from the langchain codebase.

Function python LangChainCore Runnables calls 1

Entity Profile

LangChainCore→ Runnables→ load_evaluators() — langchain Function Reference

Dependency Diagram

graph TD
  deb188fb_75ee_300b_4914_553e7825e929["load_evaluators()"]
  1183e0f4_44a5_0674_c4b1_e5f99e15eda8["loading.py"]
  deb188fb_75ee_300b_4914_553e7825e929 -->|defined in| 1183e0f4_44a5_0674_c4b1_e5f99e15eda8
  cf30d9e6_90ba_3953_4427_17b28a12e180["load_evaluator()"]
  deb188fb_75ee_300b_4914_553e7825e929 -->|calls| cf30d9e6_90ba_3953_4427_17b28a12e180
  style deb188fb_75ee_300b_4914_553e7825e929 fill:#6366f1,stroke:#818cf8,color:#fff

Relationship Graph

Source Code

libs/langchain/langchain_classic/evaluation/loading.py lines 182–219

def load_evaluators(
    evaluators: Sequence[EvaluatorType],
    *,
    llm: BaseLanguageModel | None = None,
    config: dict | None = None,
    **kwargs: Any,
) -> list[Chain | StringEvaluator]:
    """Load evaluators specified by a list of evaluator types.

    Parameters
    ----------
    evaluators : Sequence[EvaluatorType]
        The list of evaluator types to load.
    llm : BaseLanguageModel, optional
        The language model to use for evaluation, if none is provided, a default
        ChatOpenAI gpt-4 model will be used.
    config : dict, optional
        A dictionary mapping evaluator types to additional keyword arguments,
        by default None
    **kwargs : Any
        Additional keyword arguments to pass to all evaluators.

    Returns:
    -------
    List[Chain]
        The loaded evaluators.

    Examples:
    --------
    >>> from langchain_classic.evaluation import load_evaluators, EvaluatorType
    >>> evaluators = [EvaluatorType.QA, EvaluatorType.CRITERIA]
    >>> loaded_evaluators = load_evaluators(evaluators, criteria="helpfulness")
    """
    loaded = []
    for evaluator in evaluators:
        _kwargs = config.get(evaluator, {}) if config else {}
        loaded.append(load_evaluator(evaluator, llm=llm, **{**kwargs, **_kwargs}))
    return loaded