langchain Architecture Docs
Home / File/ test_runner_utils.py — langchain Source File

test_runner_utils.py — langchain Source File

Architecture documentation for test_runner_utils.py, a python file in the langchain codebase. 13 imports, 0 dependents.

File python CoreAbstractions MessageSchema 13 imports 14 functions

Entity Profile

Dependency Diagram

graph LR
  7f25b56c_cd66_13e0_281a_a50e80a0f23b["test_runner_utils.py"]
  8dfa0cac_d802_3ccd_f710_43a5e70da3a5["uuid"]
  7f25b56c_cd66_13e0_281a_a50e80a0f23b --> 8dfa0cac_d802_3ccd_f710_43a5e70da3a5
  cfe2bde5_180e_e3b0_df2b_55b3ebaca8e7["collections.abc"]
  7f25b56c_cd66_13e0_281a_a50e80a0f23b --> cfe2bde5_180e_e3b0_df2b_55b3ebaca8e7
  af34f08b_0ede_2b87_0db6_983d74ed0249["datetime"]
  7f25b56c_cd66_13e0_281a_a50e80a0f23b --> af34f08b_0ede_2b87_0db6_983d74ed0249
  8e2034b7_ceb8_963f_29fc_2ea6b50ef9b3["typing"]
  7f25b56c_cd66_13e0_281a_a50e80a0f23b --> 8e2034b7_ceb8_963f_29fc_2ea6b50ef9b3
  6ebcaae2_3bc1_badf_b751_e164ff2776c4["unittest"]
  7f25b56c_cd66_13e0_281a_a50e80a0f23b --> 6ebcaae2_3bc1_badf_b751_e164ff2776c4
  120e2591_3e15_b895_72b6_cb26195e40a6["pytest"]
  7f25b56c_cd66_13e0_281a_a50e80a0f23b --> 120e2591_3e15_b895_72b6_cb26195e40a6
  0a0b1957_08a9_6988_c388_b08d8a4c17ad["freezegun"]
  7f25b56c_cd66_13e0_281a_a50e80a0f23b --> 0a0b1957_08a9_6988_c388_b08d8a4c17ad
  54dedaf3_11d6_7e5e_cfd7_a3ee7571bff1["langsmith.client"]
  7f25b56c_cd66_13e0_281a_a50e80a0f23b --> 54dedaf3_11d6_7e5e_cfd7_a3ee7571bff1
  ee1d35f7_eb2d_f3ba_5b09_7f4815d92b12["langsmith.schemas"]
  7f25b56c_cd66_13e0_281a_a50e80a0f23b --> ee1d35f7_eb2d_f3ba_5b09_7f4815d92b12
  08e4321c_76f7_5a89_d283_e4d93734a53b["langchain_classic.chains.transform"]
  7f25b56c_cd66_13e0_281a_a50e80a0f23b --> 08e4321c_76f7_5a89_d283_e4d93734a53b
  80ba050c_b2d4_debe_4a34_dd7112c2d790["langchain_classic.smith.evaluation.runner_utils"]
  7f25b56c_cd66_13e0_281a_a50e80a0f23b --> 80ba050c_b2d4_debe_4a34_dd7112c2d790
  28442c0b_15db_9749_e71f_8fb95692f0f9["tests.unit_tests.llms.fake_chat_model"]
  7f25b56c_cd66_13e0_281a_a50e80a0f23b --> 28442c0b_15db_9749_e71f_8fb95692f0f9
  7e88f5ce_ff41_7d87_8fb2_f355489a149e["tests.unit_tests.llms.fake_llm"]
  7f25b56c_cd66_13e0_281a_a50e80a0f23b --> 7e88f5ce_ff41_7d87_8fb2_f355489a149e
  style 7f25b56c_cd66_13e0_281a_a50e80a0f23b fill:#6366f1,stroke:#818cf8,color:#fff

Relationship Graph

Source Code

"""Test the LangSmith evaluation helpers."""

import uuid
from collections.abc import Iterator
from datetime import datetime, timezone
from typing import Any
from unittest import mock

import pytest
from freezegun import freeze_time
from langsmith.client import Client
from langsmith.schemas import Dataset, Example

from langchain_classic.chains.transform import TransformChain
from langchain_classic.smith.evaluation.runner_utils import (
    InputFormatError,
    _get_messages,
    _get_prompt,
    _run_llm,
    _run_llm_or_chain,
    _validate_example_inputs_for_chain,
    _validate_example_inputs_for_language_model,
    arun_on_dataset,
)
from tests.unit_tests.llms.fake_chat_model import FakeChatModel
from tests.unit_tests.llms.fake_llm import FakeLLM

_CREATED_AT = datetime(2015, 1, 1, 0, 0, 0, tzinfo=timezone.utc)
_TENANT_ID = "7a3d2b56-cd5b-44e5-846f-7eb6e8144ce4"
_EXAMPLE_MESSAGE = {
    "data": {"content": "Foo", "example": False, "additional_kwargs": {}},
    "type": "human",
}
_VALID_MESSAGES = [
    {"messages": [_EXAMPLE_MESSAGE], "other_key": "value"},
    {"messages": [], "other_key": "value"},
    {
        "messages": [[_EXAMPLE_MESSAGE, _EXAMPLE_MESSAGE]],
        "other_key": "value",
    },
    {"any_key": [_EXAMPLE_MESSAGE]},
    {"any_key": [[_EXAMPLE_MESSAGE, _EXAMPLE_MESSAGE]]},
]
_VALID_PROMPTS = [
    {"prompts": ["foo"], "other_key": "value"},
    {"prompt": "foo", "other_key": ["bar", "baz"]},
    {"some_key": "foo"},
    {"some_key": ["foo"]},
]

_INVALID_PROMPTS = (
    [
        {"prompts": "foo"},
        {"prompt": ["foo"]},
        {"some_key": 3},
        {"some_key": "foo", "other_key": "bar"},
    ],
)


// ... (295 more lines)

Domain

Subdomains

Functions

Dependencies

  • collections.abc
  • datetime
  • freezegun
  • langchain_classic.chains.transform
  • langchain_classic.smith.evaluation.runner_utils
  • langsmith.client
  • langsmith.schemas
  • pytest
  • tests.unit_tests.llms.fake_chat_model
  • tests.unit_tests.llms.fake_llm
  • typing
  • unittest
  • uuid

Source

Frequently Asked Questions

What does test_runner_utils.py do?
test_runner_utils.py is a source file in the langchain codebase, written in python. It belongs to the CoreAbstractions domain, MessageSchema subdomain.
What functions are defined in test_runner_utils.py?
test_runner_utils.py defines 14 function(s): test__get_messages_invalid, test__get_messages_valid, test__get_prompts_invalid, test__get_prompts_valid, test__validate_example_inputs_for_chain_input_mapper, test__validate_example_inputs_for_chain_multi_io, test__validate_example_inputs_for_chain_single_input, test__validate_example_inputs_for_chain_single_input_multi_expect, test__validate_example_inputs_for_language_model, test__validate_example_inputs_for_language_model_invalid, and 4 more.
What does test_runner_utils.py depend on?
test_runner_utils.py imports 13 module(s): collections.abc, datetime, freezegun, langchain_classic.chains.transform, langchain_classic.smith.evaluation.runner_utils, langsmith.client, langsmith.schemas, pytest, and 5 more.
Where is test_runner_utils.py in the architecture?
test_runner_utils.py is located at libs/langchain/tests/unit_tests/smith/evaluation/test_runner_utils.py (domain: CoreAbstractions, subdomain: MessageSchema, directory: libs/langchain/tests/unit_tests/smith/evaluation).

Analyze Your Own Codebase

Get architecture documentation, dependency graphs, and domain analysis for your codebase in minutes.

Try Supermodel Free