langchain Architecture Docs
Home / Function/ test_json_equality_evaluator_evaluate_lists_permutation_invariant() — langchain Function Reference

test_json_equality_evaluator_evaluate_lists_permutation_invariant() — langchain Function Reference

Architecture documentation for the test_json_equality_evaluator_evaluate_lists_permutation_invariant() function in test_base.py from the langchain codebase.

Function python LangChainCore LanguageModelBase

Entity Profile

Dependency Diagram

graph TD
  05f57c60_0dc6_c384_42fb_9532c7d3389d["test_json_equality_evaluator_evaluate_lists_permutation_invariant()"]
  5699e682_332a_59f6_5c7c_d1fcf8332e65["test_base.py"]
  05f57c60_0dc6_c384_42fb_9532c7d3389d -->|defined in| 5699e682_332a_59f6_5c7c_d1fcf8332e65
  style 05f57c60_0dc6_c384_42fb_9532c7d3389d fill:#6366f1,stroke:#818cf8,color:#fff

Relationship Graph

Source Code

libs/langchain/tests/unit_tests/evaluation/parsing/test_base.py lines 127–176

def test_json_equality_evaluator_evaluate_lists_permutation_invariant() -> None:
    evaluator = JsonEqualityEvaluator()
    prediction = '[{"a": 1, "b": 2}, {"a": 2, "b": 3}]'
    reference = '[{"a": 2, "b": 3}, {"a": 1, "b": 2}]'
    result = evaluator.evaluate_strings(prediction=prediction, reference=reference)
    assert result == {"score": True}

    prediction = '[{"a": 1, "b": 2}, {"a": 2, "b": 3}]'
    reference = '[{"a": 2, "b": 3}, {"a": 1, "b": 4}]'
    result = evaluator.evaluate_strings(prediction=prediction, reference=reference)
    assert result == {"score": False}

    prediction = '[{"a": 1, "b": 2}, {"a": 2, "b": 3}]'
    reference = '[{"a": 2, "b": 3}]'
    result = evaluator.evaluate_strings(prediction=prediction, reference=reference)
    assert result == {"score": False}

    prediction = '[{"a": 1, "b": 2}, {"a": 2, "b": 3}]'
    reference = '[{"a": 2, "b": 3}, {"a": 1, "b": 2}, {"a": 3, "b": 4}]'
    result = evaluator.evaluate_strings(prediction=prediction, reference=reference)
    assert result == {"score": False}

    prediction = '[{"a": 1, "b": 2}, {"a": 2, "b": 3}]'
    reference = '[{"a": 2, "b": 3}, {"b": 2,"a": 1}, {"a": 3, "b": 4}]'
    result = evaluator.evaluate_strings(prediction=reference, reference=prediction)
    assert result == {"score": False}

    # Limit tests
    prediction = (
        "[" + ",".join([f'{{"a": {i}, "b": {i + 1}}}' for i in range(1000)]) + "]"
    )
    rlist = [f'{{"a": {i}, "b": {i + 1}}}' for i in range(1000)]
    random.shuffle(rlist)
    reference = "[" + ",".join(rlist) + "]"
    result = evaluator.evaluate_strings(prediction=prediction, reference=reference)
    assert result == {"score": True}

    prediction = (
        "[" + ",".join([f'{{"b": {i + 1}, "a": {i}}}' for i in range(1000)]) + "]"
    )
    reference = (
        "["
        + ",".join(
            [f'{{"a": {i + 1}, "b": {i + 2}}}' for i in range(999)]
            + ['{"a": 1000, "b": 1001}'],
        )
        + "]"
    )
    result = evaluator.evaluate_strings(prediction=prediction, reference=reference)
    assert result == {"score": False}

Domain

Subdomains

Defined In

Source

Frequently Asked Questions

What does test_json_equality_evaluator_evaluate_lists_permutation_invariant() do?
test_json_equality_evaluator_evaluate_lists_permutation_invariant() is a function in the langchain codebase, defined in libs/langchain/tests/unit_tests/evaluation/parsing/test_base.py.
Where is test_json_equality_evaluator_evaluate_lists_permutation_invariant() defined?
test_json_equality_evaluator_evaluate_lists_permutation_invariant() is defined in libs/langchain/tests/unit_tests/evaluation/parsing/test_base.py at line 127.

Analyze Your Own Codebase

Get architecture documentation, dependency graphs, and domain analysis for your codebase in minutes.

Try Supermodel Free