_collect_test_results() — langchain Function Reference

Architecture documentation for the _collect_test_results() function in runner_utils.py from the langchain codebase.

Function python LangChainCore ApiManagement calls 3 called by 1

Entity Profile

LangChainCore→ ApiManagement→ _collect_test_results() — langchain Function Reference

Dependency Diagram

graph TD
  7bd0a459_a7f0_719c_faf9_2cf0ffd65a8c["_collect_test_results()"]
  3aaa6e94_b6a8_1c13_86d0_1709a1d93909["_DatasetRunContainer"]
  7bd0a459_a7f0_719c_faf9_2cf0ffd65a8c -->|defined in| 3aaa6e94_b6a8_1c13_86d0_1709a1d93909
  f2fb82ef_40a0_07e3_1d8e_3a52a5a502ce["finish()"]
  f2fb82ef_40a0_07e3_1d8e_3a52a5a502ce -->|calls| 7bd0a459_a7f0_719c_faf9_2cf0ffd65a8c
  983ed1c6_8485_7927_a832_f9e88ee9bb16["_collect_metrics()"]
  7bd0a459_a7f0_719c_faf9_2cf0ffd65a8c -->|calls| 983ed1c6_8485_7927_a832_f9e88ee9bb16
  31505a45_4c7a_ea20_3a86_a98f76b1d2b7["_run_batch_evaluators()"]
  7bd0a459_a7f0_719c_faf9_2cf0ffd65a8c -->|calls| 31505a45_4c7a_ea20_3a86_a98f76b1d2b7
  bb416a9d_32e6_298b_d0bd_82d7d53d9821["_merge_test_outputs()"]
  7bd0a459_a7f0_719c_faf9_2cf0ffd65a8c -->|calls| bb416a9d_32e6_298b_d0bd_82d7d53d9821
  style 7bd0a459_a7f0_719c_faf9_2cf0ffd65a8c fill:#6366f1,stroke:#818cf8,color:#fff

Relationship Graph

Source Code

libs/langchain/langchain_classic/smith/evaluation/runner_utils.py lines 1180–1196

    def _collect_test_results(
        self,
        batch_results: list[dict | str | LLMResult | ChatResult],
    ) -> TestResult:
        logger.info("Waiting for evaluators to complete.")
        wait_for_all_evaluators()
        all_eval_results, all_runs = self._collect_metrics()
        aggregate_feedback = None
        if self.batch_evaluators:
            logger.info("Running session evaluators.")
            aggregate_feedback = self._run_batch_evaluators(all_runs)
        results = self._merge_test_outputs(batch_results, all_eval_results)
        return TestResult(
            project_name=self.project.name,
            results=results,
            aggregate_metrics=aggregate_feedback,
        )