evaluate_run() — langchain Function Reference

Architecture documentation for the evaluate_run() function in string_run_evaluator.py from the langchain codebase.

Function python LangChainCore ApiManagement calls 1

Entity Profile

LangChainCore→ ApiManagement→ evaluate_run() — langchain Function Reference

Dependency Diagram

graph TD
  d591bff6_3d3a_7aa7_e2e2_5d3743d9a6e0["evaluate_run()"]
  96e174c1_4107_a88b_8d98_3f22e5d7fb20["StringRunEvaluatorChain"]
  d591bff6_3d3a_7aa7_e2e2_5d3743d9a6e0 -->|defined in| 96e174c1_4107_a88b_8d98_3f22e5d7fb20
  2029cd8d_b9d5_3985_a052_b3b4c7b21157["_prepare_evaluator_output()"]
  d591bff6_3d3a_7aa7_e2e2_5d3743d9a6e0 -->|calls| 2029cd8d_b9d5_3985_a052_b3b4c7b21157
  style d591bff6_3d3a_7aa7_e2e2_5d3743d9a6e0 fill:#6366f1,stroke:#818cf8,color:#fff

Relationship Graph

Source Code

libs/langchain/langchain_classic/smith/evaluation/string_run_evaluator.py lines 366–382

    def evaluate_run(
        self,
        run: Run,
        example: Example | None = None,
        evaluator_run_id: uuid.UUID | None = None,
    ) -> EvaluationResult:
        """Evaluate an example."""
        try:
            result = self({"run": run, "example": example}, include_run_info=True)
            return self._prepare_evaluator_output(result)
        except Exception as e:
            _logger.exception("Error evaluating run %s", run.id)
            return EvaluationResult(
                key=self.string_evaluator.evaluation_name,
                comment=f"Error evaluating run {run.id}: {e}",
                # TODO: Add run ID once we can declare it via callbacks
            )