get_aggregate_feedback() — langchain Function Reference

Architecture documentation for the get_aggregate_feedback() function in runner_utils.py from the langchain codebase.

Function python LangChainCore MessageInterface calls 1 called by 1

Entity Profile

LangChainCore→ MessageInterface→ get_aggregate_feedback() — langchain Function Reference

Dependency Diagram

graph TD
  e45d7c4e_b434_e04f_5d6a_822b3cb84b5a["get_aggregate_feedback()"]
  2faa7d51_38e2_8d63_1f1e_b482b4401e76["TestResult"]
  e45d7c4e_b434_e04f_5d6a_822b3cb84b5a -->|defined in| 2faa7d51_38e2_8d63_1f1e_b482b4401e76
  f2fb82ef_40a0_07e3_1d8e_3a52a5a502ce["finish()"]
  f2fb82ef_40a0_07e3_1d8e_3a52a5a502ce -->|calls| e45d7c4e_b434_e04f_5d6a_822b3cb84b5a
  c8c431c9_edb6_7a8a_6dd8_508fa675ba6c["to_dataframe()"]
  e45d7c4e_b434_e04f_5d6a_822b3cb84b5a -->|calls| c8c431c9_edb6_7a8a_6dd8_508fa675ba6c
  style e45d7c4e_b434_e04f_5d6a_822b3cb84b5a fill:#6366f1,stroke:#818cf8,color:#fff

Relationship Graph

Source Code

libs/langchain/langchain_classic/smith/evaluation/runner_utils.py lines 83–102

    def get_aggregate_feedback(
        self,
    ) -> pd.DataFrame:
        """Return quantiles for the feedback scores.

        This method calculates and prints the quantiles for the feedback scores
        across all feedback keys.

        Returns:
            A DataFrame containing the quantiles for each feedback key.
        """
        df = self.to_dataframe()
        # Drop all things starting with inputs., outputs., and reference
        to_drop = [
            col
            for col in df.columns
            if col.startswith(("inputs.", "outputs.", "reference"))
            or col in {"input", "output"}
        ]
        return df.describe(include="all").drop(to_drop, axis=1)