langchain Architecture Docs
Home / Function/ test_usage_metadata() — langchain Function Reference

test_usage_metadata() — langchain Function Reference

Architecture documentation for the test_usage_metadata() function in chat_models.py from the langchain codebase.

Function python LangChainCore LanguageModelBase calls 5

Entity Profile

Dependency Diagram

graph TD
  e34f3878_b3a6_a60e_4abc_1db299378d2f["test_usage_metadata()"]
  971e928f_9c9b_ce7a_b93d_e762f2f5aa54["ChatModelIntegrationTests"]
  e34f3878_b3a6_a60e_4abc_1db299378d2f -->|defined in| 971e928f_9c9b_ce7a_b93d_e762f2f5aa54
  c8ace908_9e36_58d9_a682_297db2b2016a["invoke_with_audio_input()"]
  e34f3878_b3a6_a60e_4abc_1db299378d2f -->|calls| c8ace908_9e36_58d9_a682_297db2b2016a
  d1acec24_5fa6_76ac_c3aa_dfe9847046c0["invoke_with_audio_output()"]
  e34f3878_b3a6_a60e_4abc_1db299378d2f -->|calls| d1acec24_5fa6_76ac_c3aa_dfe9847046c0
  b5d5f3a0_cf7a_b6e1_b6b3_c97891b6f0a0["invoke_with_reasoning_output()"]
  e34f3878_b3a6_a60e_4abc_1db299378d2f -->|calls| b5d5f3a0_cf7a_b6e1_b6b3_c97891b6f0a0
  16308e84_efbf_1f25_6f4d_adf9ca75bc0e["invoke_with_cache_read_input()"]
  e34f3878_b3a6_a60e_4abc_1db299378d2f -->|calls| 16308e84_efbf_1f25_6f4d_adf9ca75bc0e
  3c44484f_fa9d_0b02_4503_c20ffefe584f["invoke_with_cache_creation_input()"]
  e34f3878_b3a6_a60e_4abc_1db299378d2f -->|calls| 3c44484f_fa9d_0b02_4503_c20ffefe584f
  style e34f3878_b3a6_a60e_4abc_1db299378d2f fill:#6366f1,stroke:#818cf8,color:#fff

Relationship Graph

Source Code

libs/standard-tests/langchain_tests/integration_tests/chat_models.py lines 1169–1343

    def test_usage_metadata(self, model: BaseChatModel) -> None:
        """Test to verify that the model returns correct usage metadata.

        This test is optional and should be skipped if the model does not return
        usage metadata (see configuration below).

        !!! warning "Behavior changed in `langchain-tests` 0.3.17"

            Additionally check for the presence of `model_name` in the response
            metadata, which is needed for usage tracking in callback handlers.

        ??? note "Configuration"

            By default, this test is run.

            To disable this feature, set `returns_usage_metadata` to `False` in your
            test class:

            ```python
            class TestMyChatModelIntegration(ChatModelIntegrationTests):
                @property
                def returns_usage_metadata(self) -> bool:
                    return False
            ```

            This test can also check the format of specific kinds of usage metadata
            based on the `supported_usage_metadata_details` property.

            This property should be configured as follows with the types of tokens that
            the model supports tracking:

            ```python
            class TestMyChatModelIntegration(ChatModelIntegrationTests):
                @property
                def supported_usage_metadata_details(self) -> dict:
                    return {
                        "invoke": [
                            "audio_input",
                            "audio_output",
                            "reasoning_output",
                            "cache_read_input",
                            "cache_creation_input",
                        ],
                        "stream": [
                            "audio_input",
                            "audio_output",
                            "reasoning_output",
                            "cache_read_input",
                            "cache_creation_input",
                        ],
                    }
            ```

        ??? question "Troubleshooting"

            If this test fails, first verify that your model returns
            `langchain_core.messages.ai.UsageMetadata` dicts
            attached to the returned `AIMessage` object in `_generate`:

            ```python
            return ChatResult(
                generations=[
                    ChatGeneration(
                        message=AIMessage(
                            content="Output text",
                            usage_metadata={
                                "input_tokens": 350,
                                "output_tokens": 240,
                                "total_tokens": 590,
                                "input_token_details": {
                                    "audio": 10,
                                    "cache_creation": 200,
                                    "cache_read": 100,
                                },
                                "output_token_details": {
                                    "audio": 10,
                                    "reasoning": 200,
                                },
                            },
                        )
                    )

Domain

Subdomains

Defined In

Calls

Source

Frequently Asked Questions

What does test_usage_metadata() do?
test_usage_metadata() is a function in the langchain codebase, defined in libs/standard-tests/langchain_tests/integration_tests/chat_models.py.
Where is test_usage_metadata() defined?
test_usage_metadata() is defined in libs/standard-tests/langchain_tests/integration_tests/chat_models.py at line 1169.
What does test_usage_metadata() call?
test_usage_metadata() calls 5 function(s): invoke_with_audio_input, invoke_with_audio_output, invoke_with_cache_creation_input, invoke_with_cache_read_input, invoke_with_reasoning_output.

Analyze Your Own Codebase

Get architecture documentation, dependency graphs, and domain analysis for your codebase in minutes.

Try Supermodel Free