langchain Architecture Docs
Home / Function/ test_count_tokens_approximately_custom_token_length() — langchain Function Reference

test_count_tokens_approximately_custom_token_length() — langchain Function Reference

Architecture documentation for the test_count_tokens_approximately_custom_token_length() function in test_utils.py from the langchain codebase.

Function python CoreAbstractions MessageSchema

Entity Profile

Dependency Diagram

graph TD
  be3bed41_da58_74fc_6ed8_cd6b6c7b8a20["test_count_tokens_approximately_custom_token_length()"]
  ea3f8d89_f44b_6738_3cb9_a740a73cfca4["test_utils.py"]
  be3bed41_da58_74fc_6ed8_cd6b6c7b8a20 -->|defined in| ea3f8d89_f44b_6738_3cb9_a740a73cfca4
  style be3bed41_da58_74fc_6ed8_cd6b6c7b8a20 fill:#6366f1,stroke:#818cf8,color:#fff

Relationship Graph

Source Code

libs/core/tests/unit_tests/messages/test_utils.py lines 1551–1559

def test_count_tokens_approximately_custom_token_length() -> None:
    messages = [
        # 11 chars + 4 role chars -> (4 tokens of length 4 / 8 tokens of length 2) + 3
        HumanMessage(content="Hello world"),
        # 7 chars + 9 role chars -> (4 tokens of length 4 / 8 tokens of length 2) + 3
        AIMessage(content="Testing"),
    ]
    assert count_tokens_approximately(messages, chars_per_token=4) == 14
    assert count_tokens_approximately(messages, chars_per_token=2) == 22

Domain

Subdomains

Defined In

Source

Frequently Asked Questions

What does test_count_tokens_approximately_custom_token_length() do?
test_count_tokens_approximately_custom_token_length() is a function in the langchain codebase, defined in libs/core/tests/unit_tests/messages/test_utils.py.
Where is test_count_tokens_approximately_custom_token_length() defined?
test_count_tokens_approximately_custom_token_length() is defined in libs/core/tests/unit_tests/messages/test_utils.py at line 1551.

Analyze Your Own Codebase

Get architecture documentation, dependency graphs, and domain analysis for your codebase in minutes.

Try Supermodel Free