test_nltk_text_splitter() — langchain Function Reference

Architecture documentation for the test_nltk_text_splitter() function in test_nlp_text_splitters.py from the langchain codebase.

Function python LangChainCore ApiManagement

Entity Profile

LangChainCore→ ApiManagement→ test_nltk_text_splitter() — langchain Function Reference

Dependency Diagram

graph TD
  db64dbcd_ab72_88fe_32d3_ef2d664f0d4c["test_nltk_text_splitter()"]
  a159cbba_51f0_5d34_8696_299b594bb0fe["test_nlp_text_splitters.py"]
  db64dbcd_ab72_88fe_32d3_ef2d664f0d4c -->|defined in| a159cbba_51f0_5d34_8696_299b594bb0fe
  style db64dbcd_ab72_88fe_32d3_ef2d664f0d4c fill:#6366f1,stroke:#818cf8,color:#fff

Relationship Graph

Source Code

libs/text-splitters/tests/integration_tests/test_nlp_text_splitters.py lines 56–63

def test_nltk_text_splitter() -> None:
    """Test splitting by sentence using NLTK."""
    text = "This is sentence one. And this is sentence two."
    separator = "|||"
    splitter = NLTKTextSplitter(separator=separator)
    output = splitter.split_text(text)
    expected_output = [f"This is sentence one.{separator}And this is sentence two."]
    assert output == expected_output