split_text() — langchain Function Reference

Architecture documentation for the split_text() function in spacy.py from the langchain codebase.

Function python DocumentProcessing TextSplitters calls 1 called by 1

Entity Profile

DocumentProcessing→ TextSplitters→ split_text() — langchain Function Reference

Dependency Diagram

graph TD
  4d3df53d_617e_94a4_daa1_eb56f7f8dd2a["split_text()"]
  f70dde90_12ee_c4f8_5e48_48f30085deb1["SpacyTextSplitter"]
  4d3df53d_617e_94a4_daa1_eb56f7f8dd2a -->|defined in| f70dde90_12ee_c4f8_5e48_48f30085deb1
  4d3df53d_617e_94a4_daa1_eb56f7f8dd2a["split_text()"]
  4d3df53d_617e_94a4_daa1_eb56f7f8dd2a -->|calls| 4d3df53d_617e_94a4_daa1_eb56f7f8dd2a
  4d3df53d_617e_94a4_daa1_eb56f7f8dd2a["split_text()"]
  4d3df53d_617e_94a4_daa1_eb56f7f8dd2a -->|calls| 4d3df53d_617e_94a4_daa1_eb56f7f8dd2a
  style 4d3df53d_617e_94a4_daa1_eb56f7f8dd2a fill:#6366f1,stroke:#818cf8,color:#fff

Relationship Graph

Source Code

libs/text-splitters/langchain_text_splitters/spacy.py lines 53–58

    def split_text(self, text: str) -> list[str]:
        splits = (
            s.text if self._strip_whitespace else s.text_with_ws
            for s in self._tokenizer(text).sents
        )
        return self._merge_splits(splits, self._separator)