KonlpyTextSplitter Class — langchain Architecture

Architecture documentation for the KonlpyTextSplitter class in konlpy.py from the langchain codebase.

Class python DocumentProcessing

Entity Profile

DocumentProcessing→ KonlpyTextSplitter Class — langchain Architecture

Dependency Diagram

graph TD
  0c8fb49b_b8f4_e4ab_dbf0_3a2ab3497af9["KonlpyTextSplitter"]
  c86e37d5_f962_cc1e_9821_b665e1359ae8["TextSplitter"]
  0c8fb49b_b8f4_e4ab_dbf0_3a2ab3497af9 -->|extends| c86e37d5_f962_cc1e_9821_b665e1359ae8
  f8344cc4_57af_1c2e_79df_d4d95c61c897["konlpy.py"]
  0c8fb49b_b8f4_e4ab_dbf0_3a2ab3497af9 -->|defined in| f8344cc4_57af_1c2e_79df_d4d95c61c897
  9f21ba81_3845_9037_a030_4df3006e880d["__init__()"]
  0c8fb49b_b8f4_e4ab_dbf0_3a2ab3497af9 -->|method| 9f21ba81_3845_9037_a030_4df3006e880d
  037c998c_cef5_1a9a_4e9b_e08cc58b04fe["split_text()"]
  0c8fb49b_b8f4_e4ab_dbf0_3a2ab3497af9 -->|method| 037c998c_cef5_1a9a_4e9b_e08cc58b04fe

Relationship Graph

Source Code

libs/text-splitters/langchain_text_splitters/konlpy.py lines 19–51

class KonlpyTextSplitter(TextSplitter):
    """Splitting text using Konlpy package.

    It is good for splitting Korean text.
    """

    def __init__(
        self,
        separator: str = "\n\n",
        **kwargs: Any,
    ) -> None:
        """Initialize the Konlpy text splitter.

        Args:
            separator: The separator to use when combining splits.

        Raises:
            ImportError: If Konlpy is not installed.
        """
        super().__init__(**kwargs)
        self._separator = separator
        if not _HAS_KONLPY:
            msg = """
                Konlpy is not installed, please install it with
                `pip install konlpy`
                """
            raise ImportError(msg)
        self.kkma = konlpy.tag.Kkma()

    @override
    def split_text(self, text: str) -> list[str]:
        splits = self.kkma.sentences(text)
        return self._merge_splits(splits, self._separator)