_process_links() — langchain Function Reference

Architecture documentation for the _process_links() function in html.py from the langchain codebase.

Function python DocumentProcessing TextSplitters calls 1 called by 1

Entity Profile

DocumentProcessing→ TextSplitters→ _process_links() — langchain Function Reference

Dependency Diagram

graph TD
  ff63d8f1_7353_0b16_2f96_7dadb57a8348["_process_links()"]
  5af47ada_f6e1_33df_ed07_12ca64351fa0["HTMLSemanticPreservingSplitter"]
  ff63d8f1_7353_0b16_2f96_7dadb57a8348 -->|defined in| 5af47ada_f6e1_33df_ed07_12ca64351fa0
  127c75d0_d814_d16e_a93c_928f021add9c["split_text()"]
  127c75d0_d814_d16e_a93c_928f021add9c -->|calls| ff63d8f1_7353_0b16_2f96_7dadb57a8348
  4134a695_a3ab_4bed_f7a0_3a766652fc3e["_find_all_tags()"]
  ff63d8f1_7353_0b16_2f96_7dadb57a8348 -->|calls| 4134a695_a3ab_4bed_f7a0_3a766652fc3e
  style ff63d8f1_7353_0b16_2f96_7dadb57a8348 fill:#6366f1,stroke:#818cf8,color:#fff

Relationship Graph

Source Code

libs/text-splitters/langchain_text_splitters/html.py lines 796–808

    def _process_links(soup: BeautifulSoup) -> None:
        """Processes the links in the HTML content.

        Args:
            soup: Parsed HTML content using BeautifulSoup.
        """
        for a_tag in _find_all_tags(soup, name="a"):
            a_href = a_tag.get("href", "")
            a_text = a_tag.get_text(strip=True)
            markdown_link = f"[{a_text}]({a_href})"
            wrapper = soup.new_tag("link-wrapper")
            wrapper.string = markdown_link
            a_tag.replace_with(NavigableString(markdown_link))