HuggingFacePipeline Class — langchain Architecture

Architecture documentation for the HuggingFacePipeline class in huggingface_pipeline.py from the langchain codebase.

Class python

Entity Profile

Dependency Diagram

graph TD
  54333c82_6644_5574_2c41_4cc818ce3595["HuggingFacePipeline"]
  ce4aa464_3868_179e_5d99_df48bc307c5f["BaseLLM"]
  54333c82_6644_5574_2c41_4cc818ce3595 -->|extends| ce4aa464_3868_179e_5d99_df48bc307c5f
  b87ebf28_5f26_0280_8cb7_68afdab14557["huggingface_pipeline.py"]
  54333c82_6644_5574_2c41_4cc818ce3595 -->|defined in| b87ebf28_5f26_0280_8cb7_68afdab14557
  624ca728_01b5_a4c5_5ab2_bee385c4f3e8["pre_init_validator()"]
  54333c82_6644_5574_2c41_4cc818ce3595 -->|method| 624ca728_01b5_a4c5_5ab2_bee385c4f3e8
  834807da_062f_4f2f_6975_506464144209["from_model_id()"]
  54333c82_6644_5574_2c41_4cc818ce3595 -->|method| 834807da_062f_4f2f_6975_506464144209
  966c8875_5d22_7b00_d348_6d0ba6dc1bde["_identifying_params()"]
  54333c82_6644_5574_2c41_4cc818ce3595 -->|method| 966c8875_5d22_7b00_d348_6d0ba6dc1bde
  d9a1b039_8a8d_d293_2ff2_8062b008602d["_llm_type()"]
  54333c82_6644_5574_2c41_4cc818ce3595 -->|method| d9a1b039_8a8d_d293_2ff2_8062b008602d
  21c788c9_e3fd_3763_dc05_03fab3c9eba5["_generate()"]
  54333c82_6644_5574_2c41_4cc818ce3595 -->|method| 21c788c9_e3fd_3763_dc05_03fab3c9eba5
  fada44e9_5715_9209_4a97_631e6f198461["_stream()"]
  54333c82_6644_5574_2c41_4cc818ce3595 -->|method| fada44e9_5715_9209_4a97_631e6f198461

Relationship Graph

Source Code

libs/partners/huggingface/langchain_huggingface/llms/huggingface_pipeline.py lines 37–422

class HuggingFacePipeline(BaseLLM):
    """HuggingFace Pipeline API.

    To use, you should have the `transformers` python package installed.

    Only supports `text-generation`, `text2text-generation`, `image-text-to-text`,
    `summarization` and `translation`  for now.

    Example using from_model_id:
        ```python
        from langchain_huggingface import HuggingFacePipeline

        hf = HuggingFacePipeline.from_model_id(
            model_id="gpt2",
            task="text-generation",
            pipeline_kwargs={"max_new_tokens": 10},
        )
        ```

    Example passing pipeline in directly:
        ```python
        from langchain_huggingface import HuggingFacePipeline
        from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline

        model_id = "gpt2"
        tokenizer = AutoTokenizer.from_pretrained(model_id)
        model = AutoModelForCausalLM.from_pretrained(model_id)
        pipe = pipeline(
            "text-generation",
            model=model,
            tokenizer=tokenizer,
            max_new_tokens=10,
        )
        hf = HuggingFacePipeline(pipeline=pipe)
        ```
    """

    pipeline: Any = None

    model_id: str | None = None
    """The model name. If not set explicitly by the user,
    it will be inferred from the provided pipeline (if available).
    If neither is provided, the DEFAULT_MODEL_ID will be used."""

    model_kwargs: dict | None = None
    """Keyword arguments passed to the model."""

    pipeline_kwargs: dict | None = None
    """Keyword arguments passed to the pipeline."""

    batch_size: int = DEFAULT_BATCH_SIZE
    """Batch size to use when passing multiple documents to generate."""

    model_config = ConfigDict(
        extra="forbid",
    )

    @model_validator(mode="before")
    @classmethod
    def pre_init_validator(cls, values: dict[str, Any]) -> dict[str, Any]:
        """Ensure model_id is set either by pipeline or user input."""
        if "model_id" not in values:
            if values.get("pipeline"):
                values["model_id"] = values["pipeline"].model.name_or_path
            else:
                values["model_id"] = DEFAULT_MODEL_ID
        return values

    @classmethod
    def from_model_id(
        cls,
        model_id: str,
        task: str,
        backend: str = "default",
        device: int | None = None,
        device_map: str | None = None,
        model_kwargs: dict | None = None,
        pipeline_kwargs: dict | None = None,
        batch_size: int = DEFAULT_BATCH_SIZE,
        **kwargs: Any,
    ) -> HuggingFacePipeline: