ChatHuggingFace Class — langchain Architecture

Source Code

libs/partners/huggingface/langchain_huggingface/chat_models/huggingface.py lines 324–1247

class ChatHuggingFace(BaseChatModel):
    r"""Hugging Face LLM's as ChatModels.

    Works with `HuggingFaceTextGenInference`, `HuggingFaceEndpoint`,
    `HuggingFaceHub`, and `HuggingFacePipeline` LLMs.

    Upon instantiating this class, the model_id is resolved from the url
    provided to the LLM, and the appropriate tokenizer is loaded from
    the HuggingFace Hub.

    Setup:
        Install `langchain-huggingface` and ensure your Hugging Face token
        is saved.

        ```bash
        pip install langchain-huggingface
        ```

        ```python
        from huggingface_hub import login

        login()  # You will be prompted for your HF key, which will then be saved locally
        ```

    Key init args — completion params:
        llm:
            LLM to be used.

    Key init args — client params:
        custom_get_token_ids:
            Optional encoder to use for counting tokens.
        metadata:
            Metadata to add to the run trace.
        tags:
            Tags to add to the run trace.
        verbose:
            Whether to print out response text.

    See full list of supported init args and their descriptions in the params
    section.

    Instantiate:
        ```python
        from langchain_huggingface import HuggingFaceEndpoint,
        ChatHuggingFace

        model = HuggingFaceEndpoint(
            repo_id="microsoft/Phi-3-mini-4k-instruct",
            task="text-generation",
            max_new_tokens=512,
            do_sample=False,
            repetition_penalty=1.03,
        )

        chat = ChatHuggingFace(llm=model, verbose=True)
        ```

    Invoke:
        ```python
        messages = [
            ("system", "You are a helpful translator. Translate the user
            sentence to French."),
            ("human", "I love programming."),
        ]

        chat(...).invoke(messages)
        ```

        ```python
        AIMessage(content='Je ai une passion pour le programme.\n\nIn
        French, we use "ai" for masculine subjects and "a" for feminine
        subjects. Since "programming" is gender-neutral in English, we
        will go with the masculine "programme".\n\nConfirmation: "J\'aime
        le programme." is more commonly used. The sentence above is
        technically accurate, but less commonly used in spoken French as
        "ai" is used less frequently in everyday speech.',
        response_metadata={'token_usage': ChatCompletionOutputUsage
        (completion_tokens=100, prompt_tokens=55, total_tokens=155),
        'model': '', 'finish_reason': 'length'},
        id='run-874c24b7-0272-4c99-b259-5d6d7facbc56-0')
        ```