ChatOllama Class — langchain Architecture

Source Code

libs/partners/ollama/langchain_ollama/chat_models.py lines 260–1606

class ChatOllama(BaseChatModel):
    r"""Ollama chat model integration.

    ???+ note "Setup"

        Install `langchain-ollama` and download any models you want to use from ollama.

        ```bash
        ollama pull gpt-oss:20b
        pip install -U langchain-ollama
        ```

    Key init args — completion params:
        model: str
            Name of Ollama model to use.
        reasoning: bool | None
            Controls the reasoning/thinking mode for
            [supported models](https://ollama.com/search?c=thinking).

            - `True`: Enables reasoning mode. The model's reasoning process will be
                captured and returned separately in the `additional_kwargs` of the
                response message, under `reasoning_content`. The main response
                content will not include the reasoning tags.
            - `False`: Disables reasoning mode. The model will not perform any reasoning,
                and the response will not include any reasoning content.
            - `None` (Default): The model will use its default reasoning behavior. Note
                however, if the model's default behavior *is* to perform reasoning, think tags
                (`<think>` and `</think>`) will be present within the main response content
                unless you set `reasoning` to `True`.
        temperature: float
            Sampling temperature. Ranges from `0.0` to `1.0`.
        num_predict: int | None
            Max number of tokens to generate.

    See full list of supported init args and their descriptions in the params section.

    Instantiate:
        ```python
        from langchain_ollama import ChatOllama

        model = ChatOllama(
            model="gpt-oss:20b",
            validate_model_on_init=True,
            temperature=0.8,
            num_predict=256,
            # other params ...
        )
        ```

    Invoke:
        ```python
        messages = [
            ("system", "You are a helpful translator. Translate the user sentence to French."),
            ("human", "I love programming."),
        ]
        model.invoke(messages)
        ```

        ```python
        AIMessage(content='J'adore le programmation. (Note: "programming" can also refer to the act of writing code, so if you meant that, I could translate it as "J'adore programmer". But since you didn\'t specify, I assumed you were talking about the activity itself, which is what "le programmation" usually refers to.)', response_metadata={'model': 'llama3', 'created_at': '2024-07-04T03:37:50.182604Z', 'message': {'role': 'assistant', 'content': ''}, 'done_reason': 'stop', 'done': True, 'total_duration': 3576619666, 'load_duration': 788524916, 'prompt_eval_count': 32, 'prompt_eval_duration': 128125000, 'eval_count': 71, 'eval_duration': 2656556000}, id='run-ba48f958-6402-41a5-b461-5e250a4ebd36-0')
        ```

    Stream:
        ```python
        for chunk in model.stream("Return the words Hello World!"):
            print(chunk.text, end="")
        ```

        ```python
        content='Hello' id='run-327ff5ad-45c8-49fe-965c-0a93982e9be1'
        content=' World' id='run-327ff5ad-45c8-49fe-965c-0a93982e9be1'
        content='!' id='run-327ff5ad-45c8-49fe-965c-0a93982e9be1'
        content='' response_metadata={'model': 'llama3', 'created_at': '2024-07-04T03:39:42.274449Z', 'message': {'role': 'assistant', 'content': ''}, 'done_reason': 'stop', 'done': True, 'total_duration': 411875125, 'load_duration': 1898166, 'prompt_eval_count': 14, 'prompt_eval_duration': 297320000, 'eval_count': 4, 'eval_duration': 111099000} id='run-327ff5ad-45c8-49fe-965c-0a93982e9be1'

        ```

        ```python
        stream = model.stream(messages)
        full = next(stream)
        for chunk in stream:
            full += chunk