test_image_tool_message() — langchain Function Reference

Architecture documentation for the test_image_tool_message() function in chat_models.py from the langchain codebase.

Function python LangChainCore LanguageModelBase

Entity Profile

LangChainCore→ LanguageModelBase→ test_image_tool_message() — langchain Function Reference

Dependency Diagram

graph TD
  c14b5c1a_34c0_d2c2_653d_4c67fb6cd418["test_image_tool_message()"]
  971e928f_9c9b_ce7a_b93d_e762f2f5aa54["ChatModelIntegrationTests"]
  c14b5c1a_34c0_d2c2_653d_4c67fb6cd418 -->|defined in| 971e928f_9c9b_ce7a_b93d_e762f2f5aa54
  style c14b5c1a_34c0_d2c2_653d_4c67fb6cd418 fill:#6366f1,stroke:#818cf8,color:#fff

Relationship Graph

Source Code

libs/standard-tests/langchain_tests/integration_tests/chat_models.py lines 2827–2941

    def test_image_tool_message(self, model: BaseChatModel) -> None:
        """Test that the model can process `ToolMessage` objects with image inputs.

        This test should be skipped if the model does not support messages of the
        Chat Completions `image_url` format:

        ```python
        ToolMessage(
            content=[
                {
                    "type": "image_url",
                    "image_url": {"url": f"data:image/jpeg;base64,{image_data}"},
                },
            ],
            tool_call_id="1",
            name="random_image",
        )
        ```

        In addition, models should support the standard LangChain `ImageContentBlock`
        format:

        ```python
        ToolMessage(
            content=[
                {
                    "type": "image",
                    "base64": image_data,
                    "mime_type": "image/jpeg",
                },
            ],
            tool_call_id="1",
            name="random_image",
        )
        ```

        This test can be skipped by setting the `supports_image_tool_message` property
        to `False` (see configuration below).

        ??? note "Configuration"

            To disable this test, set `supports_image_tool_message` to `False` in your
            test class:

            ```python
            class TestMyChatModelIntegration(ChatModelIntegrationTests):
                @property
                def supports_image_tool_message(self) -> bool:
                    return False
            ```

        ??? question "Troubleshooting"

            If this test fails, check that the model can correctly handle messages
            with image content blocks in `ToolMessage` objects, including base64-encoded
            images. Otherwise, set the `supports_image_tool_message` property to
            `False`.

        """
        if not self.supports_image_tool_message:
            pytest.skip("Model does not support image tool message.")

        image_url = "https://raw.githubusercontent.com/langchain-ai/docs/4d11d08b6b0e210bd456943f7a22febbd168b543/src/images/agentic-rag-output.png"
        image_data = base64.b64encode(httpx.get(image_url).content).decode("utf-8")

        # OpenAI CC format, base64 data
        oai_format_message = ToolMessage(
            content=[
                {
                    "type": "image_url",
                    "image_url": {"url": f"data:image/png;base64,{image_data}"},
                },
            ],
            tool_call_id="1",
            name="random_image",
        )

        # Standard LangChain format, base64 data
        standard_format_message = ToolMessage(
            content=[
                {