test_image_inputs() — langchain Function Reference

Architecture documentation for the test_image_inputs() function in chat_models.py from the langchain codebase.

Function python LangChainCore LanguageModelBase

Entity Profile

LangChainCore→ LanguageModelBase→ test_image_inputs() — langchain Function Reference

Dependency Diagram

graph TD
  c0d5c982_0187_8793_db7e_8ce97aac7b51["test_image_inputs()"]
  971e928f_9c9b_ce7a_b93d_e762f2f5aa54["ChatModelIntegrationTests"]
  c0d5c982_0187_8793_db7e_8ce97aac7b51 -->|defined in| 971e928f_9c9b_ce7a_b93d_e762f2f5aa54
  style c0d5c982_0187_8793_db7e_8ce97aac7b51 fill:#6366f1,stroke:#818cf8,color:#fff

Relationship Graph

Source Code

libs/standard-tests/langchain_tests/integration_tests/chat_models.py lines 2716–2825

    def test_image_inputs(self, model: BaseChatModel) -> None:
        """Test that the model can process image inputs.

        This test should be skipped (see configuration below) if the model does not
        support image inputs. These will take the shape of the LangChain
        `ImageContentBlock`:

        ```python
        {
            "type": "image",
            "base64": "<base64 image data>",
            "mime_type": "image/jpeg",  # or appropriate MIME type
        }
        ```

        For backward-compatibility, we must also support OpenAI chat completions
        image content blocks containing base64-encoded images:

        ```python
        [
            {"type": "text", "text": "describe the weather in this image"},
            {
                "type": "image_url",
                "image_url": {"url": f"data:image/jpeg;base64,{image_data}"},
            },
        ]
        ```

        See docs for [Multimodality](https://docs.langchain.com/oss/python/langchain/models#multimodal).

        If the property `supports_image_urls` is set to `True`, the test will also
        check that we can process content blocks of the form:

        ```python
        {
            "type": "image",
            "url": "<url>",
        }
        ```

        ??? note "Configuration"

            To disable this test, set `supports_image_inputs` to `False` in your
            test class:

            ```python
            class TestMyChatModelIntegration(ChatModelIntegrationTests):
                @property
                def supports_image_inputs(self) -> bool:
                    return False

                # Can also explicitly disable testing image URLs:
                @property
                def supports_image_urls(self) -> bool:
                    return False
            ```

        ??? question "Troubleshooting"

            If this test fails, check that the model can correctly handle messages
            with image content blocks, including base64-encoded images. Otherwise, set
            the `supports_image_inputs` property to `False`.

        """
        if not self.supports_image_inputs:
            pytest.skip("Model does not support image message.")

        image_url = "https://raw.githubusercontent.com/langchain-ai/docs/4d11d08b6b0e210bd456943f7a22febbd168b543/src/images/agentic-rag-output.png"
        image_data = base64.b64encode(httpx.get(image_url).content).decode("utf-8")

        # OpenAI CC format, base64 data
        message = HumanMessage(
            content=[
                {"type": "text", "text": "Give a concise description of this image."},
                {
                    "type": "image_url",
                    "image_url": {"url": f"data:image/png;base64,{image_data}"},
                },
            ],
        )
        _ = model.invoke([message])