langchain Architecture Docs
Home / Function/ _stream() — langchain Function Reference

_stream() — langchain Function Reference

Architecture documentation for the _stream() function in huggingface.py from the langchain codebase.

Function python LangChainCore LanguageModelBase calls 5 called by 1

Entity Profile

Dependency Diagram

graph TD
  ced9b52f_7bf4_4dc3_bc59_a48a9563d9bc["_stream()"]
  8cf0d6c0_abf8_3ee2_fd00_8bfc8c02058a["ChatHuggingFace"]
  ced9b52f_7bf4_4dc3_bc59_a48a9563d9bc -->|defined in| 8cf0d6c0_abf8_3ee2_fd00_8bfc8c02058a
  0e798408_eb45_2a4b_1ddb_579fdb006f07["_generate()"]
  0e798408_eb45_2a4b_1ddb_579fdb006f07 -->|calls| ced9b52f_7bf4_4dc3_bc59_a48a9563d9bc
  9722f3bb_f150_918c_aa80_b6c7608bdffa["_should_stream_usage()"]
  ced9b52f_7bf4_4dc3_bc59_a48a9563d9bc -->|calls| 9722f3bb_f150_918c_aa80_b6c7608bdffa
  4c7378b7_670a_b9c9_e573_d0052d7e8308["_create_message_dicts()"]
  ced9b52f_7bf4_4dc3_bc59_a48a9563d9bc -->|calls| 4c7378b7_670a_b9c9_e573_d0052d7e8308
  e77e373b_eb20_e566_3771_32082ca0dcc2["_to_chat_prompt()"]
  ced9b52f_7bf4_4dc3_bc59_a48a9563d9bc -->|calls| e77e373b_eb20_e566_3771_32082ca0dcc2
  2b662de8_c3da_0d48_1bc1_b88b1d6c6022["_is_huggingface_endpoint()"]
  ced9b52f_7bf4_4dc3_bc59_a48a9563d9bc -->|calls| 2b662de8_c3da_0d48_1bc1_b88b1d6c6022
  5fa8863f_86a4_f68d_871c_bd4a2d0d8448["_convert_chunk_to_message_chunk()"]
  ced9b52f_7bf4_4dc3_bc59_a48a9563d9bc -->|calls| 5fa8863f_86a4_f68d_871c_bd4a2d0d8448
  style ced9b52f_7bf4_4dc3_bc59_a48a9563d9bc fill:#6366f1,stroke:#818cf8,color:#fff

Relationship Graph

Source Code

libs/partners/huggingface/langchain_huggingface/chat_models/huggingface.py lines 822–889

    def _stream(
        self,
        messages: list[BaseMessage],
        stop: list[str] | None = None,
        run_manager: CallbackManagerForLLMRun | None = None,
        *,
        stream_usage: bool | None = None,
        **kwargs: Any,
    ) -> Iterator[ChatGenerationChunk]:
        if _is_huggingface_endpoint(self.llm):
            stream_usage = self._should_stream_usage(
                stream_usage=stream_usage, **kwargs
            )
            if stream_usage:
                kwargs["stream_options"] = {"include_usage": stream_usage}
            message_dicts, params = self._create_message_dicts(messages, stop)
            params = {**params, **kwargs, "stream": True}

            default_chunk_class: type[BaseMessageChunk] = AIMessageChunk
            for chunk in self.llm.client.chat_completion(
                messages=message_dicts, **params
            ):
                if len(chunk["choices"]) == 0:
                    if usage := chunk.get("usage"):
                        usage_msg = AIMessageChunk(
                            content="",
                            additional_kwargs={},
                            response_metadata={},
                            usage_metadata={
                                "input_tokens": usage.get("prompt_tokens", 0),
                                "output_tokens": usage.get("completion_tokens", 0),
                                "total_tokens": usage.get("total_tokens", 0),
                            },
                        )
                        yield ChatGenerationChunk(message=usage_msg)
                    continue

                choice = chunk["choices"][0]
                message_chunk = _convert_chunk_to_message_chunk(
                    chunk, default_chunk_class
                )
                generation_info = {}
                if finish_reason := choice.get("finish_reason"):
                    generation_info["finish_reason"] = finish_reason
                    generation_info["model_name"] = self.model_id
                logprobs = choice.get("logprobs")
                if logprobs:
                    generation_info["logprobs"] = logprobs
                default_chunk_class = message_chunk.__class__
                generation_chunk = ChatGenerationChunk(
                    message=message_chunk, generation_info=generation_info or None
                )
                if run_manager:
                    run_manager.on_llm_new_token(
                        generation_chunk.text, chunk=generation_chunk, logprobs=logprobs
                    )
                yield generation_chunk
        else:
            llm_input = self._to_chat_prompt(messages)
            stream_iter = self.llm._stream(
                llm_input, stop=stop, run_manager=run_manager, **kwargs
            )
            for chunk in stream_iter:  # chunk is a GenerationChunk
                chat_chunk = ChatGenerationChunk(
                    message=AIMessageChunk(content=chunk.text),
                    generation_info=chunk.generation_info,
                )
                yield chat_chunk

Domain

Subdomains

Defined In

Calls

Called By

Source

Frequently Asked Questions

What does _stream() do?
_stream() is a function in the langchain codebase, defined in libs/partners/huggingface/langchain_huggingface/chat_models/huggingface.py.
Where is _stream() defined?
_stream() is defined in libs/partners/huggingface/langchain_huggingface/chat_models/huggingface.py at line 822.
What does _stream() call?
_stream() calls 5 function(s): _convert_chunk_to_message_chunk, _create_message_dicts, _is_huggingface_endpoint, _should_stream_usage, _to_chat_prompt.
What calls _stream()?
_stream() is called by 1 function(s): _generate.

Analyze Your Own Codebase

Get architecture documentation, dependency graphs, and domain analysis for your codebase in minutes.

Try Supermodel Free