_stream() — langchain Function Reference

Architecture documentation for the _stream() function in huggingface_endpoint.py from the langchain codebase.

Function python LangChainCore LanguageModelBase calls 1 called by 1

Entity Profile

LangChainCore→ LanguageModelBase→ _stream() — langchain Function Reference

Dependency Diagram

graph TD
  5b897346_9859_18d1_14ba_55cddb3d7da7["_stream()"]
  e6b830e0_e492_47f6_6d0a_65d69c36da10["HuggingFaceEndpoint"]
  5b897346_9859_18d1_14ba_55cddb3d7da7 -->|defined in| e6b830e0_e492_47f6_6d0a_65d69c36da10
  477387e3_0b93_4ca4_57d9_1d292e64d8f9["_call()"]
  477387e3_0b93_4ca4_57d9_1d292e64d8f9 -->|calls| 5b897346_9859_18d1_14ba_55cddb3d7da7
  d3d02905_01f3_3fb8_f596_5694650af72c["_invocation_params()"]
  5b897346_9859_18d1_14ba_55cddb3d7da7 -->|calls| d3d02905_01f3_3fb8_f596_5694650af72c
  style 5b897346_9859_18d1_14ba_55cddb3d7da7 fill:#6366f1,stroke:#818cf8,color:#fff

Relationship Graph

Source Code

libs/partners/huggingface/langchain_huggingface/llms/huggingface_endpoint.py lines 384–419

    def _stream(
        self,
        prompt: str,
        stop: list[str] | None = None,
        run_manager: CallbackManagerForLLMRun | None = None,
        **kwargs: Any,
    ) -> Iterator[GenerationChunk]:
        invocation_params = self._invocation_params(stop, **kwargs)

        for response in self.client.text_generation(
            prompt, **invocation_params, stream=True
        ):
            # identify stop sequence in generated text, if any
            stop_seq_found: str | None = None
            for stop_seq in invocation_params["stop"]:
                if stop_seq in response:
                    stop_seq_found = stop_seq

            # identify text to yield
            text: str | None = None
            if stop_seq_found:
                text = response[: response.index(stop_seq_found)]
            else:
                text = response

            # yield text, if any
            if text:
                chunk = GenerationChunk(text=text)

                if run_manager:
                    run_manager.on_llm_new_token(chunk.text)
                yield chunk

            # break if stop sequence found
            if stop_seq_found:
                break