When you call an LLM API without streaming, the API waits until the entire response is generated, then sends it back in one shot. For short answers, that is acceptable. For longer responses, the user stares at a blank screen while the model is already producing tokens. Streaming sends partial output to the client as it is generated, so the response appears incrementally.

The second problem is state. Chat-style LLM APIs are usually stateless unless you explicitly use a provider's conversation-state feature. With the OpenAI-compatible Chat Completions interface used in this course, each request must include the context the model needs. If you want a back-and-forth conversation, your application has to manage the message history.

In this chapter, you will learn:

The chapter ends with a streaming chatbot that maintains conversation history and handles context-window limits deliberately.