> For the complete documentation index, see [llms.txt](https://docs.gptboost.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.gptboost.io/first-steps/streaming.md).

# Streaming

**Why Choose Streaming?**

Streaming is a great feature meant to enable real-time, dynamic interactions between developers and the OpenAI API. It provides an efficient way to receive responses from the OpenAI API incrementally, as the data becomes available. This real-time interaction offers enhanced flexibility and responsiveness, making it a valuable addition to your toolkit.

Developers often opt for streaming in scenarios where real-time responses are critical and standard request-response mechanisms may not suffice. 

**How to stream the response?**

Simply add `stream=True` to your request to receive the response in chunks as they're generated, instead of waiting for the complete response to be assembled.

{% tabs %}
{% tab title="Python" %}

```python
# This code is for v1+ of the openai package https://pypi.org/project/openai/
from openai import OpenAI 

client = OpenAI(
    api_key= os.getenv("OPENAI_API_KEY"),
    base_url = "https://turbo.gptboost.io/v1"
)

# Make a request to OpenAI API
stream = client.chat.completions.create(
    model="gpt-4",
    messages=[
        {"role": "user", "content": "Tell me a story about a happy developer."}
    ],
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "")
```

{% endtab %}

{% tab title="cURL" %}

```powershell
curl --request 'https://turbo.gptboost.io/v1/chat/completions' \
--header 'Authorization: Bearer $OPENAI_API_KEY' \
--header 'Content-Type: application/json' \
--data '{
    "model": "gpt-3.5-turbo",
    "messages": [
        {
            "role": "user", 
            "content": "Tell me an interesting fact about koalas!"
        }
    ], 
    "stream": true
}'
```

{% endtab %}

{% tab title="NodeJS" %}

```javascript
// This code is for v4+ of the openai package: npmjs.com/package/openai
import OpenAI from 'openai';

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
  baseURL: "https://turbo.gptboost.io/v1"
})

async function generateStream(prompt) {
  const stream = await openai.chat.completions.create({
    model: "gpt-3.5-turbo",
    messages: [{"role": "user", "content": prompt}],
    stream: true,
  });
  for await (const part of stream) {
    console.log(part.choices[0].delta.content);
  }
}

generateStream("Tell me one curious fact")

```

{% endtab %}

{% tab title="Vercel" %}

```javascript
// This code is for v4 of the openai package: npmjs.com/package/openai
import OpenAI from 'openai';
import { OpenAIStream, StreamingTextResponse } from 'ai';
 
// Can be 'nodejs', but Vercel recommends using 'edge'
export const runtime = 'edge';
 
const openai = new OpenAI({
  apiKey: process.env.OPEN_API_KEY,
  baseURL: "https://turbo.gptboost.io/v1",
});

// This method must be named GET
export async function GET() {

  // Make a request to OpenAI's API
  const response = await openai.chat.completions.create({
    model: 'gpt-3.5-turbo',
    stream: true,
    messages: [{ role: 'user', content: 'Say this is a test.' }],
  });
  
  //"Save the streaming results in a variable to prevent the stream from being exhausted."
  const [logStream, responseStream] = response.tee ? response.tee() : [response, response];

    // Convert the response into a friendly text-stream
    const stream = OpenAIStream(response);
    // Respond with the stream and headers
    return new StreamingTextResponse(stream);
}

```

{% endtab %}
{% endtabs %}

**What are some use cases?**

Here are a few use cases where streaming shines:

* **Conversational Interfaces:** Streaming is perfect for building chatbots and conversational interfaces that require instant responses to user input. It allows for a more natural, interactive conversation flow.
* **Live Transcriptions:** In applications like transcription services or live captioning, streaming provides immediate results as audio or speech input is processed. This is particularly useful in live event scenarios.
* **Interactive Gaming:** For interactive games that demand dynamic and real-time AI-generated content, streaming ensures seamless integration of AI-driven elements.
* **On-the-Fly Content Generation:** In applications where content is generated on the fly, such as dynamic content creation for websites or marketing materials, streaming ensures rapid content delivery.
* **Simultaneous Data Processing:** For tasks requiring parallel processing of multiple inputs, like language translation or sentiment analysis for a stream of social media posts, streaming enhances efficiency.