Azure Integration

Seamless Integration Azure OpenAI deployment

You can use GPTBoost with your Azure-OpenAI deployment, and it requires no additional code changes beyond replacing the azure_endpoint with the GPTBoost URL. Similar to communicating with OpenAI API, GPTBoost acts as a proxy between the client and your Azure OpenAI deployment.

To start, make sure you have an active Azure Subscription and that you've activated Azure OpenAI for it. If you haven't done this yet, check on how to set up Azure OpenAI Services.

In case, you already have your Azure deployments ready, you can proceed to add an Azure Key to GPTBoost.

Set up Azure OpenAI Services

Remember, you'll need Python version 3.7.1 or later to meet the minimum requirements.

1. Start by setting up your Azure subscription - there's a free tier available.

2. Access to Azure OpenAI Services is granted through an application. Apply for access by filling out the form at https://aka.ms/oai/access. They'll ask you a few questions, and then you just need to wait for Microsoft to approve your request.

3. Deploy the OpenAI model following the official Azure documentation.

Add your Azure Key to GPTBoost

Setting up the monitoring of your Azure OpenAI model statistics in GPTBoost is pretty straightforward.

1. Head over to your GPTBoost account -> OpenAI API Keys.

2. Next, select Add Key.

3. For Provider choose Azure OpenAI from the drop-down menu.

1. Fill in your credentials. You'll need the Azure OpenAI API Key, the API version and the Azure endpoint. The Endpoint and Keys can be found in the Resource Management section of your Azure portal. More info on these credentials can be found here.

Adding an Azure OpenAi Key in GPTBoost

Code Examples

Once your Azure OpenAI deployment is set and the Azure Key is authorized in GTPBoost, you can start monitoring all requests by replacing the azure_endpoint with the GPTBoost URL.

Python

# This example is for v1+ of the openai: https://pypi.org/project/openai/
from openai import AzureOpenAI

client = AzureOpenAI(
    # Set GPTBoost API base URL for azure_endpoint
    azure_endpoint = "https://turbo.gptboost.io/v1",
    api_key = "AZURE_OPENAI_KEY",
    api_version = "2023-07-01-preview",
)

message = [
    {
        "role":"user",
        "content":'Write a tagline for an ice cream shop. '
    }
]

deployment_name="gpt-35-turbo-16k"

response = client.chat.completions.create(
    model=deployment_name, 
    messages=message
)

print(response.choices[0].message.content)

NodeJS

import { OpenAIClient, AzureKeyCredential } from "@azure/openai" ;

async function main(){

    const client = new OpenAIClient(
        // Set GPTBoost API base URL for azure_endpoint
        "https://turbo.gptboost.io/v1",
        new AzureKeyCredential("AZURE_OPENAI_KEY"),
        "2023-07-01-preview" //Azure API Version
        );
    
    const messages = [
        { role: "user", content: "Say 'Hello, World'" },
    ];

    
    const { id, created, choices, usage } = await client.getChatCompletions("gpt-35-turbo-16k", messages)
    
    for (const choice of choices) {
        console.log(choice.message.content);
    }
}

main().catch((err) => {
    console.error("The sample encountered an error:", err);
});