Qualcomm AI Inference Suite: Getting Started is Easy

This blog post was originally published at Qualcomm’s website. It is reprinted here with the permission of Qualcomm.

Once you have a key, it is simply a matter of choosing how to connect to the inference endpoint.  If you are most comfortable with Python, an SDK is provided along with documentation so that you can connect using familiar Python coding conventions. Many other programming languages and tools (20+) are supported if you prefer to use OpenAI compliant APIs.

Sample scenario

Let’s create a sample scenario to showcase use.  You want AI to tell you if a given customer review is positive, negative, or neutral. Once you have that data, you could later analyze which products or services are liked and which are not.

Python SDK example

Going step by step, we’ll build up our sample code.

Import functions you’ll need from the Imagine SDK:

from imagine import ChatMessage, ImagineClient

Set your API key and endpoint in env variables or in code:

myendpoint = "https://aisuite.cirrascale.com/apis/v2"
myapikey = "useyourownkeyhere"

Create your client for the API using the endpoint and key:

client = ImagineClient(endpoint=myendpoint, api_key=myapikey)

Set up your request, choosing which LLM to use:

mymodel = "Llama-3.1-8B"

feedback = "Feedback: ' We loved this hotel, we will be coming back.'"

mycontent = "You are an expert at sentiment analysis. Please evaluate the following customer feedback and answer only with positive, negative, or neutral. Only answer positive, negative, or neutral in lowercase. " + feedback

Call the chat function with your request:

mymessage = ChatMessage(role="user", content=mycontent)

chat_response = client.chat(
    messages=[mymessage],
    model=mymodel,
)

Print out the result:

print(chat_response.first_content)

Try changing the text of the feedback variable to different sets of text about any product and give it a try.

Try it out

As you can see, the process of using inference on a scalable platform like the Qualcomm AI Inference Suite is as easy as using any other simple API, but it provides access to very fast results using even larger models.  Even though this is just a small example, it could be integrated into a microservice to provide lots of customer sentiment analysis as part of a production data pipeline.

Like what you are seeing? Connect with fellow developers, get the latest news and prompt technical support by joining our Developer Discord.

Ray Stephenson
Developer Relations Lead, Cloud, Qualcomm Technologies