OpenAI.ImageRecognition

Class Declaration

 Protected Class ImageRecognition
 Inherits OpenAI.ChatCompletion

Remarks

This class represents an API image interpretation response. Refer to the OpenAI documentation on the /v1/chat/completions endpoint with the gpt-4-vision-preview model for further details.

To generate the AI assistant's first response, pass one or more images and a text prompt to the ImageRecognition.Create shared method. This will return an instance of ImageRecognition containing the response.

Chat conversations are stored as instances of the ChatCompletionData class. You can review and modify the conversation for any given ImageRecognition by referring to its ChatLog As OpenAI.ChatCompletionData property.

A chat message may be attributed to one of three entities: "user", "assistant", or "system". However, for image recognition chat sessions you will usually only use "user".

  OpenAI.APIKey = "YOUR API KEY"
  Dim url As String = "https://upload.wikimedia.org/wikipedia/commons/9/99/Aerial_view_of_the_White_House.jpg"
  Dim response As OpenAI.ImageRecognition = OpenAI.ImageRecognition.Create("What is this a photo of?", url)
  Dim answer As String = response.GetResult() ' This is an aerial photo of the the White House in Washington, DC.

Once you have created the initial response, you can continue the conversation in context by calling the GenerateNext() method.

  OpenAI.APIKey = "YOUR API KEY"
  Dim url1 As String = "https://upload.wikimedia.org/wikipedia/commons/9/99/Aerial_view_of_the_White_House.jpg"
  Dim url2 As String = "https://upload.wikimedia.org/wikipedia/commons/b/bd/Taj_Mahal%2C_Agra%2C_India_edit3.jpg"
  
  Dim response As OpenAI.ImageRecognition = OpenAI.ImageRecognition.Create("What is this a photo of?", url1)
  Dim answer As String = response.GetResult() ' This is an aerial photo of the the White House in Washington, DC.
  
  response = response.GenerateNext("user", "What direction is the camera facing?")
  answer = response.GetResult() ' The camera is facing northward.
  
  response = response.GenerateNext("user",  "Is this a picture of the same building?", url2)
  answer = response.GetResult() ' No, this is a photo of the Taj Mahal located in Agra, India.

  ' etc.

You can include more than one image in a single request by using an array:

  OpenAI.APIKey = "YOUR API KEY"
  Dim urls() As String
  urls.Append("https://upload.wikimedia.org/wikipedia/commons/9/99/Aerial_view_of_the_White_House.jpg")
  urls.Append("https://upload.wikimedia.org/wikipedia/commons/b/bd/Taj_Mahal%2C_Agra%2C_India_edit3.jpg")
  Dim response As OpenAI.ImageRecognition = OpenAI.ImageRecognition.Create("Are these photos of the same building?", urls)
  Dim answer As String = response.GetResult() ' No, the first image is of the White House but the second image is of the Taj Mahal.

Image formats

You may use either Xojo Picture objects or URLs to pictures on the Internet. Using URLs will improve performance since the picture data doesn't have to be encoded directly into the request.