Initial Considerations:

This library uses WebRTC to integrate with the RealtimeAPI. The connection and request for a session is already implemented inside this library (check the StartSession action) and it is transparent to the user.

Preparation:

1. Go to OpenAI and create an API Key.

2. Go to the Settings of this library and add the created key to the setting OpenAIKey.

3. You're ready to go.

Usage:

To use this library in an app, follow these steps:

1. Drag the RealtimeAgent block to the screen you want to have the agent.

2. Configure the agent and the respective tools using the input parameters of the block.

3. Begin the session and end it by using the respective StartSession and StopSession actions.

4. Handle the messages received and the tool calls by catching the events of the block.

5. Send text messages to the agent with the SendMessage action.

6. Reply with the outputs of a certain tool call using the action SubmitToolOutput.

OpenAI Documentation:

https://platform.openai.com/docs/guides/realtime

https://platform.openai.com/docs/api-reference/realtime