Cloud inference wrapper
The following is intended for informational purposes only, and may not be incorporated into any contract. No purchasing decisions should be made based on the following materials. Unity is not committing to deliver any functionality, features or code. The development, timing and release of all products, functionality and features are at the sole discretion of Unity, and are subject to change.
If you have accepted to use functional cookies and logged in using your Unity ID at the top right of the page, then sharing feedback is as simple as clicking a card below, selecting a topic’s importance, adding your point of view, and submitting. If you prefer not to accept functional cookies or log in, you will be prompted to enter an email address and validate it, so we know how to reach out when the topic evolves. For more information read the Feedback and privacy terms.
What? This is an HTTP wrapper sample code for server API calls when it may not make sense to use Sentis local inference. This would be a convenience sample in our documentation that keeps the tensors (model inputs and outputs) very similar to what they would look like with Sentis local inference. Thus, if you want to swap between local and cloud inference, you can change the smallest amount of Unity C# code.
Why? There are cases where it’s preferable to host your AI model on a cloud server, such as with large language models (LLMs) that are very large on disk. For these cases, you can use this sample cloud wrapper code for API calls so that you don’t have to write your own.
*Note: This sample code would be adaptable to most hosted AI model inference infrastructure. Let us know if you would prefer a Unity-specific cloud-hosted inference solution. Such a solution would have game-specific features that today’s generic cloud vendors don’t provide, such as direct integration into the Unity Editor and game-behavior-based budgeting for inference costs.