OctoAI is a compute service to run, tune (or customize), and scale your generative AI models. OctoAI lets you easily get started with ready to use model templates, or convert any container or Python code into a production-grade endpoint, within minutes. You can then easily build against the endpoint within your web or mobile apps.

The first step in getting OctoAI's launch promotion* pricing for base SDXL images is signing up for a free guided trial. Then after successful completion, committing to a minimum of 10 million images generated per year.

At sign up, you get $10 of free credit, which can be used until the end of the first month. You can enter your credit card at any time, and your account will be automatically charged to keep your credit replenished. This will be a minimum of $10 or a maximum amount that you set. We will auto-reload your account when the balance reaches 10% of your reload amount. Your account must have a positive balance for you to use the service.

OctoAI accelerates foundation models like Stable Diffusion 1.5 and SDXL, reducing latency and thus delivering a more engaging user experience in your applications for your users. Through the OctoAI Image Gen Solution, your inference calls benefit from the lower latencies and accelerated models even when you choose to create images with your selected imported or bespoke fine tuning assets.

At launch, OctoAI will support the NVIDIA T4, the NVIDIA A10G, and the NVIDIA A100. Additional GPU targets will be added in future.

OctoML is SOC 2 Type II certified. Keeping our customers’ data private and secure is a top priority, and we have internal systems to ensure appropriate handling of customer data. The SOC 2 Type II certification provides independent validation of these processes and safeguards. We do not persist the inputs, outputs, nor intermediate computations of your inferences, except for runtime logs that you choose to expose in your container. For encryption in transit, we ensure that all connections from customer to the OctoAI compute service require TLS, without you having to manage TLS certifications yourself. We also use encryption at rest for any data that we write to disk.

When a new replica is started, it takes a certain amount of time for the compute infrastructure to be provisioned, data to be loaded, and the service to be ready for an inference query. This is known as a cold start. You are not billed for the cold start time.

When an endpoint does not receive any inference query for a preset timeout period, it will terminate running replicas until it reaches the count set for minimum replicas. If the minimum replicas is set to 0, all running replicas will be terminated and you will not be billed when the endpoint is not in use. You can set the timeout, minimum replicas, and maximum replicas for your endpoints to meet your application architecture and requirements.

You can choose to keep your endpoints public or private. Private endpoints can be accessed by using the API token(s) generated within OctoAI.