Sign up
Log in
Sign up
Log in

OctoML and AWS Better Together

OctoAI is a compute service to run, tune, and scale your generative AI, built on top of AWS. It allows developers to quickly and cost effectively take generative AI applications to production on AWS.

AWS Partner Network logo

Benefits of OctoAI, powered by AWS

OctoAI complements the AWS core infrastructure offerings to ensure models are run in a hardware configuration that is optimized for the model and for the application. OctoAI's model acceleration reduces latency and cost for popular foundation models including Stable Diffusion, Whisper, LLaMA and Falcon, as well as custom models built or trained by customers.

Speedometer Icon

Ease of Use

  • Ready to use deployment templates for popular OSS models

  • Customize OSS models

  • Easily integrate with app dev and model dev workflows

  • Auto-selection of hardware

Graph Icon

Efficiency

  • Fastest foundation models for generative AI made possible through our model acceleration technology

  • Accelerate and run your custom models

  • Flexibility to make price-performance tradeoffs

Globe Icon

Make Accessible

  • Customers may select and run accelerated OSS foundation models, fine tune models, upgrade to new models as they emerge, or bring their own custom models

  • No lock-in into the model or service

OctoAI powered by AWS

Performance Impact

OctoML Model Acceleration on AWS

All Posts
Oct 19, 2023
4 minutes
AI art creation platform NightCafe Studio delivers over 400K daily SDXL image creations for users, powered by OctoAI

Nightcafe Studio, a pioneer in AI art creation, recently worked with the OctoML team to launch a new personalized image generation application for its customers. Within a period of two weeks, the new application’s usage scaled 4x from its initial daily usage of 100K, to over 400K daily images, with no drop in latency or image creation success rates for end users.

Blog Author - Pete Sarabia
Pete Sarabia
Oct 30, 2023
9 minutes
Stable Diffusion at scale: Designing for millions of daily image generations

Running generative models in the cloud requires reliable infrastructure that can load and run these large models efficiently, and scale up and down quickly to handle bursts of traffic. Some OctoAI customers today generate over a million Stable Diffusion images a day. OctoAI is built to support this kind of scale and burstiness in a predictable manner.

Blog Author - Andrew Whitaker
Blog Author - Greg Flanagan
Blog Author - Sameer Farooqui
Andrew Whitaker & Greg Flanagan & Sameer Farooqui
Aug 30, 2023
5 minutes
Making the Llama 2 Herd Work for You on OctoAI

OpenAI deserves its flowers for creating the “iPhone opportunity” for the AI industry with ChatGPT. Thanks to the magnanimity of Meta, the incredible power of world-class LLMs will be accessible to everyone. Meta has catalyzed an infinite opportunity with its Android moment in releasing Llama 2. Sharing a high quality commercially viable open source Large Language Model (LLM) allows every company and developer to shine, not just one.

Blog Author - Jason Knight
Jason Knight

Start building with ease in minutes using OctoAI

Our mission is empowering developers to build AI applications that delight users by leveraging fast models running on the most efficient hardware. Sign up and start building in minutes.

Sign Up