Latitude Games lowers costs by 5x and unlocks new game experiences with Mixtral on OctoAI

Latitude Games is an award winning game developer and the creator of AI Dungeon, a role playing game that delivers engaging large language model (LLM) powered interactive experiences to players. AI Dungeon brings the momentum and excitement of language model progress to gamers, and key to this is its rapid adoption of impactful new models - from the earliest GPT models, to AI21’s Jurassic-1 and Jurassic-2 models, to GPT-3.5 Turbo and GPT 4. With the launch of Mixtral from Mistral AI, the team received overwhelming interest from gamers to have Mixtral added to AI Dungeon.

Working with OctoAI, Latitude was able to evaluate the new Mixtral model within days of its release, and add it to AI Dungeon just 10 days after Mixtral’s initial announcement. Today, Mixtral on OctoAI is the default model for the hundreds of thousands of monthly active users on the platform and serves billions of tokens for AI Dungeon every day. The switch to Mixtral on OctoAI has also reduced Latitude’s inference costs by over 5x, enabling them to accelerate planned product enhancements and add new AI-powered experiences for players.

Read on to learn more about Latitude Games and their adoption of Mixtral on OctoAI. You can get started with Mixtral on OctoAI on the OctoAI Text Gen Solution today.

AI Dungeon brings the excitement of LLM progress to AI gaming

AI Dungeon delivers an interactive role playing game (RPG) experience to players, building on the power of leading-edge LLMs. Taking inspiration from games like Dungeons and Dragons, AI Dungeon is built to maximize the flexibility and creative freedom possible in the game environment.

While many AI-powered games attempt to abstract away the specifics of the generative AI foundation models and technology under the hood, AI Dungeon is transparent about these. Players get to choose the models that power their gaming experience. AI Dungeon has also been proactive in adding new models to the platform and to communicating these additions and enhancements to players. Since the games initial introduction in 2019, AI Dungeon has iteratively added and evolved its list of story generator models across several generations of open and closed-source GPT models from OpenAI and AI21. This transparency and speed have effectively acted as a bridge connecting end players to the excitement and progress in the world of language model evolution.

Prior to late 2023, AI Dungeon was primarily built on the early GPT models followed by proprietary models from OpenAI and AI21. Open source models had been evolving, but none had started approaching the quality and performance of GPT. This changed with Mixtral, which quickly established itself as a tipping point in the evolution of open source models. With the quality and flexibility demonstrated by Mixtral and the broader buzz among early adopters, there was an overwhelming level of interest in Mixtral from AI Dungeon players.

Early surveys by the Latitude Games team validated the strong demand for Mixtral, and the team was eager to quickly get the model into AI Dungeon and to players. Equally critical was to make sure it was a production grade service, ready for traffic and scaling, with predictable latency and availability.

Mixtral on OctoAI: Production grade Mixtral endpoint for AI Dungeon

Working with the OctoAI team, Latitude Games was able to validate that the reliability and latency possible on OctoAI was sufficient for their needs. The competitive pricing was an additional bonus, with initial estimates indicating that it could have an order of magnitude improvement to inference costs. With this assurance, the team introduced Mixtral to AI Dungeon as an early access offer for players. The adoption and response validated the demand, and user feedback continued to overwhelmingly highlight preference for Mixtral. Surveys and forum discussions indicated that over 75% of users preferred the Mixtral model as the promotional premium model available for free actions. With this validation and the demonstrated performance of the endpoint through the early access, the team moved Mixtral to general availability and out of early access. With this, Mixtral is also now the default model that new players start with and the promotional premium model available on the platform.

Higher intelligence to cost ratio unlocks new products and experiences

Significantly higher intelligence to cost ratios don’t just make existing things cheaper and better, they unlock entirely new applications and products that wouldn’t have been possible otherwise. Latitude has been at the cutting edge of AI and games since the first GPT-2 model was released, and in the process, constantly pushing the boundaries of how to build the AI games people dream of playing. The almost order of magnitude improvement in the intelligence to cost ratio unlocks a new type of game experience that the team currently have in development. The new capabilities use significantly more advanced AI systems to deliver more powerful experiences to players, and would not be possible without this improvement in costs.

As better open source models get deployed at lower prices, there will be more innovative products built and launched that wouldn’t have been possible with previous generations of AI technology. This ongoing shift in the intelligence to cost ratio will be key to unlocking future waves of products and capabilities powered by AI.

Timely inclusion of Mixtral, new product capabilities, and a 5x reduction in inference costs

Latitude Games was able to launch Mixtral into AI Dungeon within days of the initial announcement, delivering a new more flexible experience and tapping into the booming player demand for Mixtral.

-Selected Reddit user comments on Mixtral addition to AI Dungeon

In addition to the new experience and speed of inclusion, Mixtral on OctoAI’s lower costs have resulted in a 5x reduction in inference costs compared to the ChatGPT (GPT 4) model. This has also unlocked the addition of new and more demanding AI-powered experience on the platform. Today, Mixtral on OctoAI servers over a billion tokens every day to hundreds of thousands of players on AI Dungeon.

The LLM evolution is far from over, and Latitude Games and OctoAI are expanding our collaboration to evaluate new models and fine tunes. These efforts are crucial for the continued iteration of models and options on AI Dungeon, and for the platform to remain on the leading-edge of language model capabilities. We’ll share more about these models as they are ready!

Try the OctoAI Text Gen Solution today

You can start evaluating LLMs and building on OctoAI Text Gen Solution with a free trial today. If you’d like to engage with the broader OctoAI community and teams, please join our OctoAI Discord.

For customers currently using a closed source LLM like GPT-3.5 Turbo, GPT-4 or GPT-4 Turbo from OpenAI, OctoAI has a new promotion to help accelerate adoption of open source LLMs. You can read more about this in the OctoAI Model Remix Program introduction.