Why I joined OctoML

In the Q&A below, Luis Ceze, CEO OctoML, chats with David Messina, OctoML’s new CMO.

The longer answer: It started with taking a nice long break between my last job, at Docker, and this one at OctoML. Throughout my career between every job, I’ve found a way to give myself time to reflect, to read and learn, to relax; and along the way to explore new avenues of technology. Originally, I was convinced that I wanted to work on developer technology again.

But by spending a lot of time with the devs/technologists in the communities that I’ve worked closely with, they helped me see that the next big transformation in software development will be the intelligent AI/ML building blocks that they are stitching into their apps and services. And that’s when I started looking for and asking for introductions to AI/ML companies.

But the vision is bigger than just developers, data scientists and ML practitioners. OctoML wants to build “a world where AI is sustainable and accessible so it can be thoughtfully used to improve lives.” This resonated with me because with AI there are the same societal risks that we’ve seen with big social media giants, where decade-defining technology has remained in the hands of the few. Whereas our mission at OctoML is focused on continuously accelerating ML, which in turn can reduce its carbon footprint and make this technology more affordable to data scientists, ML researchers, developers everywhere.

For those in the know, that perception glosses over the fact that there are significant hurdles to overcome in the workflow of getting trained models deployed into production. Behind the scenes there are heroes on ML teams that are using “black belt” level skills to manually pull incredibly complex technical levers to get their models to performance and accuracy levels, where they can deliver a good user experience. Even with all the work they do, they are often making major trade offs in terms of speed-to-market and cost optimization. Compounding the challenge is the scarcity of engineers who have the knowledge to accelerate ML models; right now they are mostly employed by hardware vendors and cloud providers and not enterprises building their own proprietary models.

And on top of Apache TVM, the OctoML platform is focused on providing automated acceleration for any type of ML model, using any combination of acceleration engines, across a myriad of hardware choices. Offering customers choice, automation and performance means that they can quickly cost-optimize their models, get them to market faster--especially out to the edge and beyond--and deliver the best AI service possible.

What makes this an incredible opportunity for our growing engineering team is that the innovations we are focused on both in our platform and open source are centered on taking the complex drudge work out of users’ and customers' jobs, and we can celebrate their successes.