Why I joined OctoML

Luis Ceze

Oct 27, 2021

Why I joined OctoML

In the Q&A below, Luis Ceze, CEO OctoML, chats with David Messina, OctoML’s new CMO.

Luis: What inspired you to want to be a part of the team here at OctoML?

David: The short answer: to work with an amazing team making a positive impact in a field, machine learning (ML)/AI, that’s going to be at the center of technology innovation for decades. I believe it will create a new wave of opportunities for society and businesses alike as long as there is a broad diversity of companies and voices actively involved in the development of the field.

The longer answer: It started with taking a nice long break between my last job, at Docker, and this one at OctoML. Throughout my career between every job, I’ve found a way to give myself time to reflect, to read and learn, to relax; and along the way to explore new avenues of technology. Originally, I was convinced that I wanted to work on developer technology again.

But by spending a lot of time with the devs/technologists in the communities that I’ve worked closely with, they helped me see that the next big transformation in software development will be the intelligent AI/ML building blocks that they are stitching into their apps and services. And that’s when I started looking for and asking for introductions to AI/ML companies.

Luis: You clearly have a passion for the developer community. What are the specific challenges OctoML solves for developers?

David: The first is the amazing fact that machine learning models are generally tethered to one hardware vendor/platform; a surprising revelation for someone who saw containers solve the software portability problem. And that OctoML’s founding story was tied to creating an open source project, Apache TVM, to address the challenges of ML portability and performance. OctoML’s use of Apache TVM ensures that ML models trained in the cloud can be deployed to edge and mobile devices, as close as possible to the end-user, to deliver the best experiences with AI-powered apps and services.

But the vision is bigger than just developers, data scientists and ML practitioners. OctoML wants to build “a world where AI is sustainable and accessible so it can be thoughtfully used to improve lives.” This resonated with me because with AI there are the same societal risks that we’ve seen with big social media giants, where decade-defining technology has remained in the hands of the few. Whereas our mission at OctoML is focused on continuously accelerating ML, which in turn can reduce its carbon footprint and make this technology more affordable to data scientists, ML researchers, developers everywhere.

Luis: What are the early signs you see that OctoML is on the right path?

David: There are numerous things that I see, but I am going to boil it down to one thing. We are committed to making exceedingly hard things easy for our customers. What has been eye-opening for me is that as an outsider I thought that the heavy-lifting in ML was only done by the data scientists who put an incredible amount of work into training and creating new state-of-the-art models.

For those in the know, that perception glosses over the fact that there are significant hurdles to overcome in the workflow of getting trained models deployed into production. Behind the scenes there are heroes on ML teams that are using “black belt” level skills to manually pull incredibly complex technical levers to get their models to performance and accuracy levels, where they can deliver a good user experience. Even with all the work they do, they are often making major trade offs in terms of speed-to-market and cost optimization. Compounding the challenge is the scarcity of engineers who have the knowledge to accelerate ML models; right now they are mostly employed by hardware vendors and cloud providers and not enterprises building their own proprietary models.

Luis: Much like at Docker, OctoML champions the open source community. Can you tell us about that?

David: The founders of OctoML created Apache TVM to foster a “diverse community of hardware vendors, compiler engineers, and ML researchers to work together to build a unified, programmable software stack; they enrich the entire ML technology ecosystem and make it accessible to the wider ML community.”

And on top of Apache TVM, the OctoML platform is focused on providing automated acceleration for any type of ML model, using any combination of acceleration engines, across a myriad of hardware choices. Offering customers choice, automation and performance means that they can quickly cost-optimize their models, get them to market faster--especially out to the edge and beyond--and deliver the best AI service possible.

What makes this an incredible opportunity for our growing engineering team is that the innovations we are focused on both in our platform and open source are centered on taking the complex drudge work out of users’ and customers' jobs, and we can celebrate their successes.

Luis: If you can recommend one book that you read during your time off, what would that be?

David: The Infinite Game by Simon Sinek. It’s a great read for anyone who loves company building. It focuses on the importance of aligning what you are doing to a “just cause” as a way to inspire both your employees and your customers. And closely tied to it, the importance of leadership in driving infinite thinking into an organization, where growing the opportunity for all-- even “worthy rivals”--leads to much greater outcomes than managing things in a finite fashion. What I’m saying doesn’t do it justice; read or listen to the book (Simon Sinek narrates it himself).

Accelerate Performance and Deployment Time