Wrangling Data: How Nomadic Secured $8.4 Million for Autonomous Vehicle Insights

To Build Autonomous Machines, Models Need Models

Creating the future of autonomous machines requires a deep understanding that sometimes your model needs a model. Companies venturing into self-driving cars, robotic manipulation, or autonomous construction equipment face the daunting task of collecting vast amounts of video data for evaluation and training.

Organizing and cataloging this extensive video data is a human-intensive job, as individuals have to meticulously watch through hours of footage. Even with fast-forwarding, the process is not scalable. NomadicML, a startup founded by CEO Mustafa Bal and CTO Varun Krishnan, aims to address this challenge for customers with a significant portion of their fleet data stored in archives.

The complexity escalates when hunting for edge cases – those rare events that hold immense value but can confuse inexperienced physical AI models. Nomadic is tackling this issue with a platform that transforms footage into a structured, searchable dataset utilizing a range of vision language models. This advancement enables enhanced fleet monitoring, the creation of unique datasets for reinforcement learning, and quicker iteration.

Recently, the company announced an $8.4 million seed round with a post-money valuation of $50 million. TQ Ventures led the funding round, accompanied by Pear VC and Jeff Dean. This investment will facilitate the onboarding of more customers and the continuous refinement of Nomadic’s platform, which recently secured first prize at Nvidia GTC’s pitch contest.

Founders’ Journey and Technical Expertise

The two founders, who crossed paths during their Computer Science studies at Harvard, encountered recurring technical hurdles in their roles at companies like Lyft and Snowflake. Bal emphasized, “We provide individuals insights on their own footage, shaping the development of their AVs and robots. This targeted approach is what propels autonomous systems builders forward, not random data.”

One can envision the challenge of fine-tuning an AV’s comprehension regarding running a red light under police direction, or isolating instances of vehicles passing under a specific type of bridge. Nomadic’s platform enables the identification of such incidents for compliance purposes and their direct integration into training pipelines.

Leading companies like Zoox, Mitsubishi Electric, Natix Network, and Zendar are leveraging Nomadic’s platform to craft intelligent machines. Antonio Puglielli, Zendar’s VP of Engineering, highlighted how Nomadic’s tool accelerated their work compared to outsourcing, emphasizing the platform’s domain expertise as a distinguishing factor.

The Rise of Model-Based Auto-Annotation Tools in Physical AI

The emergence of model-based auto-annotation tools is reshaping physical AI workflows. Established data labeling firms like Scale, Kognic, and Encord are developing AI tools for this purpose, while Nvidia introduced the Alpamayo family of open-source models adaptable to tackle this challenge.

Varun argues that Nomadic’s tool transcends being merely a labeler; it functions as an “agentic reasoning system,” comprehending required actions and contextualizing them through multiple models. Backers anticipate Nomadic’s specialization in this infrastructure to be a competitive advantage.

Tanger, a partner at TQ Ventures, praised Nomadic’s talent, highlighting Krishnan’s international chess master accolade and the collective scientific publication track record of the company’s engineers.

Currently, Nomadic is focused on developing specific tools such as understanding lane changes from camera footage or pinpointing precise locations for a robot’s grippers in videos. The future challenge lies in developing similar tools for non-visual data like lidar sensor readings and integrating sensor data across diverse modes.

Bal emphasized the complexity of handling terabytes of video, analyzing them against numerous parameter models, and deriving accurate insights. The arduous task underscores the significance of Nomadic’s mission in the realm of autonomous systems.