Imagine Kevin Durant swapping his basketball sneakers for tennis shoes and clinching a Grand Slam against Rafael Nadal. It sounds improbable, yet that’s the metaphor some market observers are using to capture the meteoric rise of Luma AI, a Silicon Valley startup now taking on OpenAI in the competitive world of video generation.
Breaking into the video generation space with a model that rivals OpenAI’s Sora is no small feat for any startup, let alone one that only entered this domain around a year ago. Founded in 2021, Luma AI previously focused on 3D content creation, a niche that kept its small team of around ten people under the radar.
Barkley Dai, who leads product development and growth at Luma AI, said that the shift to video generation in December 2023 spurred the company’s expansion, bringing on 40 artificial intelligence experts. “Talent is the most important asset for building video models,” he said.
Dai also credits Luma AI’s success to a combination of technical ability, timing, and execution.
To compete with giants like OpenAI and Google, Luma AI made substantial algorithmic and infrastructure optimizations. Its Dream Machine AI builds on the same diffusion transformer (DiT) architecture used by Sora, but with proprietary improvements that are said to ensure high-quality output while reducing training and inference costs.
Launched on June 13 this year, the public release of Dream Machine AI filled a gap in the video generation market. At the time, it was one of few available options aside from Kuaishou’s Kling AI. Luma AI’s freemium strategy further fueled adoption.
Within four days of its launch, Dream Machine AI had attracted over one million users—all without a marketing budget. Dai said that this was achieved purely through word-of-mouth and endorsements from key opinion leaders (KOLs).
However, the company’s challenge now lies in sustaining user interest beyond the initial excitement. On November 26, nearly six months after Dream Machine AI’s launch, the company unveiled its evolution into a full-fledged creative platform, accessible on iOS and the web. As part of this update, Luma AI also introduced Photon, its first proprietary image generation model, signaling an expansion of its creative toolkit.
Unlike tools from Midjourney or Adobe, Dream Machine AI was crafted with beginners in mind. Jiacheng Yang, a product designer at Luma AI, explained that the platform’s mission is to create an AI-powered visual tool that anyone—regardless of their familiarity with AI or design—can use effortlessly.
The platform’s functionality is built around five core features:
- Generating and editing images through natural language conversation.
- Offering AI-suggested creative ideas and style options based on user prompts.
- Using visual references to generate images that match a given subject or style.
- Transforming AI-generated images into videos, showcasing subjects from various perspectives.
- Publishing AI-generated assets on a shareable dashboard, enabling collaborative brainstorming.
Why launch an image-focused platform alongside a video generation model? Dai emphasized that video generation alone isn’t enough to appeal to the broad spectrum of users in AI-driven visual tools. Luma AI recognized that image generation has a wider range of applications and decided to build a platform that highlights its model’s versatility while remaining user-centric.
Despite its growth, Luma AI operates in a fiercely competitive environment where differentiation is key. Dream Machine AI’s advanced language comprehension stands out as best-in-class, and its text design and inpainting capabilities outshine those of both Midjourney and GPT-based tools, ensuring text in visuals is clear, professional, and visually engaging.
Yet, Luma AI has taken an unconventional marketing strategy, opting to spend nothing on advertising. “Burning money on marketing is only worth it if the ROI is significant. In this still-nascent AI market, it’s better to invest in product development than in marketing campaigns,” Dai said.
KrASIA Connection features translated and adapted content that was originally published by 36Kr. This article was written by Zhou Xinyu for 36Kr.