Nvidia open-sources Flora for cheaper robot video labeling

Nvidia has open-sourced Flora, an AI data engine designed to annotate robot training videos and generate reward signals, according to The Decoder. The company says the system can label robot video at a fraction of the cost of commercial AI models, which could matter for robotics teams dealing with very large video datasets.

Flora is aimed at a specific bottleneck: robots need video examples with text labels and feedback, but labeling enough footage for training and quality checks can become expensive. Nvidia’s pitch is that open models, run on its hardware, can turn that process into a scalable in-house pipeline rather than a per-call dependency on closed models.

What Flora is built to label

According to Nvidia’s GEAR research lab, robot video data needs three different kinds of annotation. One label type identifies what action is happening and when it occurs in the clip. A second scores how well the robot is progressing through a task, step by step. A third judges whether the task was completed successfully.

The Decoder reports that many existing systems handle only one of those jobs. Large language models can cover several, but Nvidia argues that they are too costly for the amount of video robotics teams need to process.

Flora is described in the paper as covering all three annotation types. Nvidia says it is already using the system while training its GR00T models, processing up to 200,000 hours of video per day across 512 GB300 GPUs. The models are available on Hugging Face.

For small robotics labs, simulation startups, and AI teams working with physical-world data, the key practical point is not just the headline cost. Flora’s appeal is that it combines segmentation, captioning, and reward labeling in one open pipeline, which could make it easier to run repeated data-processing passes without sending every clip to a commercial model.

Three Qwen models, each with a separate job

Flora is made up of three fine-tuned Qwen3.5 models. A 9B model divides video into action segments. A 27B model then writes descriptions while taking into account what happened before and after each action.

The third model, another 27B system called Flora-Rewarder, changes behavior based on the prompt. It can rate an episode’s success from 1 to 5, classify every half-second step as normal, suboptimal, or failure, and assign scores using rubrics it defines itself.

That structure makes Flora more like a data factory than a single chatbot-style model. For readers building automation around video review, the closest analogy is a multi-stage workflow: first detect events, then describe them, then score them for whether the task is going well.

This also explains why the comparison with general AI assistants is complicated. Nvidia’s system is specialized for robotics video, while models such as Claude and Google’s Gemini family are broader systems used across many tasks.

Nvidia’s cost claim comes with caveats

In a 50-video test, Nvidia says Flora performed slightly better than GPT-6 Astra, even though OpenAI’s model was allowed to use its full reasoning capacity. Nvidia estimates Flora can label one hour of video for about 72 cents, compared with roughly $88.50 for GPT-6 Astra and about $42 for Claude Opus 5 in the same test.

Those Flora figures are not production measurements. The Decoder notes that Nvidia calculated them from an assumed rental cost of $10 per GPU per hour. That means teams should treat the numbers as a projection, not a guaranteed operating bill.

The benchmark picture is also uneven. On WGO-Bench, Flora is best at finding action boundaries, but GPT-6 Astra leads in overall score and Gemini 3.8 Flash is ahead on description accuracy. Nvidia’s researchers attribute Flora’s advantage mainly to better temporal localization.

On Nvidia’s DenseCaptionBench, Flora leads across all metrics for human recordings. For robot videos, it ranks second behind Astra. The researchers also acknowledge a testing issue: GPT-6 Astra is used to evaluate descriptions while also being one of the systems being tested.

For teams deciding whether to build on Flora, the lesson is straightforward: its strongest case appears to be action detection and robotics-specific reward labeling, not necessarily the most polished natural-language description in every setting.

Why robotics teams may pay attention

Flora’s most distinctive result is on RoboRewardBench. Nvidia’s paper says Flora’s error rate is 27.8 percent lower than Astra’s, with the difference tied to partially completed tasks. In one example, Flora gives partial credit when a robot tips over a bottle but does not knock it off the table, matching the human reference more closely than Astra.

Flora-Rewarder also catches 53.3 percent of failed steps within an episode. The Decoder reports that no competitor exceeds 11.7 percent on that measure, and Astra catches none. The tradeoff is false positives: Flora incorrectly marks about 11 percent of normal steps as errors. Astra still leads on rubric-based scoring and trajectory ranking.

The system can also be very sensitive to short movements. In one video with many hand motions, Flora identifies 34 of 37 actions, while Astra detects four. But Nvidia’s model can over-segment longer actions and sometimes misdescribe what it sees, including one case where it described wiping a plate with a cloth as folding a napkin.

Nvidia positions Flora as an open alternative to commercial services such as Scale AI’s Physical AI Data Engine. The release also fits a broader company strategy: Nvidia has been investing in open AI models and automated data processing to support robotics development and demand for its chips. For more context on that open-model push, see our coverage of Nvidia’s open-weight models for enterprise and PC agents.

Google is moving in a related direction with Gemini 4 Argon, which The Decoder says targets a similar part of the robotics training process by identifying work steps and failed actions in training videos. We covered that release here: Google releases Gemini 4 Argon.