APIUp to 28% cheaper than official pricesTry the API →
Hermes
Back to News

Nvidia's Nemotron 3.5 Super VL Trains Scientific AI Agents With Verifiable Rewards

NVIDIA announced Nemotron for Science at SC26, an open stack for building scientific research agents. Includes Nemotron 3.5 Super VL , a 120B multimodal reasoning model available on Hugging Face. Litmus-Bench v0.2 ships 50,000 chemistry reasoning questions for training and evaluation.

Nvidia's Nemotron 3.5 Super VL Trains Scientific AI Agents With Verifiable Rewards
Source
NVIDIA AI
Published
Author
NVIDIA AI
Read
5 min read

Takeaways

  • NVIDIA announced Nemotron for Science at SC26, an open stack for building scientific research agents.
  • Includes Nemotron 3.5 Super VL , a 120B multimodal reasoning model available on Hugging Face.
  • Litmus-Bench v0.2 ships 50,000 chemistry reasoning questions for training and evaluation.
  • Recipe uses GRPO plus LoRA with verifiable rewards via NeMo Gym and NeMo RL .
  • Drug metabolism example: SFT added 12% accuracy, RL added another 17% on top.
  • Argonne, Lila Sciences, Terray Therapeutics, Therna and Julich Supercomputing Centre are early adopters.

At SC26, the annual high-performance computing conference, Nvidia introduced Nemotron for Science , a collection of models, data, and post-training tools for scientific AI agents. Research teams can use the package to build systems that interpret multimodal inputs, plan experiments, call simulators, and check answers against measurable results.

Four pieces, one training path

The package supports the full adaptation workflow, from evaluating a base model to supervised fine-tuning and reinforcement learning with domain-specific rewards.

ComponentPurposeAccess
Nemotron 3.5 Super VLA 120-billion-parameter multimodal reasoning model for scientific workflows.Open weights through the model card.
Litmus-Bench v0.2Nearly 50,000 chemistry questions for training and evaluation.Open dataset on Hugging Face.
NeMo GymInfrastructure for turning experimental records, simulators, and scoring functions into reinforcement learning environments.Source code on NeMo Gym.
NeMo RLPost-training framework for GRPO, LoRA fine-tuning, and distillation.Source code on NeMo RL.

Nvidia’s drug-metabolism walkthrough shows how the components connect and provides a template for other fields with experimental ground truth or a reliable simulator.

Domain feedback moves the scores

Scientific agents often execute several dependent steps: interpreting a hypothesis, selecting an experiment, running a tool, evaluating its output, and revising the plan. An error early in that chain can distort every later step, especially when the base model lacks relevant chemistry or biology knowledge.

Nvidia reports that Nemotron 3.5 Super VL ranked first on seven of 11 science benchmarks in its evaluation, including scores of 75.9 on ChemBench and 39 on ether0 . Benchmark results depend on prompts, comparison models, and evaluation settings, so the model card and benchmark documentation remain necessary for reproducing those rankings.

For the drug-metabolism task, Nvidia says supervised fine-tuning increased accuracy by 12%, followed by a further 17% gain from reinforcement learning with verifiable rewards. The published figures do not clarify whether those gains are relative percentages or percentage points, and they include no confidence intervals.

A checker supplies the reward

Reinforcement learning with verifiable rewards, or RLVR, scores each answer with an automatic checker. In Nvidia’s metabolism example, the model receives a parent molecule and an enzyme, then predicts a metabolite as a SMILES string, a text notation for molecular structures.

The verifier calculates Tanimoto similarity between molecular fingerprints, which encode structural features for comparison. Scores range from 0 for no fingerprint overlap to 1 for identical fingerprints. This produces a cheap, repeatable reward, though fingerprint similarity alone cannot establish chemical validity or biological activity.

Group Relative Policy Optimization generates several candidate answers for each prompt and updates the model according to their scores within that group. Low Rank Adapters add small trainable matrices with the base weights frozen, reducing the optimizer state and memory required for parameter updates. The 120-billion-parameter base model still determines much of the memory needed for rollouts and inference.

NeMo Gym defines the task environment and verifier, while NeMo RL manages rollout workers, the reward service, and training. Adapting the pipeline to another domain primarily requires a prompt schema, representative records, and a verifier that returns a scalar score.

A teacher filters the warm-up data

Teams facing slow or unstable reinforcement learning can begin with model distillation. A stronger teacher generates reasoning traces in the same environment, the verifier scores each trace, and accepted examples form a supervised fine-tuning corpus for the student model.

Nvidia says it ran Anthropic’s Claude Sonnet 5.5 across 1,934 metabolism prompts and retained 1,123 traces with Tanimoto scores of at least 0.7. Nemotron trains on those filtered examples before entering the reinforcement learning loop.

After the environment is registered, Nvidia’s example launches GRPO with the following command:

The configuration requests four nodes with four GPUs each and caps the run at 100 training steps. Hardware type, numerical precision, sequence length, rollout count, and checkpoint strategy will determine the actual memory use and runtime.

Partners are starting with narrow workloads

Nvidia named Lila Sciences, Terray Therapeutics, and Therna among the life-sciences companies customizing Nemotron 3.5 Super VL. Argonne National Laboratory is integrating the model into a multi-agent system for designing and evaluating protein-based therapeutic candidates, while the Jülich Supercomputing Centre is also collaborating on scientific applications.

Nvidia reports that Lila Sciences reached 61% accuracy on a quantum-dot synthesis task, exceeded GPT-6 Astra in that evaluation, and reduced inference costs by a factor of 10 to 20. The release provides insufficient methodology to assess the comparison independently, including the exact test set, serving configuration, and cost calculation.

Useful when truth is computable

The strongest candidates are tasks with structured experimental records and a scoring function derived from reference data, scientific software, or an inexpensive simulator.

WorkloadPossible verifier
Metabolite predictionMolecular similarity, chemical-validity checks, or reaction software
Reaction outcome predictionReference products, atom mapping, or mass-balance checks
Materials property estimationMeasured values or physics-based simulation
Protein designStructure, energy, binding, or assay-based scores

The verifier sets the ceiling

  • Reward quality: A weak proxy can favor answers that score well without satisfying the underlying scientific objective.
  • Task coverage: Open-ended hypothesis generation offers few reliable automatic checks and therefore provides a weaker RL signal.
  • Data quality: Noisy measurements, inconsistent schemas, and narrow experimental coverage can transfer directly into the trained policy.
  • Compute: LoRA reduces training overhead, but rollouts from a 120-billion-parameter model still require substantial accelerator memory.

Access comes with a large compute bill

Nemotron 3.5 Super VL, Litmus-Bench v0.2, NeMo Gym, and NeMo RL are available through Hugging Face and GitHub. Nvidia publishes the model weights, dataset, and training code, with the applicable terms documented in each model card or repository.

The sample configuration allocates 16 GPUs across four nodes, making it a reference topology rather than a universal minimum. Teams evaluating the stack should calculate memory and throughput from the chosen checkpoint precision, context length, number of rollouts per prompt, adapter configuration, and target training duration.

Reporting is indexed from AlphaSignal. Rights remain with the original publisher and cited sources.

Read original report

Sources

  1. 01x.com
  2. 02developer.nvidia.com
  3. 03globenewswire.com
  4. 04techstrong.ai
  5. 05cdomagazine.tech