APIUp to 28% cheaper than official pricesTry the API →
Hermes
Back to News

HeyGen's Code2Video Arena Lets Human Votes Decide the Best AI Motion Designer

Kaggle launched Code2Video Arena with HeyGen and HyperFrames for head-to-head model voting on motion graphics. Models receive identical briefs, write code, and users vote blind on which output wins. Built on HeyGen's Code2Video Benchmark of 168 launch-video briefs scored on five axes.

HeyGen's Code2Video Arena Lets Human Votes Decide the Best AI Motion Designer
Source
Kaggle
Published
Author
Kaggle
Read
4 min read

Takeaways

  • Kaggle launched Code2Video Arena with HeyGen and HyperFrames for head-to-head model voting on motion graphics.
  • Models receive identical briefs, write code, and users vote blind on which output wins.
  • Built on HeyGen's Code2Video Benchmark of 168 launch-video briefs scored on five axes.
  • Current top three: Gemini 4 Carbon, Claude Opus 5.5, GPT-6 Astra, all High tier.
  • Human voting flips the earlier automated leaderboard, where Claude Opus 5.5 held first place.
  • Open-weight models land within 50 Elo points of frontier labs on the same tasks.

Kaggle, HeyGen, and HyperFrames have launched Code2Video Arena , a public benchmark that asks two AI models to turn the same creative brief into coded motion graphics. People compare the resulting videos side by side, and each vote updates a preference-based leaderboard. The format gives developers a human signal for qualities that pass-or-fail tests struggle to capture, including composition, timing, and visual polish.

Blind votes, reproducible renders

Each matchup presents two videos labeled A and B. Both models receive the same brief, write code for a motion graphic, and send that code through the same rendering framework. Voters choose the stronger result without seeing which model produced it.

Model identities appear after the vote, limiting the influence of brand recognition during evaluation. The leaderboard changes as responses accumulate, so its rankings represent current voter preferences rather than fixed benchmark scores.

A controlled path from brief to frames

The Arena builds on the Code2Video Benchmark , released by HeyGen with Kaggle and Google DeepMind. Conventional coding benchmarks can verify whether code runs or passes tests. Code2Video evaluates whether that code produces a coherent, intentionally designed motion graphic.

  • Briefs: The benchmark contains 168 segments modeled on product-launch videos, including hooks, problem statements, product introductions, feature demonstrations, benefits, social proof, calls to action, and brand outros.
  • References: Each brief includes a human-made video for comparison.
  • Model input: A model receives the design brief and HeyGen system prompt in a single call, without external tools.
  • Model output: The model returns one HyperFrames composition using HTML for layout and GSAP for animation.
  • Rendering: HyperFrames, HeyGen’s open-source HTML-to-video framework, converts the composition into a deterministic video.
AxisWhat it measures
EngagementHow effectively the video attracts and sustains attention
Prompt intentAlignment with the requested message and creative direction
CompositionLayout, hierarchy, balance, and use of space
TemporalTiming, sequencing, transitions, and pacing
CraftTypography, consistency, detail, and overall finish

The original benchmark ran leading language models through all 168 prompts, then used a judge model for round-robin comparisons against human references and competing systems. The Arena adds public human preference data to that automated evaluation, allowing researchers to compare machine judgments with viewer choices.

Human votes reorder the top three

The launch leaderboard places Gemini 4 Carbon ahead of Claude Opus 5.5 and GPT-6 Astra:

RankModel
1Gemini 4 Carbon (High)
2Claude Opus 5.5 (High)
3GPT-6 Astra (High)

That order differs from the earlier automated Code2Video leaderboard, which placed Claude Opus 5.5 first under the single-call setup. The divergence suggests that human voters and the judge model weigh some visual decisions differently, although the ranking may shift as more votes arrive.

Arena results should be read with their vote totals and any reported uncertainty. Small differences can reflect the sampled briefs, the voter pool, or limited comparisons between particular model pairs.

Why code fits the test

HeyGen’s approach treats code generation as a controllable production stack for short product and brand videos. HTML defines the visual structure, GSAP controls movement, and HyperFrames renders the same composition consistently. Evaluators can rerun an output, inspect its source, and trace visual defects to specific layout or animation decisions.

Determinism addresses rendering reproducibility. Reliable comparisons still require matching browser versions, fonts, assets, viewport settings, and frame rates. The benchmark covers coded motion graphics; diffusion-based and photorealistic video generation fall outside its scope.

An independent implementation from Artificial Analysis, Code2Video-AA , applies similar briefs to additional models. Its results place Qwen 3.8 Max, DeepSeek V4.1 Flash, Kimi K3, and GLM 5.3 within 50 Elo points of first place. Elo summarizes pairwise wins, so scores from separate implementations remain comparable only when prompts, render environments, judges, and voting methods align.

Read the leaderboard without overreading it

Teams building tools that turn briefs, slides, or product copy into animated explainers can use the benchmark as a model-selection input. A production evaluation should cover several additional factors:

  1. Combine human and automated signals. Arena votes capture overall preference, while per-axis scores help identify failures in prompt adherence, composition, timing, or craft.
  2. Test representative briefs. Product demos, educational explainers, social clips, and brand videos impose different requirements. An internal set drawn from real workloads will reveal whether the public ranking transfers.
  3. Constrain the motion system. Define approved components, typefaces, assets, durations, easing functions, and transition patterns. Validate generated HTML, asset loading, and frame timing before rendering.
  4. Measure operational costs. Compare latency, token use, retry rates, render failures, licensing terms, and hardware requirements. The narrow Elo spread makes open-weight models useful candidates for testing, with deployment depending on the full operating profile.
  5. Track ranking stability. Recheck vote counts and confidence estimates as the Arena grows, especially when leading models have similar scores.

Code2Video extends Kaggle’s head-to-head Arena format from conversational answers to creative artifacts. By pairing blind human votes with inspectable code and repeatable renders, it gives developers a practical way to compare how models translate the same design brief into motion.

Reporting is indexed from AlphaSignal. Rights remain with the original publisher and cited sources.

Read original report

Sources

  1. 01x.com
  2. 02kaggle.com
  3. 03heygen.com
  4. 04heygen.com
  5. 05artificialanalysis.ai
  6. 06superpowerdaily.com