APIUp to 28% cheaper than official pricesTry the API →
Hermes
Back to News

Imperial College Researcher Shows Self-Attention Solves Quantum Pair Correlations Where Neural Nets Fail

New paper argues self-attention is a real-space Dyson iteration, with softmax acting as a two-body self-energy. DeepSets sum-pooling is shown to be mean-field (Hartree), so sensitivity to any single other particle shrinks as 1/N.

Imperial College Researcher Shows Self-Attention Solves Quantum Pair Correlations Where Neural Nets Fail
Source
AlphaSignal
Published
Author
AlphaSignal
Read
2 min read

Takeaways

  • New paper argues self-attention is a real-space Dyson iteration, with softmax acting as a two-body self-energy.
  • DeepSets sum-pooling is shown to be mean-field (Hartree), so sensitivity to any single other particle shrinks as 1/N.
  • Because Hamiltonians are pairwise, iterating a learned two-body kernel can in principle capture all correlated energetics.
  • On a six-particle fractal backflow target, attention recovers the nested nodal pockets while DeepSets smooths them away.
  • Attention tracks the exact pairwise Jacobian over four decades of separation, but uses 116,226 parameters versus DeepSets' 50,690.
  • Julia code is available in Halcyon.jl ; there are no real-molecule or quantum Monte Carlo benchmarks yet.

Many machine-learned interatomic potentials achieve sub-meV-per-atom errors on standard benchmarks by predicting local atomic contributions and summing them into a total energy. Electron wavefunctions and excited states pose a harder problem because their values can depend on specific, widely separated particle pairs. A paper by Jarvist Moore Frost proposes a physical explanation for the difference: self-attention behaves like a real-space Dyson iteration, with its softmax kernel acting as a learned two-body self-energy. The interpretation offers an architecture criterion based on the range and pair specificity of the target correlations.

Where uniform pooling loses the pair

Frost compares two architecture families used in quantum machine learning. Interatomic potentials commonly combine finite-cutoff equivariant message passing with a sum over predicted atomic energies. Neural wavefunction models such as FermiNet instead represent correlated electrons through one-electron and pairwise feature streams. PsiFormer simplifies that design by applying multi-head self-attention to one-electron features, and it has improved correlation estimates for molecular dimers.

An atomic-energy sum readout and a normalized feature mean have different derivatives, so the paper’s argument requires a precise boundary. Summing local energies is the natural way to produce an extensive total energy, and interactions already encoded upstream survive that operation. Frost’s criticism targets finite-cutoff features combined with uniform, permutation-invariant mixing when the desired dependence remains nonlocal and pair-specific.

For a normalized mean update, the derivative of particle i ’s output with respect to particle j scales as 1/N . This off-diagonal Jacobian measures how strongly moving one particle changes another particle’s representation. The scaling can suffice for nearsighted ground-state energies, where nearby atoms dominate local contributions. A sharply correlated wavefunction may require one distant particle to exert an order-one effect on another.

The 1/N scaling appears directly in a small NumPy sketch. This example illustrates the distinction and does not reproduce the paper’s implementation:

Reporting is indexed from AlphaSignal. Rights remain with the original publisher and cited sources.

Read original report

Sources

  1. 01arxiv.org
  2. 02arxiv.org
  3. 03github.com
  4. 04github.com
  5. 05bioscience.com.pk