The Four Realms of Neural Networks
Four lenses on deep learning: PDE solvers, manifold geometry, gauge fields, and quantum-like attention. Each reveals structure the one below cannot express.
Key ideas
A residual network behaves like a numerical solver for a differential equation, with coefficients learned by gradient descent and boundary conditions set by the prompt.
Training flattens tangled data manifolds until classes separate, and weights act as a connection on a fiber bundle over the data, so geometry matters more than parameter count.
Attention works like a measurement whose basis the prompt selects, so AGI should be a small core model with dynamic memory for each person, not one model for everyone.
Read the full essay
Written by Wayland Zhang, founder of Dnalyaw.