What If We Stopped Using GPUs? | YC Paper Club
A session on alternative compute covering the argument for abandoning backpropagation in favor of zero-order optimization and sharded training, optical computing that performs diffusion-style image generation with propagating light, the state of neuromorphic computing in 2026, and how cultured human neurons learned to play Doom through reinforcement learning.
Alternative Compute: Beyond Backprop, Light, Neuromorphics, and Living Neurons
The session opens with a deceptively simple question: if you could ask a super-intelligent alien one question, what would it be? The speaker’s answer is not about energy but about compute itself, namely how they count floating point operations. Deep learning hardware has been co-adapting with architectures and optimizers for over a decade. Since AlexNet in 2012, the industry optimized for raw FLOPs per dollar when CNNs dominated, then pivoted toward memory capacity and bandwidth once transformers arrived. Attention’s quadratic cost made memory the bottleneck, and FLOPs-per-joule gains have largely stalled over the last two years. The argument is that the cost per gigaflop must fall by orders of magnitude before anything approaches human-level intelligence, given that the human brain runs on roughly 20 watts.
The Case Against Backpropagation
The speaker argues that backprop is unlikely to survive the next decade. The brain is massively feedforward and does not fire signals backward to update synapses, which would require a weight transfer problem to be solved. Hebbian and related local learning rules approximate learning without the bookkeeping of stored activations. Cortical columns, surrounded by heavy inhibition, let local assemblies learn semi-independently, a structural feature that may be more relevant than the wiring diagram.
This motivates zero-order optimization. Rather than computing gradients, the method perturbs parameters in random directions with a small radius, takes finite differences from paired forward passes, and scales the update accordingly. The approach, SPSA or central difference, can optimize non-differentiable loss landscapes and avoids the local minima where backprop often stalls. In early experiments training a billion-parameter LSTM for next-token prediction, SPSA outperformed the other methods the speaker tested.
The weakness is scale. Gradient estimate noise grows with model size, so a naive 10B-parameter model would require an infeasible number of perturbations. The proposed fix is sharding: cluster a web-scale corpus, train a small expert on each shard, and combine them through a learned router. The gradient noise becomes bounded by the size of each expert, so only around 64 perturbations per step may suffice. The speaker reports that performance improves as the model is sharded more finely, a scaling law along a dimension that had not been characterized before. The efficiency case rests on the forward pass becoming cheap. In optics, a forward pass is essentially free, and in neuromorphic or analog hardware it can be very low power. Zero-order methods only pay off if that premise holds.
Computing With Light
The case for optical compute starts with communication. Nearly all data movement between AI racks and across continents already happens in photons, because light offers roughly 10,000 times lower loss and 10,000 times larger bandwidth than electrons. The argument goes further: converting between digital and optical domains at every boundary wastes energy, so the ideal is to keep computation in light entirely.
Optics also has a structural advantage. Photons do not interact with each other, so many beams can coexist in the same space without interference. A matrix-vector multiply in electronics scales with n squared because each element must be charged and discharged, whereas in optics inputs are modulated once, passed through a fixed weight mask, and summed at the detector, so energy scales roughly with n. Precision is also converging. Electronic processors used 32- or 64-bit representations, but AI workloads now tolerate much lower bit depths, while optical links carry increasingly wide bit depths. The two domains are meeting in the middle.
The obstacles are concrete. Digital-to-analog and analog-to-digital conversion dominate energy budgets, and programming and calibrating devices costs energy too. Passive optics handles linear operations well, but nonlinear activations require extra mechanisms that transistors provide for free. One experiment used intense short pulses in multimode fiber to induce optical nonlinearities, but it added considerable complexity. The practical takeaway is that early useful optical compute must be carefully matched to a use case where the physics provides a genuine advantage.
The demonstration applies this to diffusion-based image generation. Diffusion models start from random noise and repeatedly apply a trained network to denoise toward a data distribution, sometimes requiring around a thousand passes per sample on general-purpose hardware. The proposal replaces this with a physical device where light propagating through a stack of phase-modulating layers performs one denoising step per pass. The approach divides the schedule into segments and holds each segment’s weights fixed, so a small number of devices can be reused across many steps without rewriting the hardware. Experiments on MNIST and Fashion-MNIST show behavior consistent with a diffusion model, with FID improving as parameters scale. Scaling follows a power law similar to digital networks, which is encouraging for the energy argument, since energy should grow more gently than compute as parameters increase. Energy comparisons against a commercial GPU favored the optical system, with the caveat that the weights were assumed to be passive, meaning fabricated once and then fixed. The prototype used a spatial light modulator with a mirror, and backpropagation was performed on a digital twin, which remains the main bottleneck.
The discussion raised several open problems. Optical storage is harder than optical computation, since writing and reading memory cheaply in the optical domain has not yet matched electronics, though phase-change materials and holographic approaches are being explored. Feature sizes are constrained by wavelength, and wavelength-division multiplexing offers one way to exploit parallelism that does not depend on spatial scale. Polarization was suggested as another degree of freedom that could supply nonlinearity, though composing it across deep networks is unresolved. The speaker’s stated next milestone is an end-to-end system at billion-parameter scale to test whether the energy advantage holds when systems scale out.
Neuromorphic Computing in 2026
Neuromorphic computing aims to borrow the brain’s organizing principles for circuits and chips. The concept dates to Carver Mead’s work at Caltech in the late 1980s. The brain combines language, reasoning, and creativity in real time, learns continuously without context windows, and learns from very few examples. A sixteen-year-old needs to see a stop sign once or twice, while an autonomous car needs many views of the same sign. All of this runs on about 20 watts.
Several properties distinguish the brain. Memory and computation are intertwined, since synaptic weights take part in computation rather than merely relaying signals. Communication happens through discrete spikes whose timing and amplitude carry information, so the system is neither purely analog nor purely digital. Structure and function change with experience, so the brain rewires itself while conventional chips remain fixed after fabrication. A spiking neuron can be modeled as a leaky capacitor that fires a discrete pulse once accumulated charge crosses a threshold, and whose timing relationships carry continuous information.
The speaker’s central observation is that modern AI, despite borrowing the word “neural,” has moved far from biology. Backpropagation, transformer attention, and large-scale training on digital hardware have no direct counterpart in the brain. The real innovation has happened in digital silicon, not in faithful imitation. The brain is the ultimate hardware-software co-design, where the two are inseparable. Lifting a biological principle onto a different substrate therefore requires care, because the best implementation is likely to be whatever is native to that physics.
The field today is best described as R&D. Three directions dominate. The first treats the brain’s energy efficiency as a problem to solve: co-locating memory and compute, which is an example of stretching the definition of neuromorphic, since companies like d-Matrix do it digitally with SRAM and logic. The second explores event-driven, spiking, and on-device learning for sensing and real-time adaptation, where Intel has built a drone-control chip. The third builds on exotic devices such as resistive memories, memristors, and coupled oscillators, with the hope of designing models around their physical behavior. The speaker’s expectation is a long gestation period, similar to the decades neural networks spent before becoming useful.
The discussion touched on whether neuromorphic systems should simply become the brain, with the reply that the brain has limits relative to digital systems in clock speed and storage, though whether biology can be scaled remains open. On packaging, the speaker considers energy dominated by moving bits across interconnects rather than switching transistors, and believes optics will be the main answer. On hardware-software co-design, the choice depends on whether one starts from a physical insight or from an architecture, but in either case the design has to respect the strengths and weaknesses of the substrate. On materials, the trend has become more exotic over time, and the speaker resists fixating on silicon versus carbon. Reproducibility also came up. Brains tolerate noise that digital systems try to eliminate. Thermodynamic computing revisits this idea, using noise as a resource, and the speaker noted that zero-order optimization depends on reproducible perturbations, which is difficult to guarantee in biological systems and may be why the approach is not obviously biologically plausible.
Teaching Brain Cells to Play Doom
The final section describes work with Cortical Labs in which human neurons grown on a multi-electrode array learned to play Doom. The framing is dynamical systems rather than matrix arithmetic. The brain is treated as an input-output machine: game state is encoded as electrical stimulation patterns across channels, with amplitude and frequency as control variables, and the spiking response is decoded into in-game actions.
The earlier Pong work could be hand-designed because the action space was tiny. Doom has joint actions such as strafing, turning, attacking, and moving, and with only 59 channels, separable outputs for dozens of actions are hard to hand-engineer. The speaker argues that scaling to higher intelligence will require learning the encoder and decoder end to end. Because gradients cannot pass through the electrode array, the encoder is trained through stochastic stimulation search, which is closer to policy learning than to reservoir computing, since the culture itself changes with each stimulation. The architecture uses PPO with a critic, a CNN and MLP encoder, and a linear decoder.
A critical caution concerns overfitting. An oversized decoder trained with backpropagation can play the game itself, and the neurons never have to learn. The team undersized the decoder, ran ablations to confirm that spikes carried task information, and zeroed the bias of the linear readout. The speaker warns that much apparent biological intelligence in recent claims, including fly-brain work, may be silicon doing the work.
Feedback is delivered through patterns of stimulation. Synchronous stimulation signals good outcomes and asynchronous stimulation signals bad ones, drawing on the free energy principle, under which the brain is thought to avoid unpredictable input. Because the early policy is poor, feedback is scaled by surprise, measured as the TD error relative to the silicon critic, so that unexpectedly bad actions receive stronger negative feedback. The speaker clarified that the neurons do not feel pain or pleasure. Feedback is defined relative to the game, and the culture is best understood as minimizing its own entropy. Entropy penalties were used instead of the usual entropy bonus, since the cells already supplied enough variability. The speaker describes the learning as two dynamical systems competing with entropy and never reaching a fixed equilibrium.
Asked how different the stimulation is from natural sensory input, the speaker answered that it is very different, and that human intelligence is optimized for survival rather than computation. The substrate happens to host intelligence well. Growing sensory organoids and attaching them to brain tissue was raised as a possibility, though it raises ethical concerns. Scalability is the hardest open question. Two problems remain: achieving frontier-level intelligence on biological substrates, and serving that intelligence to billions of people. The speaker noted that distributed computing on living neurons is not yet understood, and that the team is working toward it.