Your Neural Network Is Not a Tiny Brain
"Neural network" is one of the great marketing coups of computer science. The name conjures something brain-like: a web of neurons firing, learning, thinking. In reality, the artificial neuron is a distant, oversimplified cousin of the biological one, and the resemblance mostly ends at the name.
The original neuron was already a caricature
The whole lineage starts in 1943, when Warren McCulloch and Walter Pitts proposed a "neuron" that just adds up weighted inputs and fires if the sum crosses a threshold. They weren't trying to model the brain faithfully — they were showing that networks of simple logic gates could, in principle, compute anything a Turing machine could. That deliberately stripped-down unit is still, eight decades later, the basic building block of GPT-4 and every other deep learning model. The "inspired by the brain" story is true, but the inspiration was a sketch on a napkin, not a biology textbook.
Spikes versus numbers
Real neurons communicate in spikes: brief, all-or-nothing electrical pulses. A huge amount of information seems to live in the timing and rhythm of those spikes, not just how many there are. Artificial neurons skip all of that. They pass around plain numbers, continuously and instantly, with no timing dynamics at all. It's the difference between Morse code and a spreadsheet cell.
Backpropagation has no known biological counterpart
Nearly every deep network on Earth learns via backpropagation, which requires precisely shuttling error signals backward through the exact same connections used to send signals forward. Brains have no obvious mechanism for this. Neurons don't seem to keep a symmetric return path for error correction, a snag neuroscientists call the "weight transport problem." Whatever the brain is doing to learn, it almost certainly isn't running calculus on a copy of itself every night.
One neuron, one number vs. one neuron, a small computer
An artificial neuron is a single multiply-and-add operation. A biological neuron is arguably a computer in its own right: thousands of dendritic branches doing local, semi-independent processing, dozens of neurotransmitter types carrying different kinds of messages, and glial cells quietly influencing signaling in ways researchers are still mapping. Comparing the two units is a bit like comparing a light switch to a jazz quartet.
No labels, no epochs, no dataset
A network like an image classifier trains on millions of labeled examples, shown over and over across many epochs, with a clean split between "training" and "being used." A toddler learns what a dog is from a handful of glimpses, no labels attached, and never stops learning afterward. There's no dataset, no batch size, no test set — just one continuous, messy stream of experience.
The wiring never sits still
A trained network's architecture is fixed: the same layers, the same connections, forever, unless someone retrains it. A brain's wiring is constantly changing — synapses strengthen, weaken, form, and prune themselves throughout life, and new neurons even appear in a few regions well into adulthood. The network is a photograph; the brain is a documentary.
A human brain runs on roughly 20 watts, about what a dim light bulb uses. Training a large model can burn through megawatt-hours. Whatever trick biology is using, it's doing vastly more with vastly less.
None of this means the brain analogy was a mistake — it's a genuinely useful myth, the kind that gets a field off the ground. It just turns out the more interesting story isn't that we built a silicon brain. It's that a few crude ideas borrowed from neuroscience, stripped of nearly everything that makes a neuron a neuron, turned out to be enough to do something remarkable anyway.
