Object recognition is a fundamental cognitive task that we perform countless times every day – such as right now when reading the words in this abstract. Yet, despite the apparent ease with which we see, object recognition is a very difficult computational problem. It is even more difficult from a biological perspective, since it involves several levels of understanding, from the level of cellular and biophysical mechanisms up to the level of brain systems and behavior. Remarkably, recent results from electrophysiology and neuroimaging covering areas as varied as face perception, categorizing morphed cars, learning to read new words, and learning to categorize monkey coos support an appealingly simple yet powerful unified account of how our brains make sense of the world. Interestingly, the computational model shows how leveraging prior learning can improve the ability to learn new tasks, and we have started to explore how this can be realized in the brain.