Philosophers, cognitive scientists, and neuroscientists tend to have very different ideas about how to define a representation. I certainly won’t settle this issue in my talk, but I hope to provide data that informs how we think about visual representations, and why we cannot have more of them at once. I will focus on temporary information buffers (attention and working memory), and argue that these buffers have a two-dimensional “map” architecture where individual items compete for cortical real estate. This competitive format leads to capacity limits that are flexible, set by the nature of the content and their locations within the physically delimited cortical space. Using visual spatial attention and visual memory as case studies, I will suggest that these competitive maps are a widespread architecture that limits cognitive capacity across broader domains, and that this map architecture has important implications for how define visual representations.