Skip navigation
Hardcover | $55.00 Short | £37.95 | ISBN: 9780262016247 | 656 pp. | 7 x 9 in | 151 b&w illus., 5 tables, 15 color plates| October 2011
Ebook | $39.00 Short | ISBN: 9780262299183 | 656 pp. | 7 x 9 in | 151 b&w illus., 5 tables, 15 color plates| October 2011
About MIT Press Ebooks

Visual Population Codes

Toward a Common Multivariate Framework for Cell Recording and Functional Imaging


Vision is a massively parallel computational process, in which the retinal image is transformed over a sequence of stages so as to emphasize behaviorally relevant information (such as object category and identity) and deemphasize other information (such as viewpoint and lighting). The processes behind vision operate by concurrent computation and message passing among neurons within a visual area and between different areas. The theoretical concept of "population code" encapsulates the idea that visual content is represented at each stage by the pattern of activity across the local population of neurons. Understanding visual population codes ultimately requires multichannel measurement and multivariate analysis of activity patterns. Over the past decade, the multivariate approach has gained significant momentum in vision research. Functional imaging and cell recording measure brain activity in fundamentally different ways, but they now use similar theoretical concepts and mathematical tools in their modeling and analyses.

With a focus on the ventral processing stream thought to underlie object recognition, this book presents recent advances in our understanding of visual population codes, novel multivariate pattern-information analysis techniques, and the beginnings of a unified perspective for cell recording and functional imaging. It serves as an introduction, overview, and reference for scientists and students across disciplines who are interested in human and primate vision and, more generally, in understanding how the brain represents and processes information.

About the Editors

Nikolaus Kriegeskorte is Principal Investigator at the Medical Research Council Cognition and Brain Sciences Unit in Cambridge, UK.

Gabriel Kreiman is Associate Professor of Ophthalmology and Neurology at Children’s Hospital Boston, Harvard Medical School. 

Table of Contents

  • Visual Population Codes
  • Computational Neuroscience
  • Terence J. Sejnowski and Tomaso A. Poggio, editors
  • The Computational Brain,
  • Patricia Churchland and Terrence Sejnowski, 1992
  • Dynamic Biological Networks: The Stomatogastic Nervous System,
  • Ronald Harris-Warrick, Eve Marder, Allen Selverston, and Maurice Moulins, eds. 1992
  • The Neurobiology of Neural Networks,
  • Daniel Gardner, ed. 1993
  • Large-Scale Neuronal Theories of the Brain,
  • Christof Koch and Joel Davis, eds. 1994
  • The Theoretical Foundations of Dendritic Function: Selected Papers of Wilfrid Rall with Commentaries,
  • Idan Segev, John Rinzel, and Gordon Shepherd, eds. 1995
  • Models of Information Processing in the Basal Ganglia,
  • James Houk, Joel Davis, and David Beiser, eds. 1995
  • Spikes: Exploring the Neural Code,
  • Fred Rieke, David Warland, Rob de Ruyter van Steveninck, and William Bialek, 1997
  • Neurons, Networks, and Motor Behavior,
  • Paul Stein, Sten Grillner, Allen Selverston, and Douglas Stuart, eds. 1997
  • Methods in Neuronal Modeling: From Ions to Networks, second edition,
  • Christof Koch and Idan Segev, eds. 1998
  • Fundamentals of Neural Network Modeling: Neuropsychology and Cognitive Neuroscience,
  • Randolph Parks, Daniel Levine, and Debra Long, eds.1998
  • Fast Oscillations in Cortical Circuits,
  • Roger Traub, John Jeffreys, and Miles Whittington, 1999
  • Computational Vision: Information Processing in Perception and Visual Behavior,
  • Hanspeter Mallot, 2000
  • Neural Engineering: Computation, Representation, and Dynamics in Neurobiological Systems,
  • Chris Eliasmith and Charles Anderson, 2003
  • The Computational Neurobiology of Reaching and Pointing,
  • Reza Shadmehr and Steven Wise, eds. 2005
  • Dynamical Systems in Neuroscience,
  • Eugene M. Izhikevich, 2006
  • Bayesian Brain: Probabilistic Approaches to Neural Coding,
  • Kenji Doya, Shin Ishii, Alexandre Pouget, and Rajesh Rao, eds. 2007
  • Computational Modeling Methods for Neuroscientists,
  • Erik De Schutter, ed. 2009
  • Neural Control Engineering,
  • Steven J. Schiff, 2011
  • Visual Population Codes: Toward a Common Multivariate Framework for Cell Recording and Functional Imaging,
  • Nikolaus Kriegeskorte and Gabriel Kreiman, eds., 2012
  • Visual Population Codes
  • Toward a Common Multivariate Framework for Cell Recording and Functional Imaging
  • edited by Nikolaus Kriegeskorte and Gabriel Kreiman
  • The MIT Press
  • Cambridge, Massachusetts
  • London, England
  • ©
  • 2012
  • Massachusetts Institute of Technology
  • All rights reserved. No part of this book may be reproduced in any form by any electronic or mechanical means (including photocopying, recording, or information storage and retrieval) without permission in writing from the publisher.
  • For information about special quantity discounts, please email
  • This book was set in Times Roman by Toppan Best-set Premedia Limited. Printed and bound in the United States of America.
  • Library of Congress Cataloging-in-Publication Data
  • Visual population codes : toward a common multivariate framework for cell recording and functional imaging / edited by Nikolaus Kriegeskorte and Gabriel Kreiman.
  •  p. cm. – (Computational neuroscience)
  • Includes bibliographical references and index.
  • ISBN 978-0-262-01624-7 (hardcover : alk. paper)
  • 1. Vision—Physiological aspects 2. Brain—Location of functions. 3. Neural transmission.  4. Primates—Physiology. 5. Comparative neurobiology. 6. Multivariate analysis. I. Kriegeskorte, Nikolaus, 1971– II. Kreiman, Gabriel, 1971–
  • QP475.V586 2012
  • 573.8'819—dc22
  • 2011007839
  • 10 9 8 7 6 5 4 3 2 1
  • Contents
  • Series Foreword ix
  • Preface xi
  • Introduction: A Guided Tour through the Book 1
  • I Theory and Experiment 21
  • 1 Grandmother Cells and Distributed Representations 23
  • Simon J. Thorpe
  • 2 Strategies for Finding Neural Codes 53
  • Sheila Nirenberg
  • 3 Multineuron Representations of Visual Attention 71
  • Jasper Poort, Arezoo Pooresmaeili, and Pieter R. Roelfsema
  • 4 Decoding Early Visual Representations from fMRI Ensemble Responses 101
  • Yukiyasu Kamitani
  • 5 Understanding Visual Representation by Developing Receptive-Field Models 133
  • Kendrick N. Kay
  • 6 System Identification, Encoding Models, and Decoding Models:
  • A Powerful New Approach to fMRI Research 163
  • Jack L. Gallant, Shinji Nishimoto, Thomas Naselaris, and Michael C. K. Wu
  • 7 Population Coding of Object Contour Shape in V4 and Posterior Inferotemporal Cortex 189
  • Anitha Pasupathy and Scott L. Brincat
  • 8 Measuring Representational Distances:
  • The Spike-Train Metrics Approach 213
  • Conor Houghton and Jonathan D. Victor
  • 9 The Role of Categories, Features, and Learning for the Representation of Visual Object Similarity in the Human Brain 245
  • Hans P. Op de Beeck
  • 10 Ultrafast Decoding from Cells in the Macaque Monkey 275
  • Chou P. Hung and James J. DiCarlo
  • 11 Representational Similarity Analysis of Object Population Codes in Humans, Monkeys, and Models 307
  • Nikolaus Kriegeskorte and Marieke Mur
  • 12 Three Virtues of Similarity-Based Multivariate Pattern Analysis:
  • An Example from the Human Object Vision Pathway 335
  • Andrew C. Connolly, M. Ida Gobbini, and James V. Haxby
  • 13 Investigating High-Level Visual Representations:
  • Objects, Bodies, and Scenes 357
  • Dwight J. Kravitz, Annie W-Y. Chan and Chris I. Baker
  • 14 To Err Is Human:
  • Correlating fMRI Decoding and Behavioral Errors to Probe the Neural Representation of Natural Scene Categories 391
  • Dirk B. Walther, Diane M. Beck, and Li Fei-Fei
  • 15 Decoding Visual Consciousness from Human Brain Signals 417
  • John-Dylan Haynes
  • 16 Probabilistic Codes and Hierarchical Inference in the Brain 441
  • Karl Friston
  • II Background and Methods 475
  • 17 Introduction to the Anatomy and Function of Visual Cortex 477
  • Kendra S. Burbank and Gabriel Kreiman
  • 18 Introduction to Statistical Learning and Pattern Classification 497
  • Jed Singer and Gabriel Kreiman
  • 19 Tutorial on Pattern Classification in Cell Recording 517
  • Ethan Meyers and Gabriel Kreiman
  • 20 Tutorial on Pattern Classification in Functional Imaging 539
  • Marieke Mur and Nikolaus Kriegeskorte
  • 21 Information-Theoretic Approaches to Pattern Analysis 565
  • Stefano Panzeri and Robin A. A. Ince
  • 22 Local Field Potentials, BOLD, and Spiking Activity:
  • Relationships and Physiological Mechanisms 599
  • Philipp Berens, Nikos K. Logothetis and Andreas S. Tolias
  • Contributors 625
  • Index 629


"Everything you wanted to know about neural coding in the visual system is in this book, and it is clearly presented by some of the best people in this important field of research."—Leo M. Chalupa, Vice President for Research, The George Washington University

"How the brain represents the world is a central question—maybe the central question— for neuroscience. This book uniquely integrates the new evidence from both single neuron and fMRI studies that are making real progress on this question in the ventral visual stream. It's a rare treat, with stimulating new ideas in every chapter, and the editors have welded chapters by cutting-edge scientists in an exceptionally coherent way."—Oliver Braddick, FMedSci, University of Oxford