Bernhard Schölkopf

Bernhard Schölkopf is Director at the Max Planck Institute for Intelligent Systems in Tübingen, Germany. He is coauthor of Learning with Kernels (2002) and is a coeditor of Advances in Kernel Methods: Support Vector Learning (1998), Advances in Large-Margin Classifiers (2000), and Kernel Methods in Computational Biology (2004), all published by the MIT Press.

  • Elements of Causal Inference

    Elements of Causal Inference

    Foundations and Learning Algorithms

    Jonas Peters, Dominik Janzing, and Bernhard Schölkopf

    A concise and self-contained introduction to causal inference, increasingly important in data science and machine learning.

    The mathematization of causality is a relatively recent development, and has become increasingly important in data science and machine learning. This book offers a self-contained and concise introduction to causal models and how to learn them from data. After explaining the need for causal models and discussing some of the principles underlying causal inference, the book teaches readers how to use causal models: how to compute intervention distributions, how to infer causal models from observational and interventional data, and how causal ideas could be exploited for classical machine learning problems. All of these topics are discussed first in terms of two variables and then in the more general multivariate case. The bivariate case turns out to be a particularly hard problem for causal learning because there are no conditional independences as used by classical methods for solving multivariate cases. The authors consider analyzing statistical asymmetries between cause and effect to be highly instructive, and they report on their decade of intensive research into this problem.

    The book is accessible to readers with a background in machine learning or statistics, and can be used in graduate courses or as a reference for researchers. The text includes code snippets that can be copied and pasted, exercises, and an appendix with a summary of the most important technical concepts.

  • Advances in Neural Information Processing Systems 19

    Advances in Neural Information Processing Systems 19

    Proceedings of the 2006 Conference

    Bernhard Schölkopf, John Platt, and Thomas Hofmann

    Papers from the 2006 flagship meeting on neural computation, with contributions from physicists, neuroscientists, mathematicians, statisticians, and computer scientists.

    The annual Neural Information Processing Systems (NIPS) conference is the flagship meeting on neural computation and machine learning. It draws a diverse group of attendees—physicists, neuroscientists, mathematicians, statisticians, and computer scientists—interested in theoretical and applied aspects of modeling, simulating, and building neural-like or intelligent systems. The presentations are interdisciplinary, with contributions in algorithms, learning theory, cognitive science, neuroscience, brain imaging, vision, speech and signal processing, reinforcement learning, and applications. Only twenty-five percent of the papers submitted are accepted for presentation at NIPS, so the quality is exceptionally high. This volume contains the papers presented at the December 2006 meeting, held in Vancouver.

  • Predicting Structured Data

    Predicting Structured Data

    Gökhan BakIr, Thomas Hofmann, Bernhard Schölkopf, Alexander J. Smola, Ben Taskar, and S.V.N Vishwanathan

    State-of-the-art algorithms and theory in a novel domain of machine learning, prediction when the output has structure.

    Machine learning develops intelligent computer systems that are able to generalize from previously seen examples. A new domain of machine learning, in which the prediction must satisfy the additional constraints found in structured data, poses one of machine learning's greatest challenges: learning functional dependencies between arbitrary input and output domains. This volume presents and analyzes the state of the art in machine learning algorithms and theory in this novel field. The contributors discuss applications as diverse as machine translation, document markup, computational biology, and information extraction, among others, providing a timely overview of an exciting field.

    Contributors Yasemin Altun, Gökhan Bakir, Olivier Bousquet, Sumit Chopra, Corinna Cortes, Hal Daumé III, Ofer Dekel, Zoubin Ghahramani, Raia Hadsell, Thomas Hofmann, Fu Jie Huang, Yann LeCun, Tobias Mann, Daniel Marcu, David McAllester, Mehryar Mohri, William Stafford Noble, Fernando Pérez-Cruz, Massimiliano Pontil, Marc'Aurelio Ranzato, Juho Rousu, Craig Saunders, Bernhard Schölkopf, Matthias W. Seeger, Shai Shalev-Shwartz, John Shawe-Taylor, Yoram Singer, Alexander J. Smola, Sandor Szedmak, Ben Taskar, Ioannis Tsochantaridis, S.V.N Vishwanathan, Jason Weston

    • Hardcover $47.00
    • Paperback $43.00
  • Semi-Supervised Learning

    Semi-Supervised Learning

    Olivier Chapelle, Bernhard Schölkopf, and Alexander Zien

    A comprehensive review of an area of machine learning that deals with the use of unlabeled data in classification problems: state-of-the-art algorithms, a taxonomy of the field, applications, benchmark experiments, and directions for future research.

    In the field of machine learning, semi-supervised learning (SSL) occupies the middle ground, between supervised learning (in which all training examples are labeled) and unsupervised learning (in which no label data are given). Interest in SSL has increased in recent years, particularly because of application domains in which unlabeled data are plentiful, such as images, text, and bioinformatics. This first comprehensive overview of SSL presents state-of-the-art algorithms, a taxonomy of the field, selected applications, benchmark experiments, and perspectives on ongoing and future research.Semi-Supervised Learning first presents the key assumptions and ideas underlying the field: smoothness, cluster or low-density separation, manifold structure, and transduction. The core of the book is the presentation of SSL methods, organized according to algorithmic strategies. After an examination of generative models, the book describes algorithms that implement the low-density separation assumption, graph-based methods, and algorithms that perform two-step learning. The book then discusses SSL applications and offers guidelines for SSL practitioners by analyzing the results of extensive benchmark experiments. Finally, the book looks at interesting directions for SSL research. The book closes with a discussion of the relationship between semi-supervised learning and transduction.

    • Hardcover $12.75
    • Paperback $42.00
  • Kernel Methods in Computational Biology

    Kernel Methods in Computational Biology

    Bernhard Schölkopf, Koji Tsuda, and Jean-Philippe Vert

    A detailed overview of current research in kernel methods and their application to computational biology.

    Modern machine learning techniques are proving to be extremely valuable for the analysis of data in computational biology problems. One branch of machine learning, kernel methods, lends itself particularly well to the difficult aspects of biological data, which include high dimensionality (as in microarray measurements), representation as discrete and structured data (as in DNA or amino acid sequences), and the need to combine heterogeneous sources of information. This book provides a detailed overview of current research in kernel methods and their applications to computational biology. Following three introductory chapters—an introduction to molecular and computational biology, a short review of kernel methods that focuses on intuitive concepts rather than technical details, and a detailed survey of recent applications of kernel methods in computational biology—the book is divided into three sections that reflect three general trends in current research. The first part presents different ideas for the design of kernel functions specifically adapted to various biological data; the second part covers different approaches to learning from heterogeneous data; and the third part offers examples of successful applications of support vector machine methods.

  • Advances in Neural Information Processing Systems 16

    Advances in Neural Information Processing Systems 16

    Proceedings of the 2003 Conference

    Sebastian Thrun, Lawrence K. Saul, and Bernhard Schölkopf

    Papers presented at the 2003 Neural Information Processing Conference by leading physicists, neuroscientists, mathematicians, statisticians, and computer scientists.

    The annual Neural Information Processing (NIPS) conference is the flagship meeting on neural computation. It draws a diverse group of attendees—physicists, neuroscientists, mathematicians, statisticians, and computer scientists. The presentations are interdisciplinary, with contributions in algorithms, learning theory, cognitive science, neuroscience, brain imaging, vision, speech and signal processing, reinforcement learning and control, emerging technologies, and applications. Only thirty percent of the papers submitted are accepted for presentation at NIPS, so the quality is exceptionally high. This volume contains all the papers presented at the 2003 conference.

  • Learning with Kernels

    Learning with Kernels

    Support Vector Machines, Regularization, Optimization, and Beyond

    Bernhard Schölkopf and Alexander J. Smola

    A comprehensive introduction to Support Vector Machines and related kernel methods.

    In the 1990s, a new type of learning algorithm was developed, based on results from statistical learning theory: the Support Vector Machine (SVM). This gave rise to a new class of theoretically elegant learning machines that use a central concept of SVMs—-kernels—for a number of learning tasks. Kernel machines provide a modular framework that can be adapted to different tasks and domains by the choice of the kernel function and the base algorithm. They are replacing neural networks in a variety of fields, including engineering, information retrieval, and bioinformatics.

    Learning with Kernels provides an introduction to SVMs and related kernel methods. Although the book begins with the basics, it also includes the latest research. It provides all of the concepts necessary to enable a reader equipped with some basic mathematical knowledge to enter the world of machine learning using theoretically well-founded yet easy-to-use kernel algorithms and to understand and apply the powerful algorithms that have been developed over the last few years.

    • Hardcover $95.00
    • Paperback $76.00
  • Advances in Large-Margin Classifiers

    Advances in Large-Margin Classifiers

    Alexander J. Smola, Peter Bartlett, Bernhard Schölkopf, and Dale Schuurmans

    The book provides an overview of recent developments in large margin classifiers, examines connections with other methods (e.g., Bayesian inference), and identifies strengths and weaknesses of the method, as well as directions for future research.

    The concept of large margins is a unifying principle for the analysis of many different approaches to the classification of data from examples, including boosting, mathematical programming, neural networks, and support vector machines. The fact that it is the margin, or confidence level, of a classification—that is, a scale parameter—rather than a raw training error that matters has become a key tool for dealing with classifiers. This book shows how this idea applies to both the theoretical analysis and the design of algorithms. The book provides an overview of recent developments in large margin classifiers, examines connections with other methods (e.g., Bayesian inference), and identifies strengths and weaknesses of the method, as well as directions for future research. Among the contributors are Manfred Opper, Vladimir Vapnik, and Grace Wahba.

  • Advances in Kernel Methods

    Advances in Kernel Methods

    Support Vector Learning

    Christopher J.C. Burges, Bernhard Schölkopf, and Alexander J. Smola

    The Support Vector Machine is a powerful new learning algorithm for solving a variety of learning and function estimation problems, such as pattern recognition, regression estimation, and operator inversion. The impetus for this collection was a workshop on Support Vector Machines held at the 1997 NIPS conference. The contributors, both university researchers and engineers developing applications for the corporate world, form a Who's Who of this exciting new area.

    Contributors Peter Bartlett, Kristin P. Bennett, Christopher J.C. Burges, Nello Cristianini, Alex Gammerman, Federico Girosi, Simon Haykin, Thorsten Joachims, Linda Kaufman, Jens Kohlmorgen, Ulrich Kreßel, Davide Mattera, Klaus-Robert Müller, Manfred Opper, Edgar E. Osuna, John C. Platt, Gunnar Rätsch, Bernhard Schölkopf, John Shawe-Taylor, Alexander J. Smola, Mark O. Stitson, Vladimir Vapnik, Volodya Vovk, Grace Wahba, Chris Watkins, Jason Weston, Robert C. Williamson

Contributor

  • Robotics

    Robotics

    Science and Systems VIII

    Nicholas Roy, Paul Newman, and Siddhartha Srinivasa

    Papers from a flagship conference reflect the latest developments in the field, including work in such rapidly advancing areas as human-robot interaction and formal methods.

    Robotics: Science and Systems VIII spans a wide spectrum of robotics, bringing together contributions from researchers working on the mathematical foundations of robotics, robotics applications, and analysis of robotics systems. This volume presents the proceedings of the eighth annual Robotics: Science and Systems (RSS) conference, held in July 2012 at the University of Sydney. The contributions reflect the exciting diversity of the field, presenting the best, the newest, and the most challenging work on such topics as mechanisms, kinematics, dynamics and control, human-robot interaction and human-centered systems, distributed systems, mobile systems and mobility, manipulation, field robotics, medical robotics, biological robotics, robot perception, and estimation and learning in robotic systems. The conference and its proceedings reflect not only the tremendous growth of robotics as a discipline but also the desire in the robotics community for a flagship event at which the best of the research in the field can be presented.

  • Dynamic Faces

    Dynamic Faces

    Insights from Experiments and Computation

    Cristóbal Curio, Heinrich H. Bülthoff, and Martin A. Giese

    State-of-the-art research on the perception of dynamic faces, a topic of importance to brain, cognitive, and computational sciences.

    The recognition of faces is a fundamental visual function with importance for social interaction and communication. Scientific interest in facial recognition has increased dramatically over the last decade. Researchers in such fields as psychology, neurophysiology, and functional imaging have published more than 10,000 studies on face processing. Almost all of these studies focus on the processing of static pictures of faces, however, with little attention paid to the recognition of dynamic faces, faces as they change over time—a topic in neuroscience that is also relevant for a variety of technical applications, including robotics, animation, and human-computer interfaces. This volume offers a state-of-the-art, interdisciplinary overview of recent work on dynamic faces from both biological and computational perspectives.

    The chapters cover a broad range of topics, including the psychophysics of dynamic face perception, results from electrophysiology and imaging, clinical deficits in patients with impairments of dynamic face processing, and computational models that provide insights about the brain mechanisms for the processing of dynamic faces. The book offers neuroscientists and biologists an essential reference for designing new experiments, and provides computer scientists with knowledge that will help them improve technical systems for the recognition, processing, synthesizing, and animating of dynamic faces.

  • Dataset Shift in Machine Learning

    Dataset Shift in Machine Learning

    Joaquin Quiñonero-Candela, Masashi Sugiyama, Anton Schwaighofer, and Neil D. Lawrence

    An overview of recent efforts in the machine learning community to deal with dataset and covariate shift, which occurs when test and training inputs and outputs have different distributions.

    Dataset shift is a common problem in predictive modeling that occurs when the joint distribution of inputs and outputs differs between training and test stages. Covariate shift, a particular case of dataset shift, occurs when only the input distribution changes. Dataset shift is present in most practical applications, for reasons ranging from the bias introduced by experimental design to the irreproducibility of the testing conditions at training time. (An example is -email spam filtering, which may fail to recognize spam that differs in form from the spam the automatic filter has been built on.) Despite this, and despite the attention given to the apparently similar problems of semi-supervised learning and active learning, dataset shift has received relatively little attention in the machine learning community until recently. This volume offers an overview of current efforts to deal with dataset and covariate shift. The chapters offer a mathematical and philosophical introduction to the problem, place dataset shift in relationship to transfer learning, transduction, local learning, active learning, and semi-supervised learning, provide theoretical views of dataset and covariate shift (including decision theoretic and Bayesian perspectives), and present algorithms for covariate shift.

    Contributors Shai Ben-David, Steffen Bickel, Karsten Borgwardt, Michael Brückner, David Corfield, Amir Globerson, Arthur Gretton, Lars Kai Hansen, Matthias Hein, Jiayuan Huang, Choon Hui Teo, Takafumi Kanamori, Klaus-Robert Müller, Sam Roweis, Neil Rubens, Tobias Scheffer, Marcel Schmittfull, Bernhard Schölkopf Hidetoshi Shimodaira, Alex Smola, Amos Storkey, Masashi Sugiyama