Robert E. Schapire

Robert E. Schapire is Principal Researcher at Microsoft Research in New York City. For their work on boosting, Freund and Schapire received both the Gödel Prize in 2003 and the Kanellakis Theory and Practice Award in 2004.

  • Boosting


    Foundations and Algorithms

    Robert E. Schapire and Yoav Freund

    An accessible introduction and essential reference for an approach to machine learning that creates highly accurate prediction rules by combining many weak and inaccurate ones.

    Boosting is an approach to machine learning based on the idea of creating a highly accurate predictor by combining many weak and inaccurate “rules of thumb.” A remarkably rich theory has evolved around boosting, with connections to a range of topics, including statistics, game theory, convex optimization, and information geometry. Boosting algorithms have also enjoyed practical success in such fields as biology, vision, and speech processing. At various times in its history, boosting has been perceived as mysterious, controversial, even paradoxical.

    This book, written by the inventors of the method, brings together, organizes, simplifies, and substantially extends two decades of research on boosting, presenting both theory and applications in a way that is accessible to readers from diverse backgrounds while also providing an authoritative reference for advanced researchers. With its introductory treatment of all material and its inclusion of exercises in every chapter, the book is appropriate for course use as well.

    The book begins with a general introduction to machine learning algorithms and their analysis; then explores the core theory of boosting, especially its ability to generalize; examines some of the myriad other theoretical viewpoints that help to explain and understand boosting; provides practical extensions of boosting for more complex learning problems; and finally presents a number of advanced theoretical topics. Numerous applications and practical illustrations are offered throughout.

    • Hardcover $57.00
    • Paperback $36.00
  • The Design and Analysis of Efficient Learning Algorithms

    Robert E. Schapire

    This monograph describes results derived from the mathematically oriented framework of computational learning theory.

    Approaches to building machines that can learn from experience abound - from connectionist learning algorithms and genetic algorithms to statistical mechanics and a learning system based on Piaget's theories of early childhood development. This monograph describes results derived from the mathematically oriented framework of computational learning theory. Focusing on the design of efficient learning algorithms and their performance, it develops a sound, theoretical foundation for studying and understanding machine learning. Since many of the results concern the fundamental problem of learning a concept from examples, Schapire begins with a brief introduction to the Valiant model, which has generated much of the research on this problem. Four self-contained chapters then consider different aspects of machine learning. Their contributions include a general technique for dramatically improving the error rate of a "weak" learning algorithm that can also be used to improve the space efficiency of many known learning algorithms; a detailed exploration of a powerful statistical method for efficiently inferring the structure of certain kinds of Boolean formulas from random examples of the formula's input-output behavior; the extension of a standard model of concept learning to accommodate concepts that exhibit uncertain or probabilistic behavior; (including a variety of tools and techniques for designing efficient learning algorithms in such a probabilistic setting); and a description of algorithms that can be used by a robot to infer the "structure" of its environment through experimentation.

    Robert E. Schapire received his doctorate from the Massachusetts Institute of Technology. He is now a member of the Artificial Intelligence Principles Research Department at AT&T Bell Laboratories.