Skip navigation

Machine Learning and Adaptive Computation

  • Page 2 of 10

This volume demonstrates the power of the Markov random field (MRF) in vision, treating the MRF both as a tool for modeling image data and, utilizing recently developed algorithms, as a means of making inferences about images. These inferences concern underlying image and scene structure as well as solutions to such problems as image reconstruction, image segmentation, 3D vision, and object labeling. It offers key findings and state-of-the-art research on both algorithms and applications.
After an introduction to the fundamental concepts used in MRFs, the book reviews some of the main algorithms for performing inference with MRFs; presents successful applications of MRFs, including segmentation, super-resolution, and image restoration, along with a comparison of various optimization methods; discusses advanced algorithmic topics; addresses limitations of the strong locality assumptions in the MRFs discussed in earlier chapters; and showcases applications that use MRFs in more complex ways, as components in bigger systems or with multiterm energy functions. The book will be an essential guide to current research on these powerful mathematical tools.

Using Complex Lexical Descriptions in Natural Language Processing

The last decade has seen computational implementations of large hand-crafted natural language grammars in formal frameworks such as Tree-Adjoining Grammar (TAG), Combinatory Categorical Grammar (CCG), Head-driven Phrase Structure Grammar (HPSG), and Lexical Functional Grammar (LFG). Grammars in these frameworks typically associate linguistically motivated rich descriptions (Supertags) with words. With the availability of parse-annotated corpora, grammars in the TAG and CCG frameworks have also been automatically extracted while maintaining the linguistic relevance of the extracted Supertags. In these frameworks, Supertags are designed so that complex linguistic constraints are localized to operate within the domain of those descriptions. While this localization increases local ambiguity, the process of disambiguation (Supertagging) provides a unique way of combining linguistic and statistical information.

This volume investigates the theme of employing statistical approaches with linguistically motivated representations and its impact on Natural Language Processing tasks. In particular, the contributors describe research in which words are associated with Supertags that are the primitives of different grammar formalisms including Lexicalized Tree-Adjoining Grammar (LTAG).

Contributors: Jens Bäcker, Srinivas Bangalore, Akshar Bharati, Pierre Boullier, Tomas By, John Chen, Stephen Clark, Berthold Crysmann, James R. Curran, Kilian Foth, Robert Frank, Karin Harbusch, Mary Harper, Saša Hasan, Aravind Joshi,Vincenzo Lombardo, Takuya Matsuzaki, Alessandro Mazzei, Wolfgang Menzel, Yusuke Miyao, Richard Moot, Alexis Nasr, Günter Neumann, Martha Palmer, Owen Rambow, Rajeev Sangal, Anoop Sarkar, Giorgio Satta, Libin Shen, Patrick Sturt, Jun’ichi Tsujii, K. Vijay-Shanker, Wen Wang, Fei Xia

In the field of machine learning, semi-supervised learning (SSL) occupies the middle ground, between supervised learning (in which all training examples are labeled) and unsupervised learning (in which no label data are given). Interest in SSL has increased in recent years, particularly because of application domains in which unlabeled data are plentiful, such as images, text, and bioinformatics. This first comprehensive overview of SSL presents state-of-the-art algorithms, a taxonomy of the field, selected applications, benchmark experiments, and perspectives on ongoing and future research.Semi-Supervised Learning first presents the key assumptions and ideas underlying the field: smoothness, cluster or low-density separation, manifold structure, and transduction. The core of the book is the presentation of SSL methods, organized according to algorithmic strategies. After an examination of generative models, the book describes algorithms that implement the low-density separation assumption, graph-based methods, and algorithms that perform two-step learning. The book then discusses SSL applications and offers guidelines for SSL practitioners by analyzing the results of extensive benchmark experiments. Finally, the book looks at interesting directions for SSL research. The book closes with a discussion of the relationship between semi-supervised learning and transduction.Olivier Chapelle and Alexander Zien are Research Scientists and Bernhard Schölkopf is Professor and Director at the Max Planck Institute for Biological Cybernetics in Tübingen. Schölkopf is coauthor of Learning with Kernels (MIT Press, 2002) and is a coeditor of Advances in Kernel Methods: Support Vector Learning (1998), Advances in Large-Margin Classifiers (2000), and Kernel Methods in Computational Biology (2004), all published by The MIT Press.

Computational systems biology aims to develop algorithms that uncover the structure and parameterization of the underlying mechanistic model—in other words, to answer specific questions about the underlying mechanisms of a biological system—in a process that can be thought of as learning or inference. This volume offers state-of-the-art perspectives from computational biology, statistics, modeling, and machine learning on new methodologies for learning and inference in biological networks.

The chapters offer practical approaches to biological inference problems ranging from genome-wide inference of genetic regulation to pathway-specific studies. Both deterministic models (based on ordinary differential equations) and stochastic models (which anticipate the increasing availability of data from small populations of cells) are considered. Several chapters emphasize Bayesian inference, so the editors have included an introduction to the philosophy of the Bayesian approach and an overview of current work on Bayesian inference. Taken together, the methods discussed by the experts in Learning and Inference in Computational Systems Biology provide a foundation upon which the next decade of research in systems biology can be built.

Contributors: Florence d'Alch e-Buc, John Angus, Matthew J. Beal, Nicholas Brunel, Ben Calderhead, Pei Gao, Mark Girolami, Andrew Golightly, Dirk Husmeier, Johannes Jaeger, Neil D. Lawrence, Juan Li, Kuang Lin, Pedro Mendes, Nicholas A. M. Monk, Eric Mjolsness, Manfred Opper, Claudia Rangel, Magnus Rattray, Andreas Ruttor, Guido Sanguinetti, Michalis Titsias, Vladislav Vyshemirsky, David L. Wild, Darren Wilkinson, Guy Yosiphon

Computational Molecular Biology series

The goal of machine learning is to program computers to use example data or past experience to solve a given problem. Many successful applications of machine learning exist already, including systems that analyze past sales data to predict customer behavior, optimize robot behavior so that a task can be completed using minimum resources, and extract knowledge from bioinformatics data. Introduction to Machine Learning is a comprehensive textbook on the subject, covering a broad array of topics not usually included in introductory machine learning texts. In order to present a unified treatment of machine learning problems and solutions, it discusses many methods from different fields, including statistics, pattern recognition, neural networks, artificial intelligence, signal processing, control, and data mining. All learning algorithms are explained so that the student can easily move from the equations in the book to a computer program.

The text covers such topics as supervised learning, Bayesian decision theory, parametric methods, multivariate methods, multilayer perceptrons, local models, hidden Markov models, assessing and comparing classification algorithms, and reinforcement learning. New to the second edition are chapters on kernel machines, graphical models, and Bayesian estimation; expanded coverage of statistical tests in a chapter on design and analysis of machine learning experiments; case studies available on the Web (with downloadable results for instructors); and many additional exercises. All chapters have been revised and updated.

Introduction to Machine Learning can be used by advanced undergraduates and graduate students who have completed courses in computer programming, probability, calculus, and linear algebra. It will also be of interest to engineers in the field who are concerned with the application of machine learning methods.

Adaptive Computation and Machine Learning series

Downloadable instructor resources available for this title: solution manual, programs, lecture slides, and file of figures in the book

Online decision making under uncertainty and time constraints represents one of the most challenging problems for robust intelligent agents. In an increasingly dynamic, interconnected, and real-time world, intelligent systems must adapt dynamically to uncertainties, update existing plans to accommodate new requests and events, and produce high-quality decisions under severe time constraints. Such online decision-making applications are becoming increasingly common: ambulance dispatching and emergency city-evacuation routing, for example, are inherently online decision-making problems; other applications include packet scheduling for Internet communications and reservation systems. This book presents a novel framework, online stochastic optimization, to address this challenge.

This framework assumes that the distribution of future requests, or an approximation thereof, is available for sampling, as is the case in many applications that make either historical data or predictive models available. It assumes additionally that the distribution of future requests is independent of current decisions, which is also the case in a variety of applications and holds significant computational advantages. The book presents several online stochastic algorithms implementing the framework, provides performance guarantees, and demonstrates a variety of applications. It discusses how to relax some of the assumptions in using historical sampling and machine learning and analyzes different underlying algorithmic problems. And finally, the book discusses the framework's possible limitations and suggests directions for future research.

The ubiquity of combinatorial optimization problems in our society is illustrated by the novel application areas for optimization technology, which range from supply chain management to sports tournament scheduling. Over the last two decades, constraint programming has emerged as a fundamental methodology to solve a variety of combinatorial problems, and rich constraint programming languages have been developed for expressing and combining constraints and specifying search procedures at a high level of abstraction. Local search approaches to combinatorial optimization are able to isolate optimal or near-optimal solutions within reasonable time constraints.

This book introduces a method for solving combinatorial optimization problems that combines constraint programming and local search, using constraints to describe and control local search, and a programming language, COMET, that supports both modeling and search abstractions in the spirit of constraint programming.

After an overview of local search including neighborhoods, heuristics, and metaheuristics, the book presents the architecture and modeling and search components of constraint-based local search and describes how constraint-based local search is supported in COMET. The book describes a variety of applications, arranged by meta-heuristics. It presents scheduling applications, along with the background necessary to understand these challenging problems. The book also includes a number of satisfiability problems, illustrating the ability of constraint-based local search approaches to cope with both satisfiability and optimization problems in a uniform fashion.

Principles and Techniques

Most tasks require a person or an automated system to reason--to reach conclusions based on available information. The framework of probabilistic graphical models, presented in this book, provides a general approach for this task. The approach is model-based, allowing interpretable models to be constructed and then manipulated by reasoning algorithms. These models can also be learned automatically from data, allowing the approach to be used in cases where manually constructing a model is difficult or even impossible. Because uncertainty is an inescapable aspect of most real-world applications, the book focuses on probabilistic models, which make the uncertainty explicit and provide models that are more faithful to reality. Probabilistic Graphical Models discusses a variety of models, spanning Bayesian networks, undirected Markov networks, discrete and continuous models, and extensions to deal with dynamical systems and relational data. For each class of models, the text describes the three fundamental cornerstones: representation, inference, and learning, presenting both basic concepts and advanced techniques. Finally, the book considers the use of the proposed framework for causal reasoning and decision making under uncertainty. The main text in each chapter provides the detailed technical development of the key ideas. Most chapters also include boxes with additional material: skill boxes, which describe techniques; case study boxes, which discuss empirical cases related to the approach described in the text, including applications in computer vision, robotics, natural language understanding, and computational biology; and concept boxes, which present significant concepts drawn from the material in the chapter. Instructors (and readers) can group chapters in various combinations, from core topics to more technically advanced material, to suit their particular needs.

Downloadable instructor resources available for this title: solution manual

Recent and Prospective Developments and Their Impact on Mathematical Physics, and Calculating Instruments and Machines

This reprint of Douglas Hartree's principal work also includes his inaugural Cambridge lecture, Calculating Machines: Recent and Prospective Developments and Their Impact on Mathematical Physics, which is extremely difficult to obtain and which makes ideal preliminary reading for the main set of lectures presented in Calculating Instruments and Machines. In these, Hartree provided the first comprehensive survey of the significant developments in computation that were going on at the time—the main directions of development in storage systems, serial machines, and parallel programming and coding, and particularly with high-speed automatic digital machines that were precursors of the modern stored program computer.

Calculating Instruments and Machines was originally published in 1949 by the University of Illinois Press. It is Volume VI in The Babbage Institute Reprint Series.

Dataset shift is a common problem in predictive modeling that occurs when the joint distribution of inputs and outputs differs between training and test stages. Covariate shift, a particular case of dataset shift, occurs when only the input distribution changes. Dataset shift is present in most practical applications, for reasons ranging from the bias introduced by experimental design to the irreproducibility of the testing conditions at training time. (An example is -email spam filtering, which may fail to recognize spam that differs in form from the spam the automatic filter has been built on.) Despite this, and despite the attention given to the apparently similar problems of semi-supervised learning and active learning, dataset shift has received relatively little attention in the machine learning community until recently. This volume offers an overview of current efforts to deal with dataset and covariate shift. The chapters offer a mathematical and philosophical introduction to the problem, place dataset shift in relationship to transfer learning, transduction, local learning, active learning, and semi-supervised learning, provide theoretical views of dataset and covariate shift (including decision theoretic and Bayesian perspectives), and present algorithms for covariate shift. Contributors [cut for catalog if necessary]Shai Ben-David, Steffen Bickel, Karsten Borgwardt, Michael Brückner, David Corfield, Amir Globerson, Arthur Gretton, Lars Kai Hansen, Matthias Hein, Jiayuan Huang, Choon Hui Teo, Takafumi Kanamori, Klaus-Robert Müller, Sam Roweis, Neil Rubens, Tobias Scheffer, Marcel Schmittfull, Bernhard Schölkopf Hidetoshi Shimodaira, Alex Smola, Amos Storkey, Masashi Sugiyama

  • Page 2 of 10