Skip navigation

Design and Design Theory

A Programming Handbook for Visual Designers and Artists

The visual arts are rapidly changing as media moves into the web, mobile devices, and architecture. When designers and artists learn the basics of writing software, they develop a new form of literacy that enables them to create new media for the present, and to imagine future media that are beyond the capacities of current software tools. This book introduces this new literacy by teaching computer programming within the context of the visual arts. It offers a comprehensive reference and text for Processing (, an open-source programming language that can be used by students, artists, designers, architects, researchers, and anyone who wants to program images, animation, and interactivity. Written by Processing’s cofounders, the book offers a definitive reference for students and professionals. Tutorial chapters make up the bulk of the book; advanced professional projects from such domains as animation, performance, and installation are discussed in interviews with their creators.

This second edition has been thoroughly updated. It is the first book to offer in-depth coverage of Processing 2.0 and 3.0, and all examples have been updated for the new syntax. Every chapter has been revised, and new chapters introduce new ways to work with data and geometry. New “synthesis” chapters offer discussion and worked examples of such topics as sketching with code, modularity, and algorithms. New interviews have been added that cover a wider range of projects. “Extension” chapters are now offered online so they can be updated to keep pace with technological developments in such fields as computer vision and electronics.

SUE.C, Larry Cuba, Mark Hansen, Lynn Hershman Leeson, Jürg Lehni, LettError, Golan Levin and Zachary Lieberman, Benjamin Maus, Manfred Mohr, Ash Nehru, Josh On, Bob Sabiston, Jennifer Steinkamp, Jared Tarbell, Steph Thirion, Robert Winter


(Pr) Casey Reas from Protein® on Vimeo.


Insight through Inquiry

Interactive visualization is emerging as a vibrant new form of communication, providing compelling presentations that allow viewers to interact directly with information in order to construct their own understandings of it. Building on a long tradition of print-based information visualization, interactive visualization utilizes the technological capabilities of computers, the Internet, and computer graphics to marshal multifaceted information in the service of making a point visually. This book offers an introduction to the field, presenting a framework for exploring historical, theoretical, and practical issues. It is not a “how-to” book tied to specific and soon-to-be-outdated software tools, but a guide to the concepts that are central to building interactive visualization projects whatever their ultimate form.

The framework the book presents (known as the ASSERT model, developed by the author), allows the reader to explore the process of interactive visualization in terms of choosing good questions to ask; finding appropriate data for answering them; structuring that information; exploring and analyzing the data; representing the data visually; and telling a story using the data. Interactive visualization draws on many disciplines to inform the final representation, and the book reflects this, covering basic principles of inquiry, data structuring, information design, statistics, cognitive theory, usability, working with spreadsheets, the Internet, and storytelling.

The mobile device is changing the ways we interact with each other and with the world. The mobile experience is distinct from the desktop or laptop experience; mobile apps require a significantly different design philosophy as well as design methods that reflect the unique experience of computing in the world. This book presents an approach to designing mobile media that takes advantage of the Internet-connected, context-aware, and media-sharing capabilities of mobile devices. It introduces tools that can be used at every stage of building a mobile application, from concept creation to commercialization, as well as real-world examples from industry and academia.

The methods outlined apply user-centered design processes to mobile devices in a way that makes these methods relevant to the mobile experience--which involves the use of systems in the complex spatial and social world rather than at a desk. The book shows how each project begins with generative research into the practices and desires of a diverse set of potential users, which grounds research and design in the real world. It then describes methods for rapid prototyping, usability evaluation, field testing, and scaling up solutions in order to bring a product to market. Building Mobile Experiences grew out of an MIT course in communicating with mobile technology; it is appropriate for classroom use and as a reference for mobile app designers.

Pixels, Numbers, and Programs

This book explores image processing from several perspectives: the creative, the theoretical (mainly mathematical), and the programmatical. It explains the basic principles of image processing, drawing on key concepts and techniques from mathematics, psychology of perception, computer science, and art, and introduces computer programming as a way to get more control over image processing operations. It does so without requiring college-level mathematics or prior programming experience. The content is supported by PixelMath, a freely available software program that helps the reader understand images as both visual and mathematical objects.

The first part of the book covers such topics as digital image representation, sampling, brightness and contrast, color models, geometric transformations, synthesizing images, stereograms, photomosaics, and fractals. The second part of the book introduces computer programming using an open-source version of the easy-to-learn Python language. It covers the basics of image analysis and pattern recognition, including edge detection, convolution, thresholding, contour representation, and K-nearest-neighbor classification. A chapter on computational photography explores such subjects as high-dynamic-range imaging, autofocusing, and methods for automatically inpainting to fill gaps or remove unwanted objects in a scene. Applications described include the design and implementation of an image-based game.

The PixelMath software provides a “transparent” view of digital images by allowing the user to view the RGB values of pixels by zooming in on an image. PixelMath provides three interfaces: the pixel calculator; the formula page, an advanced extension of the calculator; and the Python window.

Principles of Interaction Design as a Cultural Practice

Digital artifacts from iPads to databases pervade our lives, and the design decisions that shape them affect how we think, act, communicate, and understand the world. But the pace of change has been so rapid that technical innovation is outstripping design. Interactors are often mystified and frustrated by their enticing but confusing new devices; meanwhile, product design teams struggle to articulate shared and enduring design goals. With Inventing the Medium, Janet Murray provides a unified vocabulary and a common methodology for the design of digital objects and environments. It will be an essential guide for both students and practitioners in this evolving field.
Murray explains that innovative interaction designers should think of all objects made with bits--whether games or Web pages, robots or the latest killer apps--as belonging to a single new medium: the digital medium. Designers can speed the process of useful and lasting innovation by focusing on the collective cultural task of inventing this new medium. Exploring strategies for maximizing the expressive power of digital artifacts, Murray identifies and examines four representational affordances of digital environments that provide the core palette for designers across applications: computational procedures, user participation, navigable space, and encyclopedic capacity.
Each chapter includes a set of Design Explorations--creative exercises for students and thought experiments for practitioners--that allow readers to apply the ideas in the chapter to particular design problems. Inventing the Medium also provides more than 200 illustrations of specific design strategies drawn from multiple genres and platforms and a glossary of design concepts.

Principles of Interaction Programming

Interactive systems and devices, from mobile phones to office copiers, do not fulfill their potential for a wide variety of reasons—not all of them technical. Press On shows that we can design better interactive systems and devices if we draw on sound computer science principles. It uses state machines and graph theory as a powerful and insightful way to analyze and design better interfaces and examines specific designs and creative solutions to design problems. Programmers—who have the technical knowledge that designers and users often lack—can be more creative and more central to interaction design than we might think. Sound programming concepts improve device design.

Press On provides the insights, concepts and programming tools to improve usability. Knowing the computer science is fundamental, but Press On also shows how essential it is to have the right approaches to manage the design of systems that people use. Particularly for complex systems, the social, psychological and ethical concerns—the wider design issues—are crucial, and these are covered in depth.

Press On highlights key principles throughout the text and provides cross-topic linkages between chapters and suggestions for further reading. Additional material, including all the program code used in the book, is available on an interactive web site. Press On is an essential textbook and reference for computer science students, programmers, and anyone interested in the design of interactive technologies.

Activity Theory and Interaction Design

Activity theory holds that the human mind is the product of our interaction with people and artifacts in the context of everyday activity. Acting with Technology makes the case for activity theory as a basis for understanding our relationship with technology. Victor Kaptelinin and Bonnie Nardi describe activity theory's principles, history, relationship to other theoretical approaches, and application to the analysis and design of technologies. The book provides the first systematic entry-level introduction to the major principles of activity theory. It describes the accumulating body of work in interaction design informed by activity theory, drawing on work from an international community of scholars and designers. Kaptelinin and Nardi examine the notion of the object of activity, describe its use in an empirical study, and discuss key debates in the development of activity theory. Finally, they outline current and future issues in activity theory, providing a comparative analysis of the theory and its leading theoretical competitors within interaction design: distributed cognition, actor-network theory, and phenomenologically inspired approaches.