Sean’s Public Notebook

Good interfaces should support all three of Bruner's modes of representation

While working on Smalltalk, Alan Kay drew inspiration from the work of early child development theorist Jean Piaget, having been exposed to Piaget during graduate school via his relationship with Seymour Papert. (Piaget was one of Papert’s mentors.) This reliance on educational theory should be unsurprising: Kay’s primary mission, after all, was the development of computers as a “personal dynamic medium” that could be used by children to help them think, create, learn and explore their own thinking. The GUI was invented as a tool for thought.

In 1967, American psychologist Jerome Bruner began researching learning in children, attempting to replicate and then modify Piaget’s experiments. In the process, Bruner noticed that children could learn just about anything if it was organized and presented to them appropriately. This was a theoretical break from Piaget, who believed children could only learn what was appropriate for their stage of development. Bruner called the appropriate representation of learning material “scaffolding,” and classified this scaffolding into three different modes of representation:

  1. Enactive representation, action-based with knowledge learned through kinesthetic action
  2. Iconic representation, image-based with knowledge learned through pictures
  3. Symbolic representation, knowledge learned through language

Bruner may have had even greater influence on Kay than Piaget. If human cognition had three different possible mentalities, then Kay felt any new computing interface should engage all three. This was going to be especially important in designing interfaces for kids. Writing in User Interface: A Personal View:

… if we agree with the evidence that the human cognitive facilities are made up of a doing mentality, an image mentality, and a symbolic mentality, then any user interface we construct should at least cater to the mechanisms that seem to be there.

Relating Bruner’s theory to principles of the GUI, Kay then breaks it down as follows:

  • Enactive: know where you are, manipulate (mouse)
  • Iconic: recognize, compare, configure, concrete (icons, windows)
  • Symbolic: tie together long chains of reasoning, abstract (Smalltalk, programming)

All of this leads Alan Kay to his pithy motto: “Doing with images makes symbols.”

References

Kay, A. (1990) User Interface: A Personal View.