Good interfaces should support all three of Bruner's modes of representation
While working on Smalltalk, Alan Kay drew inspiration from the work of early child development theorist Jean Piaget, having been exposed to Piaget during graduate school via his relationship with Seymour Papert. (Piaget was one of Papert’s mentors.) This reliance on educational theory should be unsurprising: Kay’s primary mission, after all, was the development of computers as a “personal dynamic medium” that could be used by children to help them think, create, learn and explore their own thinking. The GUI was invented as a tool for thought.
In 1967, American psychologist Jerome Bruner began researching learning in children, attempting to replicate and then modify Piaget’s experiments. In the process, Bruner noticed that children could learn just about anything if it was organized and presented to them appropriately. This was a theoretical break from Piaget, who believed children could only learn what was appropriate for their stage of development. Bruner called the appropriate representation of learning material “scaffolding,” and classified this scaffolding into three different modes of representation:
- Enactive representation, action-based with knowledge learned through kinesthetic action
- Iconic representation, image-based with knowledge learned through pictures
- Symbolic representation, knowledge learned through language
Bruner may have had even greater influence on Kay than Piaget. If human cognition had three different possible mentalities, then Kay felt any new computing interface should engage all three. This was going to be especially important in designing interfaces for kids. Writing in User Interface: A Personal View:
… if we agree with the evidence that the human cognitive facilities are made up of a doing mentality, an image mentality, and a symbolic mentality, then any user interface we construct should at least cater to the mechanisms that seem to be there.
Relating Bruner’s theory to principles of the GUI, Kay then breaks it down as follows:
- Enactive: know where you are, manipulate (mouse)
- Iconic: recognize, compare, configure, concrete (icons, windows)
- Symbolic: tie together long chains of reasoning, abstract (Smalltalk, programming)
All of this leads Alan Kay to his pithy motto: “Doing with images makes symbols.”
References
Kay, A. (1990) User Interface: A Personal View.
Links to this note
-
Interfaces can enable entirely new modes of thought
…to be transformational, it needs to support all possible representation modalities (visual, kinesthetic, symbolic, maybe even auditory!). Good interfaces should support all three of Bruner’s modes of representation. Kinesthetic might even be the most important, because direct manipulation and rearrangement of a representation (often through epistemic actions) seems to draw on deep-seated reasoning abilities that we can’t access through other means. (Epistemic actions are taken to think, not to progress toward a goal.) References
-
The GUI was invented as a tool for thought
…Bruner was highly influential on Kay, which led him to assert (paraphrasing here) that Good interfaces should support all three of Bruner’s modes of representation.