Visual Thinking

1. Visual perception is an active, constructive process driven by attention

We see very little at any given instant, but we can sample any part of our visual environment so rapidly with swift eye movement, that we think we have all of it at once in our consciousness experience.

Perception is not passive. Our brains actively construct our visual experience by rapidly sampling the environment through eye movements. This process is driven by attention, which directs our gaze to relevant information based on our current cognitive tasks and goals. The illusion of seeing everything at once is created by the brain's ability to quickly access any part of the visual field as needed.

Visual queries guide perception. We formulate visual queries to find specific information in our environment. These queries trigger eye movements to areas that are likely to contain the desired information. For example, when looking at a map to plan a route, we might make visual queries to find the starting and ending points, then search for connecting roads.

Key components of active perception:

• Rapid eye movements (saccades)
• Attention-driven sampling of the environment
• Visual queries based on current cognitive tasks
• Integration of sampled information into a coherent percept

2. The brain processes visual information through feature detection and pattern recognition

Neurons sensitive to more complex conjunction patterns are only found farther up the what processing pathway, and these cannot be used to plan eye movements.

Feature detection occurs early. The primary visual cortex (V1) contains neurons that respond to basic features like orientation, color, and motion. This early processing stage allows for rapid detection of simple visual elements, which can guide attention and eye movements.

Pattern recognition builds complexity. As visual information moves through the "what" pathway, increasingly complex patterns are processed. This hierarchical processing allows for the recognition of objects and scenes:

1. V1: Basic features (edges, colors)
2. V2 and V4: More complex shapes and patterns
3. Inferotemporal cortex: Object and scene recognition

Implications for design:

• Use basic features (color, orientation) to make important elements stand out
• Exploit pattern recognition for efficient communication of complex information
• Consider the hierarchy of visual processing when organizing information

3. Visual working memory has limited capacity but is crucial for cognitive tasks

We can remember at most only a half dozen temporal patterns in an hour of video, and these may not be the important or stereotyped ones.

Limited capacity. Visual working memory can only hold about three objects or patterns at a time. This limitation is a fundamental bottleneck in visual thinking and problem-solving.

Crucial for cognition. Despite its limited capacity, visual working memory is essential for integrating information across eye movements and maintaining a coherent representation of the visual world. It allows us to compare and manipulate visual information in our minds.

Strategies to work within visual working memory limitations:

• Break complex information into manageable chunks
• Use external representations (diagrams, notes) to offload memory demands
• Design interfaces that minimize the need to hold multiple items in memory simultaneously
• Leverage pattern recognition to compress information into meaningful units

4. Visual thinking involves a dance between internal and external representations

Meaning is what the brain performs in a dance with the external environment. In this dance tokens of meaning are spun off into electronic and social media and tokens of meaning are likewise picked up.

Internal-external interplay. Visual thinking is not confined to the mind but involves a constant interaction between internal mental processes and external representations. This dance allows us to offload cognitive demands onto the environment and leverage the power of visual pattern recognition.

Cognitive tools. External representations, such as diagrams, sketches, and digital interfaces, serve as cognitive tools that extend our mental capabilities. They allow us to manipulate complex ideas, discover new patterns, and communicate abstract concepts more effectively.

Examples of internal-external interactions in visual thinking:

• Sketching ideas to externalize and refine mental concepts
• Using diagrams to organize and analyze complex relationships
• Interacting with data visualizations to explore patterns and trends
• Collaborative problem-solving using shared visual representations

5. Effective design supports efficient visual queries and pattern finding

To support efficient visual search, a design should be given large-scale as well as small scale structure.

Hierarchical organization. Effective visual designs provide structure at multiple scales, allowing viewers to quickly navigate and find relevant information. This hierarchical organization supports efficient visual queries by guiding attention from broad overviews to specific details.

Pattern-friendly representations. Good designs transform data and concepts into visual patterns that are easily recognized and interpreted by the human brain. This approach leverages our innate pattern-finding abilities to facilitate rapid understanding and insight.

Design principles for supporting visual queries and pattern finding:

• Use consistent visual hierarchies to organize information
• Employ visual encodings that map naturally to data attributes
• Provide overview+detail views to support both global and local pattern recognition
• Use visual grouping techniques (proximity, similarity) to highlight relationships
• Incorporate interactive elements that reveal additional information on demand

6. Color perception is based on opponent-process theory and has specific design implications

Color sequences that vary mainly in luminance will be the most effective in revealing patterns in the data.

Opponent-process theory. Color perception is based on three opponent channels: red-green, yellow-blue, and black-white (luminance). This understanding has important implications for color design in visual displays.

Design considerations. Effective use of color in design requires careful consideration of the properties of these opponent channels:

Key principles for color design:

• Use luminance contrast for fine detail and text legibility
• Employ distinct hues (red, green, blue, yellow) for categorical differences
• Avoid relying on color alone for critical information (consider colorblind users)
• Use color saturation to indicate intensity or importance
• Create color scales that vary in both hue and luminance for continuous data

7. 2.5D design principles optimize the use of depth cues in visual displays

Depth cues should be used selectively to support design goals. It does not matter if they are combined in ways that are inconsistent with realism.

Selective use of depth. 2.5D design involves the strategic application of depth cues to support specific design goals, rather than aiming for full three-dimensional realism. This approach allows designers to leverage the benefits of depth perception while avoiding the complexities and potential drawbacks of full 3D representations.

Balancing clarity and depth. The key to effective 2.5D design is finding the right balance between providing depth information and maintaining visual clarity. This often involves using a subset of depth cues and applying them selectively to different elements of the design.

2.5D design principles:

• Minimize occlusion of important information
• Use cast shadows to indicate spatial relationships
• Apply selective focus to guide attention
• Employ linear perspective for spatial context
• Maintain text and critical elements in the image plane for legibility

8. Visual and verbal narratives have distinct strengths in communicating information

Natural language is full of qualifiers such as "if," "and," "but," "otherwise," "nevertheless," and "while." This is not formal mathematical logic, but it does allow for a kind of abstract reasoning.

Complementary strengths. Visual and verbal modes of communication have distinct advantages in conveying different types of information. Verbal language excels at expressing abstract concepts and logical relationships, while visual representations are powerful for conveying spatial relationships and patterns.

Hybrid approaches. Effective communication often combines visual and verbal elements to leverage the strengths of both modes. This integration allows for rich, multi-layered narratives that can address complex ideas and relationships.

Characteristics of visual and verbal communication:
Visual:

• Rapid pattern recognition
• Spatial relationships
• Emotional impact
• Simultaneous presentation of information

Verbal:

• Abstract concepts
• Logical relationships
• Temporal sequences
• Precise definitions

9. Sketching is a powerful tool for creative visual thinking and design

The power of sketching as a thinking tool comes from a combination of four things. The first is the fact that a line can represent many things because of the flexible interpretive pattern-finding capability of the visual system.

Cognitive offloading. Sketching allows designers to externalize their thoughts, reducing the cognitive load of holding complex ideas in working memory. This frees up mental resources for creative problem-solving and idea generation.

Flexible interpretation. The ambiguity of sketches enables multiple interpretations, fostering creative thinking and unexpected insights. Designers can discover new possibilities by reinterpreting their own sketches from different perspectives.

Benefits of sketching in the design process:

• Rapid externalization of ideas
• Support for iterative refinement
• Facilitation of communication and collaboration
• Discovery of new connections and possibilities
• Integration of mental imagery with external representations

10. Visual expertise develops through practice and automation of cognitive processes

As we get skilled at a particular task, like chopping onions, the operation eventually becomes semi-automatic. This frees up our higher-level control processes to deal with higher level problems, such as how to deal with an extra person coming to dinner.

Skill development. Visual expertise is developed through repeated practice, leading to the automation of lower-level perceptual and cognitive processes. This automation frees up mental resources for higher-level thinking and problem-solving.

Perceptual learning. As we gain expertise in a visual domain, we develop specialized pattern recognition capabilities that allow us to quickly identify relevant features and relationships. This perceptual learning is domain-specific and can lead to dramatic improvements in performance.

Stages of visual skill development:

1. Effortful, attention-demanding processing
2. Gradual automation of basic processes
3. Development of specialized pattern recognition
4. Integration of automated processes into higher-level cognitive strategies
5. Continuous refinement and adaptation to new challenges

Last updated: November 22, 2024