Seven Concurrency Models in Seven Weeks

1. Concurrency and Parallelism are distinct but crucial for modern systems.

Concurrency is about dealing with lots of things at once. Parallelism is about doing lots of things at once.

Modern demands. Software today must handle multiple simultaneous events (concurrency) and leverage multi-core processors for speed (parallelism). The "multicore crisis" means performance gains no longer come automatically from faster single cores; developers must actively exploit parallelism. Beyond speed, concurrency enables responsiveness, fault tolerance, efficiency, and simplicity in complex, real-world systems.

Problem vs. Solution. Concurrency is often an aspect of the problem domain, requiring the system to manage many tasks that may overlap in time. Parallelism is an aspect of the solution domain, using simultaneous execution to speed up computation. While related and often confused, understanding the difference helps choose the right tools.

Hardware evolution. Parallelism exists at multiple hardware levels: bit-level (e.g., 32-bit vs. 8-bit operations), instruction-level (pipelining, out-of-order execution), data parallelism (SIMD, GPUs), and task-level (multiple CPUs). Task-level parallelism is further divided by memory architecture: shared memory (easier but limited scalability) and distributed memory (scalable, necessary for fault tolerance).

2. Threads and Locks: The foundational, yet error-prone, model.

Threads-and-locks programming is like a Ford Model T. It will get you from point A to point B, but it is primitive, difficult to drive, and both unreliable and dangerous compared to newer technology.

Direct hardware mapping. Threads and locks are simple formalizations of underlying hardware capabilities, making them widely available but offering minimal programmer support. This low-level control leads to significant challenges when multiple threads access shared mutable state.

Core perils. The primary dangers include race conditions (behavior depending on timing), deadlock (threads waiting indefinitely for each other's locks), and memory visibility issues (changes in one thread not visible to others due to optimizations). These issues are notoriously difficult to debug because they are often timing-dependent and hard to reproduce consistently.

Mitigation rules. While java.util.concurrent provides improved tools like ReentrantLock, AtomicInteger, and thread pools, the fundamental paradigm remains challenging. Best practices like synchronizing all shared variable access, acquiring locks in a fixed order, avoiding alien method calls while holding locks, and minimizing lock hold times are crucial but hard to enforce consistently in large codebases, making maintenance difficult.

3. Functional Programming: Immutability simplifies parallel execution.

Because they eliminate mutable state, functional programs are intrinsically thread-safe and easily parallelized.

Avoiding shared state. Functional programming models computation as expression evaluation using pure, side-effect-free functions. By avoiding mutable state entirely, functional programs eliminate the root cause of most concurrency bugs like race conditions and memory visibility problems associated with shared mutable data.

Effortless parallelism. Referential transparency, the property that a function call can be replaced by its result without changing program behavior, allows functional code to be reordered or executed in parallel safely. Operations like map and reduce on immutable data structures can often be trivially parallelized using library functions (e.g., Clojure's pmap, fold, reducers).

Lazy evaluation. Techniques like lazy sequences (generating elements only when needed) allow functional programs to work with potentially infinite data streams or datasets larger than memory without performance or memory issues. This contrasts with imperative approaches that might require explicit memory management or complex state handling for large data.

4. The Clojure Way: Managed mutable state complements functional purity.

Persistent data structures separate identity from state.

Pragmatic hybrid. Clojure is an impure functional language that balances functional purity with carefully managed mutable state. It recognizes that some problems inherently involve state change but provides concurrency-aware mechanisms to handle this safely, unlike the default mutable variables in imperative languages.

Identity vs. State. Clojure's persistent data structures (like vectors, maps, sets) are immutable; modifications return new versions while preserving old ones. This separates the "identity" (the conceptual variable) from its "state" (the sequence of values over time). Threads accessing a variable see a consistent, immutable state, avoiding concurrent modification issues.

Managed concurrency. Clojure provides distinct mutable variable types for different needs:

• Atoms: For independent, synchronous updates to a single value (using compare-and-set).
• Agents: For independent, asynchronous updates to a single value (updates processed serially).
• Refs (STM): For coordinated, synchronous updates to multiple values within atomic, isolated transactions that retry on conflict.

5. Actors: Isolated state and message passing enable robust concurrency and distribution.

The key in making great and growable systems is much more to design how its modules communicate rather than what their internal properties and behaviors should be.

Encapsulated state. Actors are concurrent entities that encapsulate their mutable state and communicate only by sending asynchronous messages to each other's mailboxes. This eliminates shared mutable state between actors, simplifying reasoning about concurrency within a single actor (which processes messages sequentially).

Resilience by design. The actor model, particularly as implemented in Erlang/Elixir (processes), excels at fault tolerance. Processes are lightweight and linked; if one fails abnormally, its linked process (often a supervisor) is notified and can take action, such as restarting the failed process ("let it crash" philosophy). This contrasts with defensive programming by separating error handling from core logic.

Seamless distribution. Actors support both shared and distributed memory architectures. Sending a message to an actor on a remote machine is often syntactically identical to sending one locally. This facilitates building systems that scale beyond a single machine and are resilient to machine failures.

6. Communicating Sequential Processes (CSP): Channels provide flexible concurrent coordination.

Channels are first class—instead of each process being tightly coupled to a single mailbox, channels can be independently created, written to, read from, and passed between processes.

Focus on the medium. Like actors, CSP involves concurrent processes communicating via messages. However, CSP emphasizes the communication channels themselves as first-class entities, rather than the processes. Channels act as thread-safe queues connecting producers and consumers, decoupling them.

Go blocks and parking. Implementations like Clojure's core.async use "go blocks" and inversion of control. Code within a go block appears sequential but is transformed into a state machine. Instead of blocking on channel operations (<!, >!), the go block "parks," yielding the thread, allowing many go blocks to be efficiently multiplexed over a small thread pool.

Simplified asynchronous I/O. This parking mechanism dramatically simplifies asynchronous programming, particularly for I/O and UI event handling. Callback-based code, often leading to "callback hell," can be rewritten as straightforward sequential code within go blocks, improving readability and maintainability. Channels can also be combined using functions like alt! (for selecting from multiple channels) and merge.

7. Data Parallelism: Harnessing hardware for massive numerical computation.

A modern GPU is a powerful data-parallel processor, capable of eclipsing the CPU when used for number-crunching...

Specialized parallelism. Data parallelism applies the same operation simultaneously to many data items. While not a general concurrency model, it's crucial for problems involving large-scale numerical computation, leveraging hardware like GPUs (Graphics Processing Units) or multi-core CPUs with SIMD instructions.

GPGPU via OpenCL. General-Purpose computing on GPUs (GPGPU) uses the GPU's parallel power for non-graphics tasks. OpenCL is a standard that allows writing portable GPGPU code using C-like kernels executed by work-items on devices. The host program manages context, queues, buffers, and kernel execution.

Performance gains. For suitable problems (e.g., matrix multiplication, simulations), GPUs can offer dramatic speedups over CPUs (often 10x-100x or more). Effective use requires structuring the problem into small work-items and managing data transfer between host and device memory, or ideally, keeping data resident on the device (e.g., integrating with OpenGL buffers).

8. The Lambda Architecture: Combining batch and stream processing for Big Data.

Big Data would not be possible without parallelism—only by bringing multiple computing resources to bear can we contemplate processing terabytes of data.

Addressing Big Data challenges. Traditional databases struggle with the scale, maintenance, complexity, and fault tolerance required for terabytes of data and high query volumes. The Lambda Architecture provides a framework for building scalable, fault-tolerant, and responsive Big Data systems.

Raw data as truth. The architecture is based on the insight that raw input data is immutable and eternally true (e.g., edits, transactions, events). By storing all raw data and deriving views from it, the system can recover from errors by recomputing views and supports arbitrary future queries on the complete historical record.

Batch and speed layers. The architecture has two main layers:

• Batch Layer: Uses batch processing (like Hadoop MapReduce) on the master dataset (all raw data) to compute comprehensive, but high-latency, batch views. Provides fault tolerance against hardware and human error.
• Speed Layer: Uses stream processing on recent data to compute incremental, low-latency, real-time views. Compensates for the batch layer's latency.
  Query results combine outputs from both layers.

Last updated: May 4, 2025