Information theory, coding and cryptography

1. Information Theory, Coding, and Cryptography are the Pillars of Modern Digital Communication.

Information theory, error control coding and cryptography are the three load-bearing pillars of modern digital communication systems.

Foundational concepts. The book centers on three interconnected fields essential for transmitting, storing, and securing digital data. Information theory quantifies information and its limits, coding deals with efficient and reliable data representation, and cryptography ensures secure communication. These disciplines are fundamental to technologies like the internet, mobile phones, and digital storage.

Interdependence. While distinct, these areas are deeply intertwined. Information theory sets the theoretical limits on compression (source coding) and reliable transmission (channel capacity), coding provides the practical methods to approach these limits, and cryptography applies coding principles to secure data against unauthorized access. Understanding their synergy is crucial for modern communication system design.

Broad applicability. The principles discussed extend beyond traditional communication systems. The author notes their relevance to electronic computers, electromechanical systems, data-processing devices, and even biological systems like nerve networks, highlighting the pervasive nature of information processing in the modern world.

2. Authored by an Expert with Extensive Academic and Industry Experience.

Ranjan Bose is currently Professor in the Department of Electrical Engineering at the Indian Institute of Technology (IIT) Delhi.

Qualified author. The book is written by a highly credentialed academic with significant practical experience. Dr. Ranjan Bose holds degrees from prestigious institutions (IIT Kanpur, University of Pennsylvania), worked as a Senior Design Engineer in the industry (Alliance Semiconductor Inc.), and has held visiting positions at international universities.

Relevant expertise. His academic career at IIT Delhi includes leadership roles (Microsoft Chair Professor, Head of Wireless Research Lab, founding Head of the Center of Excellence in Cyber Systems and Information Assurance) and numerous awards (URSI Young Scientist, Humboldt Fellowship, INAE Young Engineers Award, AICTE Career Award, BOYSCAST Fellowship, Dr Vikram Sarabhai Research Award). He is also involved in national educational initiatives (NPTEL video courses, Virtual Labs) and industry startups.

Teaching focus. Dr. Bose's extensive experience in delivering lectures on information theory, coding, and cryptography directly informs the book's content and pedagogical approach, ensuring it is grounded in both theoretical depth and practical relevance for students and professionals.

3. Designed for Students and Practicing Engineers: Accessible, Practical, and Balanced.

The primary aim of this book is to arouse the curiosity of the students.

Target audience. The book is specifically tailored for undergraduate and postgraduate students in electrical/electronics engineering and computer science, aiming to build a strong foundation in the core areas. It is also intended as a quick reference for practicing engineers.

Educational philosophy. The author emphasizes basic concepts and fundamental principles, motivating their application to practical problems. The goal is a "lively introduction" that is "fascinating-to-read, simple-to-understand, logical-to-follow and motivating-to-design," moving beyond rote learning towards developing engineering skills.

Balanced approach. A key challenge addressed is balancing mathematical rigor with accessibility. Quoting Richard W. Hamming, the author aims to use mathematics only where essential, providing intuitive explanations and numerous examples to make complex topics graspable for students without an exceptionally strong mathematical background.

4. The Book is Structured into Four Logical Parts.

The entire book has been divided into three logical parts: Part I—Information Theory and Source Coding, Part II—Error Control Coding (Channel Coding), and Part III—Coding for Secure Communications.

Comprehensive coverage. The book systematically covers the three core pillars, dividing the material into logical sections to guide the reader through the interconnected topics. The preface to the first edition outlines three parts, while the third edition's table of contents shows four parts, indicating a refinement in structure.

Progressive learning. The structure moves from fundamental concepts (Information Theory) to practical applications in ensuring reliable communication (Error Control Coding) and finally to securing information (Cryptography). This flow allows readers to build knowledge progressively.

Detailed breakdown. The four-part structure in the third edition is:

• Part I: Information Theory and Source Coding (Chapters 1-2)
• Part II: Error Control Coding (Channel Coding) (Chapters 3-6)
• Part III: Codes on Graph (Chapters 7-8)
• Part IV: Coding for Secure Communications (Chapters 9-10)
  This organization provides a clear roadmap through the various sub-disciplines covered.

5. Part I Focuses on Information Theory and Source Coding for Data Compression.

Part I deals with the fundamental concepts of information theory and source coding.

Core principles. This section introduces the theoretical underpinnings of information, including concepts like uncertainty, self-information, mutual information, and entropy. It establishes quantitative measures for information content.

Data compression. A major application of information theory is source coding, which aims to represent data using the minimum possible number of bits. Techniques covered include:

• Huffman Coding
• Shannon-Fano-Elias Coding
• Arithmetic Coding
• Lempel-Ziv Algorithm
• Run Length Encoding
• Rate Distortion Theory
  These methods are crucial for efficient data storage and transmission.

Multimedia applications. The concepts are applied to practical areas like image and video compression standards (JPEG, MPEG), demonstrating how theoretical principles translate into widely used technologies for handling multimedia data efficiently.

6. Part II Delves into Error Control Coding for Reliable Communication.

This part introduces the reader to the fascinating world of Error Control Coding (Channel Coding).

Combating noise. This section addresses the challenge of transmitting data reliably over noisy channels. Error control coding adds controlled redundancy to messages to enable detection and correction of errors introduced during transmission.

Key coding techniques. Various classes of error correcting codes are explored, including:

• Linear Block Codes (Hamming Codes, LDPC Codes)
• Cyclic Codes (Fire Codes, Golay Codes, CRC Codes)
• Bose–Chaudhuri Hocquenghem (BCH) Codes (Reed-Solomon Codes)
• Space–Time Codes (New in 3rd Edition)
  These codes offer different trade-offs between error correction capability, redundancy, and implementation complexity.

Decoding strategies. Alongside encoding, the principles and methods for decoding these codes are discussed, including syndrome decoding and techniques specific to BCH and RS codes, highlighting how the added redundancy is used to recover the original message.

7. Part III Explores Codes on Graph, Including Convolutional and Trellis Codes.

This part consists of following two fascinating chapters that deal with codes with memory.

Codes with memory. Unlike block codes which process data in fixed, independent blocks, codes on graph, particularly convolutional codes, incorporate memory, meaning the encoding of the current input depends on previous inputs. This structure is often represented using graphs like trellises.

Convolutional codes. Chapter 7 covers the details of convolutional codes, including their polynomial and matrix descriptions, distance notions, and the widely used Viterbi decoding algorithm. It also introduces Turbo Codes, a powerful class of concatenated convolutional codes, and their iterative decoding techniques.

Combined coding and modulation. Chapter 8 focuses on Trellis Coded Modulation (TCM), a technique that integrates coding and modulation to achieve coding gain without expanding bandwidth. Concepts like set partitioning and Ungerboeck’s design rules are discussed, along with TCM for different channel types.

8. Part IV Covers Coding for Secure Communications, Including Cryptography.

This part is dedicated to the topics related to secure communications.

Ensuring secrecy. This section shifts focus to protecting information from unauthorized access and tampering. Cryptography provides the tools and techniques for secure communication and data storage.

Encryption methods. Various cryptographic algorithms are presented, categorized into:

• Symmetric (Secret Key) Cryptography (DES, IDEA, RC Ciphers)
• Asymmetric (Public-Key) Cryptography (RSA Algorithm, Diffie-Hellman)
  The fundamental operations (substitution, transposition, XOR) and concepts like keys, digital signatures, and one-way hashing are explained.

Emerging areas. The third edition includes a new chapter on Physical Layer Security, which leverages information theory principles for secure communication. Other modern topics like Elliptic Curve Cryptography, Quantum Cryptography, Biometric Encryption, and Chaos-based Cryptography are also introduced, reflecting the evolving landscape of secure communications.

9. The Third Edition Introduces New Topics and Enhanced Learning Tools.

The main objectives of the revision are to update the material of the previous edition, to make it easier to teach and to add Learning Objectives in each chapter.

Updated content. The third edition incorporates recent advancements and feedback to ensure the material is current and relevant. This includes adding new topics like Relative Entropy, Video Compression Standards, Parallel Gaussian Channels, MIMO Systems, and Physical Layer Security.

New chapters. Two entirely new chapters have been added:

• Chapter 6: Space Time Codes
• Chapter 10: Physical Layer Security
  These additions reflect the growing importance of these areas in modern communication systems.

Improved pedagogy. The revision focuses on making the book more student-centric and easier for instructors to use. This includes the addition of clear Learning Objectives for each chapter and section, aligning the content with an outcomes-based learning approach.

10. Integrated Learning and Assessment Tools Support Student Understanding.

The third edition is student-centric, and follows the outcomes-based approach.

Structured learning. Each chapter follows a consistent organizational structure designed to facilitate learning. This includes clearly stated Learning Objectives at the beginning of each chapter and section, helping students anticipate the material and instructors measure understanding.

Reinforced concepts. Learning is supported through various integrated tools:

• In-text Examples: Over 210 solved examples illustrate concepts immediately after introduction.
• Learning Reviews: In-text exercises linked to LOs allow students to check understanding.
• Chapter-end Problems: Nearly 200 problems (categorized by difficulty/Bloom's Taxonomy level) reinforce material.
• Computer Problems: Over 70 problems challenge students and build practical skills.
• Project Ideas: Over 40 challenging ideas for individual or supervised projects.

Additional resources. The book is supplemented by online resources accessible via QR codes and a dedicated website, providing access to video overviews, solutions, lecture slides, and quizzes, enhancing the learning experience beyond the physical text.

11. Balancing Mathematical Rigor with Intuitive Explanations.

While writing this book, I had to take a decision regarding how mathematical the book should be.

Accessibility goal. The author explicitly addresses the challenge of presenting complex topics without overwhelming readers who may not have a strong mathematical background. The aim is to avoid scaring students away while still maintaining necessary rigor.

Strategic use of math. Mathematics is employed only where essential and to the extent required for understanding the core concepts. The author prioritizes intuitive explanations wherever possible, making the material more approachable.

Learning through examples. A key pedagogical choice is the heavy reliance on examples. The author believes "teaching by example is a very effective method of instruction" and ensures that new concepts are followed by numerical examples to solidify understanding. This practical approach complements the theoretical discussions.

Last updated: May 8, 2025