UNIQUE SERIES AKTU Syllabus
BEC072

Information Theory & Coding

L-T-P3-0-0 Credits3 Session2025-26 TypeTheory
Appears in 1 semester

B.Tech Electronics & Communication Engineering · Sem 7 (Departmental Elective-IV)

Download PDF

Syllabus

5 units · 40 lecture hours
Unit1

Unit 1: Entropy

8 hrs

Entropy, Joint Entropy and Conditional Entropy, Relative Entropy and Mutual Information, Relationship Between Entropy and Mutual Information, Chain Rules for Entropy, Relative Entropy and Mutual Information, Jensen’s Inequality and Its Consequences, Log Sum Inequality and Its Applications, Data- Processing Inequality, Fano’s Inequality.

Unit2

Unit 2: Asymptotic Equipartition Property

8 hrs

Asymptotic Equipartition Property Theorem. Consequences of the AEP: Data Compression, High-Probability Sets and the Typical Set Data Compression: Examples of Codes, Kraft Inequality, Optimal Codes, Bounds on the Optimal Code Length, Kraft Inequality for Uniquely Decodable Codes, Huffman Codes, Optimality of Huffman Codes, Shannon– Fano–Elias Coding.

Unit3

Unit 3: Channel Capacity

8 hrs

Channel Capacity for Various Binary Channels, Symmetric Channels, Properties of Channel Capacity, Preview of Channel Coding Theorem, Jointly Typical Sequences, Channel Coding Theorem, Channel capacity Theorem.

Unit4

Unit 4: Block Codes

8 hrs

Introduction to block codes, Single-parity check codes, Product codes, Repetition codes, Hamming codes, Minimum distance of block codes, Soft- decision decoding, Automatic-repeat-request schemes. Linear Block codes: Definition of linear Block Codes, Generator matrices, Standard array, Parity-check matrices, Error detection and correction.

Unit5

Unit 5: Convolution codes

8 hrs

Encoding convolutional codes, Generator matrices for convolutional codes, Generator polynomials for convolutional codes, Graphical representation of convolutional codes, Viterbi Algorithm, Binary Cycle Codes, BCH codes. RS codes, Golay codes.

Course Outcomes (COs)5 outcomes

After completion of the course student will be able to:

  1. CO1Explain each block involved in digital communication thoroughly with applications.
  2. CO2Apply the knowledge of basic concepts of probability and entropies to analyze the behavior of a communication system.
  3. CO3Analyze the use of source coding and evaluating all the techniques of source coding.
  4. CO4Examine the significance of channel coding and evaluating all available techniques of channel coding and decoding with challenges.
  5. CO5Examine various error control coding techniques.

Text Books

  1. Bose, Information Theory, Coding and Cryptography, McGraw-Hill Education, 3rd Edition, (2016).
  2. Joy A. Thomas, Thomas M. Cover, “Elements of information theory”, Wiley-Interscience; 2nd edition (July 18, 2006).
  3. S. Gravano, “Introduction to Error Control Codes” OUP Oxford (24 May 2001).
  4. Robert B. Ash, “Information Theory”, Dover Publications (November 1, 1990).
  5. Todd k Moon, “Error Correction Coding: Mathematical Methods and Algorithms ” Wiley, 2005.

Reference Books

  1. Simon Haykin, “Digital communication”, John Wiley.
  2. Ranjan Bose, “ITC and Cryptography”, Tata McGraw-Hill.
  3. Roberto Togneri, Christopher J.S deSilva, “Fundamentals of Information Theory and Coding Design”, CRC Press.

Official AKTU syllabus

This subject is on page 9 of the original AKTU PDF.

Official AKTU syllabus PDF ↗