Information Theory

This quiz covers the fundamental concepts and principles of Information Theory, including entropy, mutual information, channel capacity, and coding theorems.

15 Questions Published

Play Quiz Practise this topic out loud

Questions

Question 1 Multiple Choice (Single Answer)

What is the unit of information in Information Theory?

Bit
Byte
Hertz
Decibel

Question 2 Multiple Choice (Single Answer)

The entropy of a random variable $X$ is defined as:

$H(X) = \sum_x p(x) \log p(x)$
$H(X) = \sum_x p(x) \log_2 p(x)$
$H(X) = \sum_x p(x)^2 \log p(x)$
$H(X) = \sum_x p(x)^2 \log_2 p(x)$

Question 3 Multiple Choice (Single Answer)

The mutual information between two random variables $X$ and $Y$ is defined as:

$I(X;Y) = H(X) + H(Y)$
$I(X;Y) = H(X) - H(Y)$
$I(X;Y) = H(X,Y) - H(X)$
$I(X;Y) = H(X,Y) - H(X) - H(Y)$

Question 4 Multiple Choice (Single Answer)

The channel capacity of a communication channel is defined as:

$C = \max_{p(x)} I(X;Y)$
$C = \max_{p(x)} H(X)$
$C = \max_{p(x)} H(Y)$
$C = \max_{p(x)} H(X,Y)$

Question 5 Multiple Choice (Single Answer)

The Shannon-Hartley theorem states that the channel capacity of a band-limited additive white Gaussian noise (AWGN) channel is given by:

$C = B \log_2 (1 + \frac{S}{N})$
$C = B \log_2 (1 + \frac{N}{S})$
$C = B \log_2 (1 + \frac{S}{N^2})$
$C = B \log_2 (1 + \frac{N^2}{S})$

Question 6 Multiple Choice (Single Answer)

The source coding theorem states that the minimum number of bits required to represent a source with entropy $H$ is:

$R = H$
$R = H + 1$
$R = H - 1$
$R = 2H$

Question 7 Multiple Choice (Single Answer)

The channel coding theorem states that it is possible to achieve reliable communication over a noisy channel with a capacity of $C$ by using a code with a rate:

$R < C$
$R = C$
$R > C$
$R \ge C$

Question 8 Multiple Choice (Single Answer)

The Huffman coding algorithm is a:

Prefix-free code
Variable-length code
Fixed-length code
Non-unique code

Question 9 Multiple Choice (Single Answer)

The Lempel-Ziv-Welch (LZW) algorithm is a:

Lossless data compression algorithm
Lossy data compression algorithm
Huffman coding algorithm
Arithmetic coding algorithm

Question 10 Multiple Choice (Single Answer)

The JPEG image compression standard uses:

Discrete cosine transform (DCT)
Discrete Fourier transform (DFT)
Walsh-Hadamard transform (WHT)
Haar wavelet transform

Question 11 Multiple Choice (Single Answer)

The MP3 audio compression standard uses:

Perceptual audio coding (PAC)
Linear predictive coding (LPC)
Adaptive differential pulse-code modulation (ADPCM)
Transform coding

Question 12 Multiple Choice (Single Answer)

The H.264 video compression standard uses:

Block-based motion compensation
Discrete cosine transform (DCT)
Quantization
Entropy coding

Question 13 Multiple Choice (Single Answer)

The information content of a message is measured in:

Bits
Bytes
Hertz
Decibels

Question 14 Multiple Choice (Single Answer)

The rate of a source code is defined as:

The number of bits required to represent a single source symbol
The number of bits required to represent a block of source symbols
The average number of bits required to represent a source symbol
The maximum number of bits required to represent a source symbol

Question 15 Multiple Choice (Single Answer)

The efficiency of a source code is defined as:

The ratio of the entropy of the source to the rate of the code
The ratio of the rate of the code to the entropy of the source
The ratio of the average length of a codeword to the entropy of the source
The ratio of the entropy of the source to the average length of a codeword

← Back to Quizzes