The unit of information in Information Theory is the bit, which represents the basic unit of information content.

The entropy of a random variable $X$ is defined as $H(X) = \sum_x p(x) \log_2 p(x)$, where $p(x)$ is the probability of occurrence of each possible value of $X$.

The mutual information between two random variables $X$ and $Y$ is defined as $I(X;Y) = H(X,Y) - H(X) - H(Y)$, where $H(X,Y)$ is the joint entropy of $X$ and $Y$, $H(X)$ is the entropy of $X$, and $H(Y)$ is the entropy of $Y$.

The channel capacity of a communication channel is defined as $C = \max_{p(x)} I(X;Y)$, where $I(X;Y)$ is the mutual information between the input and output of the channel, and $p(x)$ is the probability distribution of the input.

The Shannon-Hartley theorem states that the channel capacity of a band-limited AWGN channel is given by $C = B \log_2 (1 + \frac{S}{N})$, where $B$ is the bandwidth of the channel, $S$ is the average signal power, and $N$ is the average noise power.

The source coding theorem states that the minimum number of bits required to represent a source with entropy $H$ is $R = H$, where $R$ is the rate of the source code.

The channel coding theorem states that it is possible to achieve reliable communication over a noisy channel with a capacity of $C$ by using a code with a rate $R < C$.

The Huffman coding algorithm is a prefix-free code, which means that no codeword is a prefix of any other codeword.

The LZW algorithm is a lossless data compression algorithm, which means that it can be used to compress data without losing any information.

The JPEG image compression standard uses the discrete cosine transform (DCT) to convert the image into a frequency domain representation, which is then quantized and compressed.

The MP3 audio compression standard uses perceptual audio coding (PAC), which exploits the psychoacoustic properties of the human ear to remove inaudible information from the audio signal.

The H.264 video compression standard uses a combination of block-based motion compensation, discrete cosine transform (DCT), quantization, and entropy coding to achieve high compression ratios.

The information content of a message is measured in bits, which represent the basic unit of information.

The rate of a source code is defined as the average number of bits required to represent a source symbol.

The efficiency of a source code is defined as the ratio of the entropy of the source to the rate of the code.