MATLAB Implementation for Calculating Shannon Information Entropy

Calculating Shannon information entropy is a fundamental concept in information theory, used to quantify the uncertainty or information content of random variables. Implementing Shannon entropy calculation in MATLAB is straightforward, with the core approach involving logarithmic operations and weighted summation based on probability distributions.

The Shannon entropy formula is defined as: H = -Σ(p_i * log2(p_i)), where p_i represents the probability of each event occurrence. In MATLAB, this can be implemented through the following key steps:

Probability Calculation: Input data can be either raw data or direct probability distributions. For raw data inputs, first use the `unique` function to count frequency occurrences and convert them to probabilities using element-wise division by the total count. Zero Probability Handling: To avoid log2(0) situations (which would return -Inf), implement conditional checks to exclude zero probability terms or apply minimal adjustments using epsilon values. Entropy Calculation: Apply element-wise log2 operations using `log2(p_i)` on non-zero probabilities, perform weighted summation with vector multiplication (`.*` operator), and finally apply negation to obtain the entropy value.

MATLAB's vectorization capabilities enable efficient implementation, particularly when combining the `unique` function for frequency counting with the `sum` function for weighted summation. The built-in `log2` function handles logarithmic computations directly.

This implementation can be encapsulated into a reusable function that accepts vector or matrix inputs, with optional parameters for data dimension handling and zero-probability treatment. The function should include input validation using `isnumeric` checks and probability normalization to ensure sum(p_i) = 1. This approach not only works for simple discrete probability distributions but can be extended to more complex data analysis scenarios through appropriate dimension parameterization.