Introduction

Hidden Markov Models (HMMs) are a powerful statistical tool used to model systems that possess a degree of randomness governed by unobservable states. Traditionally, HMMs are used with a single observable sequence, but there is a growing interest in extending them to handle multiple observed variables. This article delves into the realm of HMMs for multiple observed variables, providing technical insights, examples, and a comprehensive summary table.

Understanding Hidden Markov Models

A Hidden Markov Model is defined by:

1. States: A finite set of states, each of which might produce an observation.
2. Observations: A sequence of observed events associated with the states.
3. Transition Model: Probabilities determining the transitions between states.
4. Emission Model: Probabilities that link each state to a possible observation.
5. Initial Distribution: Probabilities assigned to each state at the initial time.

Mathematical Formulation

For an HMM with $N$ states $S = \{ S_1, S_2, ..., S_N \}$ and $M$ possible observations $V = { v_1, v_2, ..., v_M }$, it is described by:

• Transition Probability Matrix $A$: $A = { a_{ij} }$, with $a_{ij} = P(X_{t+1} = S_j | X_t = S_i)$. • Emission Probability Matrix $B$: $B = { b_j(v_k) }$, with $b_j(v_k) = P(O_t = v_k | X_t = S_j)$. • Initial State Distribution $\pi$: $\pi = { \pi_i }$, where $\pi_i = P(X_1 = S_i)$.

Extending HMMs to Multiple Observed Variables

In many real-world applications, systems may be influenced by several sources of observable data. Extending HMMs to multiple observable variables involves linking multiple observation sequences to the same underlying state structure. This can be particularly useful in domains like multi-channel signal processing, healthcare, and financial analysis.

Structure of Multi-observed HMMs

When dealing with multiple observed variables, let's say $K$ observation sequences $O^{(1)}, O^{(2)}, ..., O^{(K)}$:

• Emission Model Extension: $B_k$, with $b^{(k)}_j(v^{(k)}_m) = P(O^{(k)}_t = v^{(k)}_m | X_t = S_j)$ for each observation sequence $k$.

Example Application: Human Activity Recognition

Consider an application in human activity recognition using wearable sensors that capture data from accelerometers, gyroscopes, and heart rate monitors:

• States: Different activities such as walking, running, sitting. • Observations: Sensor data from multiple devices, e.g., $O^{(1)}$ from accelerometer, $O^{(2)}$ from gyroscope, and $O^{(3)}$ from heart rate monitor. • Each observation sequence provides a partial view of the underlying activity, allowing for more accurate and robust inference of the current state.

Key Algorithms

1. Forward Algorithm: Computes the probability of the observation sequences up to time $t$ given the model.
2. Backward Algorithm: Computes the probability of the ending segment of observation sequences from time $t+1$ to the end.
3. Baum-Welch Algorithm: An Expectation-Maximization (EM) approach to optimize the model parameters given the observations.
4. Viterbi Algorithm: Determines the most probable sequence of states given the observed sequences.

Handling Multiple Observation Sequences

For $K$ sequences, the forward probability $\alpha_t(i)$ and the backward probability $\beta_t(i)$ need to account for all sequences:

• Forward: $\alpha_t(i) = \sum_{i=1}^{N} \alpha_{t-1}(i) a_{ij} \prod_{k=1}^{K} b^{(k)}_j(O^{(k)}_t)$ • Backward: $\beta_t(i) = \sum_{j=1}^{N} a_{ij} \prod_{k=1}^{K} b^{(k)}_j(O^{(k)}_{t+1}) \beta_{t+1}(j)$

Summary Table

Conclusion

The extension of Hidden Markov Models to handle multiple observed variables opens up new possibilities for complex data analysis and pattern recognition. By integrating multiple data streams into a single probabilistic framework, these models can achieve more accurate and robust state inferences which are crucial for advanced applications ranging from healthcare to finance. Written with mathematical rigor and practical examples, this article provides a foundational understanding to pursue further studies or applications in multi-observed HMMs.