Markov Models

[NicolaBernini]

Markov Models

Definition

Let's consider a stochastic process $S_{t}$ : a time indexed list of random variables related to some domain

This may represent a dynamic system which evolves according to some non deterministic laws

What model is good for this kind of systems?

An interesting class of models is represenrted by the Markov Models class which indentifies a set of models characterized by the Markov Property Assumption which consists of assuming the dynamic does not depend on the past history but only on the current state so it is said the process is without memory

$$ P(S_{t+\Delta t} | {S_{\tau}}) = P(S_{t+\Delta t} | S_{t}) \quad \tau \in [1,t] $$

Types

The following types of Markov Models exist

Courtesy of Wikipedia

Markov Chain

The Markov Chain is the simplest type of Markov Model, consisting of the following Category (in the sense of Category Theory):

• $\mathcal{S}$ : Objects Space or State Space which contains all the possible states for the system $S \in \mathcal{S}$
• $P(S_{t + \Delta t} | S_{t})$ : Stochastic Morphism Space defininig a probabilistic connction between 2 elements in the State Space

Hidden Markov Model

The Hidden Markov Model introduces a decoupling between the State Space or Latent Space and Observation Space

The HMM backbone is a Markov Chain defining an evolution in the Latent Space, but the latent - observable space decoupling, introduces 2 additional elements

• $\mathcal{Z}$ : Observable Space so that $Z \in \mathcal{Z}$
• $P(Z|S)$ : Likelihood (according to Bayesian Framework terminology) which acts as Pseudo-Functor or Cross Category Mapping as it defines a probabilistic relationship between the Latent Space (belonging to State Category) and the Observable Space (belonging to Observation Category)

In order to estimate Latent State from Observations an Inference needs to be performed and the following types are possible

• Filtering : $P(S_{t} | \{Z_{\tau}\}) \quad \tau \in [t_{0},t]$ which consists of estimating the present hence only the most recent state $S_{y}$ from the available knowledge
• Smoothing : $P(\{S_{\tau}\}_{\tau \in [t_{1},t] \quad t_{1} \ge t_{0}} | \{Z_{\tau}\}_{\tau \in [t_{0},t]})$ which consists of estimating the past hence last states $\{S_{\tau}\}$ in the recent past $[t_{1}, t]$ from the available knowledge
• Prediction : $P(\{S_{\tau}\}_{\tau \in [t, t_{1}] \quad t_{1} > t} | \{Z_{\tau}\}_{\tau \in [t_{0},t]})$ which is focused on estimating the future hence the states $\{S_{\tau}\}$ in some future timeframe $[t, t_{1}]$ from the available knowledge