Hidden Markov models (HMMs)
A hidden Markov model infers a sequence of states you cannot see from a sequence of observations you can, using the odds of each state following another and of each state producing each observation.
A hidden Markov model, or HMM, is a probability model for a sequence in which the states you care about are hidden. It builds on a Markov chain, which assumes that only the current state matters for predicting the next one. In an HMM you never see the states themselves. Each hidden state produces an observation with some probability, called its emission probability. The model is fixed by three sets of numbers: start probabilities, transition probabilities between hidden states, and emission probabilities.
An influential 1989 tutorial by Rabiner framed HMMs around three problems. Likelihood asks how probable a sequence of observations is. The forward algorithm answers it by summing over every hidden path, folded into one table. Decoding asks which hidden sequence best explains the observations, and the Viterbi algorithm answers it. Learning asks what the probabilities should be, given only observations. The Baum–Welch algorithm, an iterative form of expectation-maximization, answers it. Each round uses the current estimates to produce better ones. Real implementations add logarithms instead of multiplying probabilities, because long products get too small for a computer to store.
The model grew out of work by Baum and colleagues at the Institute for Defense Analyses in Princeton in the 1960s. Around 1972, two laboratories independently applied HMMs to speech. Paired with Gaussian mixture models for the sounds, HMMs became the dominant approach to speech recognition by the 1990s. By 2012, hybrid systems that fed deep neural network outputs into an HMM had beaten the classic HMM and Gaussian mixture systems.
In language processing, an HMM tagger labels each word with a part of speech such as noun or verb. On English test sets, HMM taggers, CRF taggers and BERT all reach about 97% accuracy. HMMs recur throughout computational biology. Pfam describes each protein family with a profile HMM. The HMMER software uses such profiles to search sequence databases for related sequences.
In many sequences, the thing you want to know is never written down.
Decode three days of ice-cream counts into hot and cold days.
- 1 · startFor the first observation, multiply each state's start probability by the chance that state produces it.
- 2 · extendFor each later cell, multiply every earlier cell by the transition probability into this state.
- 3 · keepKeep only the largest of those products, remember which earlier state it came from, then multiply by the emission probability.
- 4 · backtraceAt the end, start from the best final cell and follow the remembered states back to the beginning.
Viterbi is the forward algorithm with max in place of sum, plus pointers that remember the winning path.
| Who | What they ask | What it works with |
|---|---|---|
| Language researcher | “Is book a noun or a verb in this sentence?” | The words of a sentence, decoded into a sequence of part-of-speech tags |
| Protein biologist | “Which known protein family does this new sequence belong to?” | An amino-acid sequence, scored against profile HMMs such as those in Pfam |
| Speech engineer in the 1990s | “Which words were spoken in this recording?” | Acoustic features, decoded by HMMs with Gaussian mixture models as the sound component |
- Finds the single most likely hidden sequence without scoring every possible one.
- Computes how likely a whole observation sequence is, in time that grows with the number of states squared times the length.
- Learns transition and emission probabilities from unlabelled sequences with the Baum–Welch algorithm.
- Profile HMMs detect distant relatives of a protein sequence in large databases.
- It assumes each state depends only on the one before, and each observation only on its own state.
- Adding extra clues, such as capital letters or word endings, is hard to do cleanly in an HMM.
- Baum–Welch training can get stuck in a local optimum, so it is usually run from several starting points.
- Top part-of-speech taggers now use neural networks such as bidirectional RNNs or Transformers.
Sources used
This explainer is written in original language. The links below support its factual claims.
- paperSpeech and Language Processing (3rd ed. draft), Appendix A: Hidden Markov Models, Jurafsky and Martin, Stanford University · read 27 Sept 2026
- paperSpeech and Language Processing (3rd ed. draft), Chapter 18: Sequence Labeling for Parts of Speech and Named Entities, Jurafsky and Martin, Stanford University · read 27 Sept 2026
- paperSpeech and Language Processing (3rd ed. draft), Chapter 16: Automatic Speech Recognition, Jurafsky and Martin, Stanford University · read 27 Sept 2026
- docsTutorial (hmmlearn documentation), hmmlearn · read 27 Sept 2026
- officialHMMER: biosequence analysis using profile hidden Markov models, HMMER · read 27 Sept 2026
- docsWelcome to Pfam's documentation, Pfam · read 27 Sept 2026
- paperAccelerated Profile HMM Searches, Eddy, PLOS Computational Biology, 2011 · read 27 Sept 2026
- paperWhat is a hidden Markov model?, Eddy, Nature Biotechnology, 2004 · read 27 Sept 2026