While text classification assigns one label per document, sequence labeling assigns one label per token. Given a sentence, predict a tag for every word. The three core sequence labeling tasks are: Part-of-Speech (POS) tagging — is each word a noun, verb, adjective, etc.? Named Entity Recognition (NER) — which words are person names, organizations, locations, dates? Chunking — which words form noun phrases, verb phrases, etc.? Sequence labeling is the backbone of information extraction. Every time a search engine highlights a person's name, a chatbot extracts a date from your message, or a medical system identifies drug names in clinical notes, sequence labeling is at work. The key challenge: labels depend on neighboring labels. In "New York City", "York" is part of a location only because "New" precedes it.

Multi-word entities like "New York City" need a way to mark where entities begin and continue. The IOB format (Inside-Outside-Beginning) solves this: B-TYPE marks the first token of an entity, I-TYPE marks continuation tokens, and O marks tokens outside any entity. So "New York City" is tagged B-LOC I-LOC I-LOC. Without IOB, you can't distinguish "New York" (one entity) from "New" and "York" (two separate entities). The extended BIOES format adds S (single-token entity) and E (end of entity) for finer boundaries. IOB-2 (the most common variant) requires every entity to start with B, even if it's not adjacent to another entity of the same type. This format is the universal standard for NER annotation and evaluation.

Hidden Markov Models were the dominant approach to POS tagging and NER before neural methods. An HMM models two probability distributions: transition probabilities — the probability of one tag following another (P(NOUN | DET) is high, P(DET | DET) is low), and emission probabilities — the probability of a word given a tag (P("cat" | NOUN) is high, P("cat" | VERB) is low). The "hidden" states are the tags (unobserved), and the "observations" are the words. The Viterbi algorithm efficiently finds the most likely tag sequence by dynamic programming, avoiding the exponential cost of checking all possible sequences. HMMs dominated POS tagging for decades, achieving ~96% accuracy. Their limitation: they only look at the current word and previous tag, missing longer-range context.

Conditional Random Fields (Lafferty et al., 2001) improved on HMMs by modeling P(tags | words) directly (discriminative) rather than modeling the joint P(tags, words) (generative). This seemingly small change has a huge practical impact: CRFs can use arbitrary overlapping features without worrying about independence assumptions. You can include features like "is the word capitalized?", "does it end in -ing?", "is the previous word 'the'?", "does it appear in a gazetteer?" all simultaneously. CRFs also model the entire tag sequence jointly, not just pairwise transitions. They became the standard for NER, achieving F1 scores of 85–90% on the CoNLL-2003 benchmark. CRFs remain important today as the output layer in neural sequence labeling models (BiLSTM-CRF).

The BiLSTM-CRF (Huang et al., 2015) combines the best of neural and structured prediction. A bidirectional LSTM reads the sentence in both directions, producing context-rich representations for each token that capture both left and right context. These representations are fed into a CRF layer that models tag dependencies and produces the globally optimal tag sequence. The BiLSTM replaces hand-crafted features with learned representations, while the CRF ensures valid tag sequences. Adding character-level CNNs (BiLSTM-CNN-CRF) captures morphological patterns like capitalization and suffixes, achieving 91.18% F1 on CoNLL-2003 NER without any hand-crafted features. This architecture was the state of the art from 2015 to 2018, when BERT-based models surpassed it.

BERT-based NER fine-tunes a pre-trained transformer for token classification. Each token's contextual representation from BERT is passed through a linear classification layer that predicts the IOB tag. Some implementations add a CRF layer on top for sequence coherence. BERT's advantage is its deep bidirectional context — it considers the entire sentence when representing each token, capturing long-range dependencies that BiLSTMs struggle with. On CoNLL-2003, BERT-based models achieve 92–93% F1, surpassing BiLSTM-CRF. For domain-specific NER (biomedical, legal, financial), domain-adapted models like BioBERT, LegalBERT, and FinBERT further improve performance by pre-training on domain text. The trade-off: BERT models are 10–100x more expensive to train and run than BiLSTM-CRF.

Evaluating sequence labeling requires care. Token-level accuracy is misleading because most tokens are O (outside any entity) — a model that tags everything as O gets 80%+ accuracy on typical NER data. The standard metric is entity-level F1: an entity is correct only if both the span boundaries and the entity type match exactly. If the gold standard says "New York City" is B-LOC I-LOC I-LOC, predicting B-LOC I-LOC O is completely wrong at the entity level, even though 2 of 3 tokens are correct. Precision measures what fraction of predicted entities are correct. Recall measures what fraction of gold entities are found. F1 is their harmonic mean. The seqeval library is the standard tool for computing these metrics.

Sequence labeling powers critical production systems across industries. Search engines use NER to identify entities in queries ("flights to Paris" → LOC:Paris) for structured search. Healthcare extracts drug names, dosages, symptoms, and conditions from clinical notes for electronic health records. Finance identifies company names, monetary amounts, and dates in SEC filings and earnings calls. Legal extracts parties, dates, clauses, and obligations from contracts. Customer support identifies product names, issue types, and account numbers from tickets. The challenge in production is domain adaptation: a model trained on news text (CoNLL-2003) performs poorly on medical text because the entity types and vocabulary are completely different. Domain-specific training data and pre-trained models are essential for production NER.