Lecture 5 — Markov Chains & Random Walks → Monte Carlo Sampling

Course 2-INF-135/15 Pravdepodobnostné algoritmy, LS 2025/26. Source slides: 05_MC.pdf (46 pages). Štátnicové syllabus topics covered here: Markov chains, transition matrix and stationary distribution; connection to eigenvalues; mixing (variation distance), strong uniform stopping times; sampling via random walks (random graphs with prescribed degrees, random knapsack solutions); reversible chains and the Metropolis–Hastings algorithm (how to build a chain with a given stationary distribution).

The one-paragraph map of the whole lecture

A Markov chain is random motion with no memory: where you go next depends only on where you are now, not on how you got there. The deep fact is that a “nice” chain forgets its starting point — run it long enough and the probability of being in state $i$ converges to a single number $\pi_i$ that does not depend on where you began. That number is the stationary distribution.

This turns into a powerful algorithmic machine. Suppose you want to sample an object from some weird, exponentially large space $S$ according to a prescribed distribution $\sigma$ (a uniform independent set, a random knapsack solution, a random shuffle of a deck). You cannot list $S$. But if you can design a chain that walks around $S$ and whose stationary distribution is exactly $\sigma$, then you just run the walk and read off where you land. Two questions then dominate the entire lecture:

1. How do I BUILD a chain with a prescribed stationary distribution $\sigma$? → the idea of reversibility / detailed balance, culminating in the Metropolis–Hastings algorithm.
2. How LONG must I run it? → mixing time, measured in variation distance, and bounded three ways: coupling, strong uniform stopping times, and eigenvalues (expanders mix fast).

The closing payoff: sampling lets you count (estimate the size of huge sets — FPRAS, #DNF), and as a bonus, expander walks recycle random bits to amplify BPP using $O(k)$ bits instead of $O(kn)$.

1. What a Markov chain is

A stochastic process is just a family of random variables indexed by time, $X = \{X_t \mid t \in T\}$, where $X_t$ is “the state at time $t$.” We assume discrete time ($t = 0, 1, 2, \dots$) and a discrete state space $\{0, 1, 2, \dots\}$ (finite or countable).

The process is a Markov chain if it has the memorylessness (Markov) property:

Intuition. The present screens off the past from the future. The only thing the chain “knows” is its current state; the entire history before that is irrelevant to what happens next. Equivalently, for times $u < t < v$, the past ($X_u$) and the future ($X_v$) are conditionally independent given the present ($X_t$): $\Pr[X_u=i, X_v=k \mid X_t=j] = \Pr[X_u=i\mid X_t=j]\cdot\Pr[X_v=k\mid X_t=j].$

The transition matrix

All the dynamics live in a single matrix. Let

be the probability of stepping from $i$ to $j$ in one move (assumed time-homogeneous — the same at every $t$). Collect them into the transition matrix $P = (p_{i,j})$. Each row sums to 1 (from $i$ you must go somewhere):

A matrix with non-negative entries and rows summing to 1 is called stochastic — and every such matrix is the transition matrix of some chain.

How distributions evolve — it’s just matrix multiplication

Let $p_i(t) = \Pr[\text{system is in state } i \text{ at time } t]$ and collect these into a row vector $\bar p(t) = (p_0(t), p_1(t), \dots)$, the distribution over states at time $t$. To be in state $i$ at time $t$, you were in some state $j$ at time $t-1$ and stepped $j \to i$:

Iterating, the $m$-step transition matrix is just the matrix power $P^{(m)} = P^m$, and

Punchline. A Markov chain is a stochastic matrix. Running the chain = repeatedly multiplying a distribution vector by $P$. Everything that follows — convergence, stationary distributions, mixing rates — is secretly a statement about the powers $P^m$ and the eigenvalues of $P$. Hold that thought; it returns in §11 (expanders).

2. First examples — random walks that solve problems (2SAT, 3SAT)

Before the theory, two examples show why we care: a random walk on assignments solves satisfiability, and the chain analysis gives the running time.

2SAT — the symmetric walk, $O(n^2)$

Algorithm (Papadimitriou). Start from any truth assignment $\alpha_0$. While the formula is unsatisfied, pick an unsatisfied clause and flip a uniformly random one of its two literals. Repeat.

Why does this make progress? Fix a satisfying assignment $\alpha^*$ (assume one exists). An unsatisfied 2-clause has both its literals wrong under the current assignment, but $\alpha^*$ satisfies the clause, so $\alpha^*$ disagrees with us on at least one of those two variables. Flipping a random one of the two therefore moves us closer to $\alpha^*$ with probability $\ge \tfrac12$.

Track $j$ = the number of variables on which we currently agree with $\alpha^*$ (Hamming distance is $n - j$). This is (a lower bound on) a chain on $\{0,1,\dots,n\}$:

• at $j=n$ we have found a satisfying assignment: $h_n = 0$;
• at $j=0$ any flip increases agreement: $h_0 = 1 + h_1$;
• in between, with prob $\tfrac12$ we go to $j+1$, with prob $\tfrac12$ to $j-1$:
  $$h_j = 1 + \tfrac12 h_{j-1} + \tfrac12 h_{j+1}.$$

Here h_j = $ expected number of steps to reach $\alpha^* starting from agreement $j$. Solving the recurrence (it telescopes to $h_j = h_{j+1} + (2j+1)$) gives

so the symmetric walk finds a satisfying assignment in expected $O(n^2)$ steps. (This is the classic gambler’s-ruin / drift argument: a fair walk on a line of length $n$ takes $\Theta(n^2)$ time to cross it.)

3SAT — the biased walk, and why restarts win

For 3SAT an unsatisfied 3-clause has three literals; $\alpha^*$ disagrees on at least one, so a random flip moves us toward $\alpha^*$ with probability only $\tfrac13$ and away with probability $\tfrac23$. The recurrence becomes biased:

A walk biased away from the target takes exponential time: $h_0 = O(2^n)$. Running one long walk is hopeless.

The fix (Schöning). Don’t run one long walk — run many short walks from fresh random starts. The key insight on the slide: “if a satisfying $\alpha$ exists, then there are many” assignments from which a short walk reaches it. A random start lands within a favourable distance often enough that capping each walk at $O(n)$ steps and restarting gives

$$h_0 = O\!\left(n^{3/2}\,(4/3)^n\right),$$

dramatically better than $2^n$. (This is the random-walk k-SAT algorithm previewed in the derandomization lecture — derandomized there by sweeping a covering code instead of guessing the start.)

Professor-pleaser. The same random-flip rule is polynomial for 2SAT and exponential for 3SAT, and the only thing that changed is the drift of the induced walk on Hamming distance: fair coin ($\tfrac12/\tfrac12$) → $O(n^2)$; biased coin ($\tfrac13/\tfrac23$) → $2^n$. The behaviour of the algorithm is entirely the behaviour of the underlying Markov chain. This is the whole reason to study chains.

3. Classifying states — which chains “settle down”?

For a chain to forget its start and converge, it must be structurally well-behaved. The vocabulary (this is the part to be able to recite in the oral):

• Reachable ($j$ reachable from $i$): $p^{(t)}_{i,j} > 0$ for some $t$. Communicating states: each reachable from the other.
• Irreducible chain: every state communicates with every other — i.e. the directed graph of the chain is a single strongly connected component. (You can get from anywhere to anywhere.)
• Recurrent / persistent state $i$: you return to it with certainty. Let $r^{(t)}_{i,j} = \Pr[\text{first visit to } j \text{ from } i \text{ is at time } t]$. Then $i$ is recurrent iff
  $$f_{i,i} = \sum_{t \ge 1} r^{(t)}_{i,i} = 1.$$
• Transient (prechodový) state: $f_{i,i} = \sum_t r^{(t)}_{i,i} < 1$ — there is positive probability you never come back.
• Absorbing state: once entered, never left ($p_{i,i}=1$).
• A recurrent chain has all states recurrent.

Among recurrent states there is a subtle but exam-relevant split. The mean return time is

• Positive recurrent: $h_{i,i} < \infty$ (you come back, and in finite expected time).
• Null recurrent: $h_{i,i} = \infty$ (you come back with probability 1, but the expected wait is infinite!).

The null-recurrent example (worth knowing — it’s a classic trap). Consider a chain on $\{1,2,3,\dots\}$ where from $i$ you go up to $i+1$ with probability $\frac{i}{i+1}$ and home to $1$ with probability $\frac{1}{i+1}$. The probability of first returning to $1$ at exactly time $t$ is

$$r^{(t)}_{1,1} = \underbrace{\tfrac12\cdot\tfrac23\cdots\tfrac{t-1}{t}}_{\text{climb }1\to t}\cdot \underbrace{\tfrac{1}{t+1}}_{t\to 1} = \frac1t\cdot\frac1{t+1} = \frac{1}{t(t+1)}.$$

Then $f_{1,1} = \sum_t \frac1{t(t+1)} = 1$ (telescopes), so state 1 is recurrent — but

$$h_{1,1} = \sum_t t\cdot\frac{1}{t(t+1)} = \sum_t \frac{1}{t+1} = \infty.$$

Null recurrent. You always come home, but the average homecoming time is infinite. This cannot happen in a finite chain (next lemma).

Two equivalent tests for recurrence

A clean characterisation via the $m$-step return probabilities:

Periodicity and ergodicity

• Period of state $i$: $\gcd\{t : p^{(t)}_{i,i} > 0\}$ — the gcd of all times at which return is possible. Aperiodic = period 1.

  Quick check the slide asks: a random walk on an undirected cycle has period 2 if the length is even (you can only come back after an even number of steps) and period 1 if the length is odd (odd cycles let you “turn around”). Bipartiteness = period 2. This is exactly why we add self-loops to kill periodicity later.

• Ergodic state: aperiodic and positive recurrent — “you come back, in finite expected time, but at irregular times.” An ergodic chain has all states ergodic.

Lemma (finite chains are well-behaved). In a finite Markov chain:

1. there is at least one recurrent state;
2. all recurrent states are positive recurrent (no null recurrence in finite chains).

Stationary distribution

A distribution $\bar\pi$ is stationary if it is a fixed point of the dynamics:

4. The Fundamental Theorem of Markov Chains

This is the theorem of the lecture — the precise statement that “nice chains forget their start.”

Theorem (Fundamental / hlavná veta). In an irreducible, finite, aperiodic Markov chain:

1. every state is ergodic;
2. there exists a unique stationary distribution $\pi$;
3. $f_{i,i} = 1$ and the mean return time is $h_{i,i} = \dfrac{1}{\pi_i}$;
4. the long-run fraction of time spent in $i$ converges to $\pi_i$: $\displaystyle \lim_{t\to\infty}\frac{N(i,t)}{t} = \pi_i$, where $N(i,t)$ counts visits to $i$ in $t$ steps.

Equivalently (the “convergence” form): 3′. $\displaystyle \lim_{t\to\infty} P^t_{j,i}$ exists and is independent of the starting state $j$; 4′. $\displaystyle \pi_i = \lim_{t\to\infty} P^t_{j,i} = \frac{1}{h_{i,i}}.$

Read parts 3′–4′ slowly — they are the magic:

Intuition. $P^t_{j,i}$ is the probability of being at $i$ after $t$ steps given you started at $j$. The theorem says this stops depending on $j$. The chain erases the memory of its origin. Where you end up is governed only by the chain’s structure, summarised in $\pi$. And $\pi_i = 1/h_{i,i}$ has a lovely reading: a state you return to quickly (small mean return time) is a state you spend a lot of time in (large stationary probability). Frequently visited ⇔ short return time.

The three hypotheses each pull their weight:

• irreducible → the limit can’t depend on $j$ (otherwise disconnected pieces keep separate fates);
• aperiodic → the limit exists (a period-2 chain oscillates forever and never settles — recall the even cycle);
• finite → positive recurrence is automatic (no null-recurrent escape to infinity).

5. Computing $\pi$ — cuts and reversibility (the load-bearing trick)

How do you actually find $\pi$? Solve $\pi P = \pi$ with $\sum_i \pi_i = 1$. Written out component-wise, the equation $\pi_i = \sum_j \pi_j P_{j,i}$ is a balance / cut condition:

The left side is flow into $i$; the right side is flow out of $i$. Stationarity = global balance: total probability flowing into each state equals total flowing out. Solving the full linear system is doable but painful. There is a much stronger, much easier-to-check sufficient condition.

Theorem (detailed balance ⇒ stationary). Let the chain be finite, irreducible, ergodic with matrix $P$, and let $\pi = (\pi_1,\dots,\pi_n)$ be non-negative with

1. $\sum_i \pi_i = 1$, and
2. $\;\pi_i P_{i,j} = \pi_j P_{j,i}\;$ for every pair $i,j$ (the detailed balance / reversibility condition).

Then $\pi$ is the (unique) stationary distribution, and the chain is time-reversible.

Proof — one line. Sum detailed balance over $i$:

Why this is the whole game. Global balance (the true stationarity equation) couples all states together — a big linear system. Detailed balance is a local, pairwise condition: just check each edge $i\!\leftrightarrow\! j$ in isolation. It is strictly stronger (not every chain is reversible), but when it holds it is trivial to verify, and — crucially — it is the lever we use to engineer a chain with a prescribed $\pi$. “Reversible” means the chain looks statistically the same run forwards or backwards in time. Remember this condition; the entire Metropolis construction in §8–§9 is just “rig the transition probabilities so detailed balance holds for the $\pi$ I want.”

6. Random walks on undirected graphs

The most important concrete chain. Given a connected undirected graph $G=(V,E)$, the simple random walk steps from $v$ to a uniformly random neighbour:

(Aperiodic $\iff$ $G$ is not bipartite — same parity-of-cycles fact as before.)

Theorem. The simple random walk on a connected non-bipartite graph converges to the stationary distribution

$$\boxed{\;\pi_v = \frac{\deg(v)}{2|E|}\;}$$

— proportional to degree. High-degree vertices are visited more.

Check. It sums to 1: $\sum_v \frac{\deg(v)}{2|E|} = \frac{2|E|}{2|E|} = 1$ (handshake lemma). And it satisfies detailed balance — the cleanest possible verification:

By the Fundamental Theorem, the mean return time to $u$ is

Hitting times and cover time — why a walk explores fast

Hitting time on an edge. If $(u,v)\in E$, then the expected time to go from $v$ to $u$ is bounded:

Cover time = expected time to visit all vertices, from the worst start.

Theorem. $\text{cover time} < 4|V|\,|E|.$

Proof idea (elegant). Fix a spanning tree. Walk the tree in a fixed closed tour (e.g. a DFS traversal, around the tree) that crosses each of its $|V|-1$ edges once in each direction — $2(|V|-1)$ edge-traversals. The expected time for each traversal of an edge $(v_i, v_{i+1})$ is $h_{v_i, v_{i+1}} < 2|E|$. Summing,

Why this matters algorithmically. A random walk explores a connected graph in polynomial time using essentially no memory — you only remember your current vertex. The headline application: undirected $s$–$t$ connectivity (USTCON) in randomized log-space. To decide whether $t$ is reachable from $s$, just walk from $s$ for $O(|V||E|) = \mathrm{poly}$ steps; if you hit $t$, accept. The walk needs only $O(\log n)$ bits (the current vertex + a counter). This is the basis of the result $\textbf{USTCON} \in \mathbf{RL}$ — and famously was later derandomized to $\mathbf{L}$ by Reingold. The cover-time bound is exactly the running-time guarantee.

7. From walks to sampling — the Monte Carlo Markov Chain (MCMC) idea

Here is the conceptual pivot of the lecture.

Problem. Sample an object from a huge space $S$ according to a prescribed distribution $\sigma$. ($S$ is exponential — you cannot enumerate it.)

Solution (MCMC). Design an ergodic Markov chain such that

• its state space is $S$ (the objects you want to sample), and
• its stationary distribution is the desired $\sigma$.

Then run the chain. By the Fundamental Theorem it converges to $\sigma$ from any starting state, so after enough steps the current state is an (almost) $\sigma$-sample.

To get several near-independent samples, run a long walk $x_0, x_1, x_2, \dots$ and take states spaced $r$ apart:

Two quantities control the cost, and they organize the rest of the lecture:

• $r$ = the mixing time — how long until one step “looks like” $\sigma$. (The hard part; §10–§11.)
• the cost of one step — want a small neighbourhood $N(x)$ so each move is cheap.

The design freedom and the two sub-problems. We get to invent the chain. That raises exactly the two questions from the map: (1) how do we force the stationary distribution to be the $\sigma$ we want (uniform? weighted?) — §8–§9; and (2) how do we bound $r$ so we know when to stop — §10–§11.

We assume a symmetric neighbourhood structure $\{N(x)\}$ on $S$: $y \in N(x) \iff x \in N(y)$ (an undirected “move graph” on the objects), and we want $N(x)$ small.

8. Building a chain with uniform stationary distribution

First the easy target: \sigma = $ **uniform** on $S. Naïvely walking the move-graph gives $\pi_v \propto \deg(v)$ (§6) — not uniform, because different objects have different numbers of neighbours. The fix is to equalize the transition probabilities by padding with self-loops.

Lemma (uniform sampler). Let $\Omega$ be a finite state space with neighbourhood structure $\{N(x)\}$, $N = \max_x |N(x)|$. Pick any $M > N$ and define the chain

$$P(x,y) = \begin{cases} 1/M & x\ne y,\ y\in N(x),\\[2pt] 0 & x\ne y,\ y\notin N(x),\\[2pt] 1 - |N(x)|/M & x = y \quad(\text{self-loop}). \end{cases}$$

If this chain is irreducible and aperiodic, its stationary distribution is uniform, $\pi_x = 1/|\Omega|$.

Proof. Every off-diagonal transition probability is the same constant $1/M$, regardless of $x$ or $y$ (the self-loop soaks up the difference in degrees). So $P(x,y)=P(y,x)=1/M$ — the matrix is symmetric. Detailed balance with the uniform $\pi_x = \pi_y = 1/|\Omega|$ reads $\pi_x P(x,y) = \pi_y P(y,x)$, which holds because both $\pi$’s and both $P$’s are equal. By §5, uniform is stationary. ∎

The idea in one phrase. Self-loops are a thermostat for degree. Every vertex “spends” $|N(x)|/M$ of its probability stepping out and parks the remaining $1 - |N(x)|/M$ on itself, so outgoing edges all carry the same weight $1/M$. Equal edge weights ⇒ symmetric matrix ⇒ uniform stationary. We choose $M > N$ strictly so every state has a positive self-loop, which also kills periodicity for free.

Example — sampling a uniform Independent Set

State space \Omega = $ all independent sets of $G. Neighbourhood: $N(x) = $ independent sets differing from $x$ in one vertex. The chain:

Uniform-IS walk.

1. $X_0 \leftarrow$ any independent set.
2. To compute $X_{i+1}$: pick a uniform random vertex $v \in V$.
   • if $v \in X_i$: set $X_{i+1} \leftarrow X_i \setminus \{v\}$ (remove);
   • if $v \notin X_i$ and $X_i \cup \{v\}$ is independent: $X_{i+1} \leftarrow X_i \cup {v}$ (add);
   • otherwise $X_{i+1} \leftarrow X_i$ (stay — this is the self-loop).

Picking $v$ among all $|V|$ vertices, and staying when the move is illegal, is exactly the “$M > N$ with self-loops” construction (here $M = |V|$). As long as $G$ has at least one edge there is always some illegal move from some state, guaranteeing a self-loop, hence aperiodicity. Stationary distribution: uniform over all independent sets.

9. Building a chain with a prescribed non-uniform $\pi$ — Metropolis

Now the general target: we want each object $x$ sampled with a prescribed probability $\pi_x$. Often $\pi$ is given only up to normalization:

Lemma (Metropolis). Given $\{N(x)\}$, $N=\max_x|N(x)|$, $M > N$, and any desired $\pi_x > 0$, define

$$P(x,y) = \begin{cases} \dfrac1M \cdot \min\!\left\{1,\ \dfrac{\pi_y}{\pi_x}\right\} & x\ne y,\ y\in N(x),\\[8pt] 0 & x\ne y,\ y\notin N(x),\\[4pt] 1 - \displaystyle\sum_{y\ne x} P(x,y) & x = y. \end{cases}$$

If irreducible and aperiodic, its stationary distribution is exactly $\pi$.

Proof — detailed balance again. Take $x\ne y$ neighbours with (WLOG) $\pi_x \le \pi_y$. Then $\min\{1, \pi_y/\pi_x\} = 1$, so $P(x,y) = 1/M$; and $\min{1,\pi_x/\pi_y} = \pi_x/\pi_y, so $P(y,x) = \frac1M\cdot\frac{\pi_x}{\pi_y}. Check:

The two beautiful features.

1. Only ratios $\pi_y/\pi_x$ appear. The unknown normalizer $B$ cancels: $\pi_y/\pi_x = b(y)/b(x)$. We can sample from $\pi$ without ever knowing the total weight $B$ — exactly the situation in every real counting/physics application.
2. The acceptance rule is a thermostat toward $\pi$. Propose a uniform neighbour move ($1/M$). Always accept a move to a more-probable state ($\pi_y \ge \pi_x$); accept a move to a less-probable state only with probability $\pi_y/\pi_x$. Detailed balance is satisfied by construction, so the walk pools where $\pi$ is large.

Metropolis for weighted Independent Sets

We want $\Pr[\text{independent set } I] \propto \lambda^{|I|}$ for a parameter $\lambda>0$:

The ratio for adding a vertex is $\pi_y/\pi_x = \lambda^{|I|+1}/\lambda^{|I|} = \lambda$; for removing, $1/\lambda$. So the algorithm needs only $\lambda$:

Metropolis-IS.

1. $X_0 \leftarrow$ some independent set.
2. To compute $X_{i+1}$: pick $v \in V$ uniformly (prob $1/M$, $M=|V|$).
   • if $v \in X_i$: remove it with probability $\min\{1, 1/\lambda\}$;
   • if $v \notin X_i$ and $X_i\cup\{v\}$ independent: add it with probability $\min\{1, \lambda\}$;
   • otherwise stay.

The detailed-balance check ($\pi_x P(x,y) = \pi_y P(y,x)$) goes through with the $\lambda$ ratios, and — the punchline on the slide — “$B$ sme nepotrebovali :-)”: we never needed the normalizer.

Metropolis–Hastings — the general recipe (asymmetric proposals)

What if the proposal mechanism itself is not symmetric? Let $Q$ be any irreducible proposal chain (q_{ij} = $ probability of proposing $j from $i$), and $\pi$ the desired distribution.

Metropolis–Hastings. From $i$, propose $j$ according to $Q$. Accept the move with probability

$$\alpha = \min\!\left\{\,\frac{\pi_j\, q_{ji}}{\pi_i\, q_{ij}},\ 1\right\}$$

(when $Q$ is symmetric, $q_{ij}=q_{ji}$, this collapses to the plain Metropolis $\alpha = \min\{\pi_j/\pi_i,\,1\}$). Otherwise stay at $i$.

The resulting transition $p_{ij} = q_{ij}\,\alpha$ satisfies detailed balance with $\pi$: the correction factor $q_{ji}/q_{ij}$ exactly cancels the asymmetry of the proposal, so $\pi_i p_{ij} = \pi_j p_{ji}$.

Professor-pleaser — the unifying statement of §8–§9. Metropolis–Hastings is a universal machine for manufacturing a Markov chain with any prescribed stationary distribution, using only the ratios of the target probabilities. It works by forcing detailed balance (§5): start from any convenient proposal walk, then bend the acceptance probabilities until $\pi_i p_{ij} = \pi_j p_{ji}$ holds on every edge. Reversibility is not a curiosity — it is the design principle of all of MCMC.

10. How long? — variation distance, mixing time, and coupling

We can build the chain; now we must know when its distribution is close enough to $\pi$. We need (a) a notion of “close,” and (b) tools to bound the time to get there.

Variation distance — the right notion of “close”

For two distributions $D_1, D_2$ on a countable $S$, the total variation distance is

Lemma (the event form). $;\displaystyle |D_1 - D_2| = \max_{A\subseteq S} |D_1(A) - D_2(A)|;$ where $D_i(A) = \sum_{x\in A}D_i(x)$.

Intuition. Variation distance is the largest disagreement on any event: no matter what yes/no question $A$ you ask, the two distributions assign $A$ probabilities differing by at most $\|D_1-D_2\|$. The maximizing $A$ is “all $x$ where $D_1 > D_2$.” The factor $\tfrac12$ makes the range $[0,1]$. This is the operational meaning: if variation distance is $\le \varepsilon$, no statistical test can tell the two apart with advantage more than $\varepsilon$.

Mixing time

Let $\pi^*$ be stationary, and $p^t_x$ the distribution after $t$ steps starting from $x$. Define

Coupling — the workhorse for bounding mixing time

How do you prove a chain mixes fast? Coupling is the dominant technique.

Definition. A coupling of a chain $M$ (on $S$) is a chain $Z_t = (X_t, Y_t)$ on $S \times S$ such that each coordinate, viewed alone, is a faithful copy of $M$:

$$\Pr[X_{t+1}=x'\mid Z_t=(x,y)] = \Pr[M_{t+1}=x'\mid M_t=x],$$

$$\Pr[Y_{t+1}=y'\mid Z_t=(x,y)] = \Pr[M_{t+1}=y'\mid M_t=y].$$

The two copies may be correlated however we like (that’s the design freedom), as long as the marginals are correct.

Lemma (Coupling Lemma). If a coupling $Z_t=(X_t,Y_t)$ satisfies, for some time $T$,

$$\Pr[X_T \ne Y_T \mid X_0=x, Y_0=y] \le \varepsilon \quad\text{for all } x,y,$$

then the mixing time obeys $\;\tau(\varepsilon) \le T.$

Why coupling works — the picture. Run two copies of the chain at once: one started at the worst state $x$, the other started in the stationary distribution $\pi$ (so the second copy is always exactly $\pi$). Couple their randomness so that once they meet, they move together forever. At any time $t$, the variation distance of copy 1 from $\pi$ is at most the probability the two copies have not yet met. So: “design a coupling that makes the two copies collide quickly, and you have bounded the mixing time.” When the copies are close, the chain is close to stationary.

11. Card shuffling — three couplings, three mixing analyses

Shuffling a deck of $n$ cards is a random walk on the symmetric group $S_n$ whose stationary distribution is uniform (every permutation equally likely). “How many shuffles to mix” = mixing time. Three classic shuffles, three techniques.

(a) Random transposition-to-top — coupling ⇒ coupon collector

A first attempt (move a random position to the top in both decks) does not couple well. The clever coupling:

Pick a uniform card value $C \in \{1,\dots,n\}$ (not a position). In deck $X$, move the card with value $C$ to the top; in deck $Y$, also move the card with value $C$ to the top.

Now the key observation: once a card has been “named” $C$, it sits in the same relative position in both decks forever after. So the two decks become identical as soon as every value has been chosen at least once — which is exactly the coupon collector problem. After $n\ln n + cn$ steps, the probability a specific card was never chosen is

(b) Top-in-at-random — strong uniform stopping time

Take the top card and insert it into a uniformly random position in the deck. A second, equally powerful technique for proving “exactly uniform now”:

Definition (strong uniform stopping time / silno uniformné pravidlo zastavenia). A (randomized) stopping rule $T$ such that for every $K$, conditioned on $T=K$, the permutation is exactly uniform. Stopping at $T$ yields a perfectly uniform sample.

The stopping rule for top-in-at-random. Track the card that was originally on the bottom. Stop the first time it gets inserted (reaches the top and goes in). Why is this strong uniform? Each card inserted below the original bottom card lands in a uniformly random position among those below it; by the time the original bottom card itself is finally inserted, the entire deck below it has been built up uniformly at random — so the full permutation is uniform, regardless of how long that took.

The time $T$ until the original bottom card rises and inserts is, by the same structure, a coupon-collector quantity again (the times $T_i - T_{i-1}$ to accumulate $i$ cards below match the inter-arrival times $V_{n-i+1}-V_{n-i}$ of the collector).

Lemma (stopping-time bound on variation distance). If a shuffle has a strong uniform stopping time $T$, then

$$\|Q^{*k} - U\| \le \Pr[T > k].$$

Proof (the clean computation). For any event $S \subseteq S_n$, split on whether the process has stopped by time $k$:

With $k = \lceil n\ln n + cn\rceil$, the coupon-collector tail gives $\Pr[T>k]\le e^{-c}$, so again $\Theta(n\log n)$ shuffles suffice.

© Riffle shuffle — birthday paradox, $\tfrac32\log_2 n$ shuffles

The realistic shuffle: split the deck into two parts and interleave. The cleanest analysis runs the inverse shuffle (assign each card a random bit, pull the 0-cards to the top keeping relative order — equivalent to the forward riffle).

After $k$ inverse shuffles each card carries a $k$-bit label (its sequence of bits). The strong uniform stopping rule: stop when all $n$ labels are distinct — at that moment the cards are sorted by a uniformly random ordering, hence uniform. With $K = 2^k$ possible labels, “all $n$ distinct” is exactly the birthday problem:

Theorem. After $k$ riffle shuffles of $n$ cards,

$$\|\mathrm{Rif}^{*k} - U\| \le \Pr[T>k] = 1 - \prod_{i=1}^{n-1}\left(1 - \frac{i}{2^k}\right).$$

By the birthday paradox the labels become distinct once $2^k \gg n^2$, i.e. $k \approx 2\log_2 n$; a sharper analysis gives the famous $\tfrac32\log_2 n \approx 7$ shuffles for $n=52$ threshold.

Professor-pleaser — three shuffles, three classics. The same mixing question is answered three different ways, each reducing the analysis to an elementary probability gem: coupon collector (random-to-top, via coupling), coupon collector again (top-in-at-random, via strong uniform stopping time), and the birthday paradox (riffle). The two proof techniques — coupling and strong uniform stopping times — are the two universal hammers for upper-bounding mixing time, and both reduce “distribution is close to uniform” to “a simple combinatorial event has occurred.”

12. Eigenvalues, expanders, and rapid mixing

Recall $\bar p(t) = \bar p(0)P^t$ — mixing is governed by the powers of $P$, hence by its eigenvalues. This is the spectral route to bounding mixing time, and the natural home of expanders.

Spectral facts for graphs

For an undirected (multi)graph let $A(G)$ be the adjacency matrix (symmetric, so real eigenvalues $\lambda_1 \ge \lambda_2 \ge \dots \ge \lambda_n$ with an orthonormal eigenbasis). Key facts (state these; they recur):

• $G$ connected $\Rightarrow \lambda_2 < \lambda_1$ (the top eigenvalue is simple);
• $|\lambda_i| \le d$ for all $i$, where $d$ is the max degree;
• $d$ is an eigenvalue $\iff G$ is $d$-regular; then $\lambda_1 = d$ with eigenvector $e_1 = \frac{1}{\sqrt n}(1,\dots,1)$ (the all-ones / uniform direction);
• $G$ bipartite $\iff$ the spectrum is symmetric ($\lambda \leftrightarrow -\lambda$); for connected $G$, bipartite $\iff -\lambda_1$ is an eigenvalue. A $d$-regular bipartite graph has $\lambda_n = -d$.

The reading. $\lambda_1 = d$ is the uniform/stationary direction. The spectral gap $d - \lambda_2$ measures how fast everything else decays under repeated multiplication by $P = A/d$: a big gap ($\lambda_2$ far below $d$) means all non-uniform components die quickly $\Rightarrow$ fast mixing. The negative end ($\lambda_n$ near $-d$) signals near-bipartiteness $\Rightarrow$ near-periodicity, which we fix with self-loops.

Expanders

An $(n,d,c)$-expander is a bipartite $d$-regular (multi)graph $G(X,Y,E)$ with $|X|=|Y|=n/2$ that expands every subset:

Theorem (expansion ⇔ spectral gap — Cheeger-type).

• If $G$ is an $(n,d,c)$-expander, then $|\lambda_2| \le d - \dfrac{c^2}{1024 + 2c^2}$.
• Conversely, if $|\lambda_2| \le d - \varepsilon$, then $G$ is an $(n,d,c)$-expander with $c \ge \dfrac{2d\varepsilon - \varepsilon^2}{d^2}$.

The two notions — combinatorial expansion (no small cuts) and a spectral gap (large $d-\lambda_2$) — are quantitatively equivalent. This is the deep structural fact behind all of expander theory.

Random walk on an expander mixes rapidly

Take the walk $P = A(G)/d$. Kill periodicity by adding self-loops: $Q = (I+P)/2$, whose eigenvalues shift to $\lambda_i' = \frac{1 + \lambda_i/d}{2} \in [0,1]$. If $\lambda_2 = d

• \varepsilon, then $\lambda_2' = 1 - \frac{\varepsilon}{2d} < 1.

Theorem (rapid mixing). For the lazy walk $Q$ on an $(n,d,c)$-expander with $\lambda_2 \le d-\varepsilon$, from any start $q^0$, the relative deviation from stationary obeys

$$\Delta(t) \le n^{1.5}\,(\lambda_2')^{t} \le n^{1.5}\left(1 - \frac{\varepsilon}{2d}\right)^{t}.$$

Since $\lambda_2' < 1$ is a constant bounded away from 1, $\Delta(t)$ drops below any $\varepsilon$ after only $O(\log n)$ steps — expanders mix in logarithmically many steps. This is what “rapidly mixing” means in its purest form.

★ The application: amplifying BPP with $O(k)$ random bits

A gorgeous payoff, and a classic exam topic linking this lecture to the complexity lecture.

Setup. A BPP algorithm $A$ uses $n$ random bits and errs with probability $\le 1/3$. We want to push the error down to $1/2^k$. The textbook way: run $A$ on $k$ independent random strings $r_1,\dots,r_k$ and take the majority; Chernoff gives