Main

Lecture 4.2: Design Methods II

The fingerprint method — replace big objects by tiny random projections: Freivalds, Schwartz–Zippel, Karp–Rabin; then universal and perfect hashing, Tutte matrices and the Isolation Lemma.

Last updated 27 June 2026

Lecture 4 (part 2) — Fingerprints, Hashing & the Isolation Lemma

Course 2-INF-135/15 Pravdepodobnostné algoritmy, LS 2025/26. Source slides: 04_metody23.pdf, slides 5–34. Continues the methods block; part 1 (§1–§4) was eliminating the adversary. This part is the second method — fingerprints (odtlačky) — and its biggest customers, hashing and matching.

Where we are

Part 1 was the first design method: randomize your strategy so the adversary has no fixed target. This part is the second method: fingerprints. The slogan is one line:

Don’t compare the giant objects — compare tiny random summaries of them.

We will see the same trick wear five different costumes (matrix products, polynomials, branching programs, text search, graph isomorphism), then watch it grow into the theory of universal hashing, and finally into the Isolation Lemma, which turns “does a matching exist?” into a determinant and makes the matching unique so parallel machines can agree on it.


5. The fingerprint method (metóda odtlačkov)

We want to decide equality / equivalence of two complex objects O1,O2O_1, O_2 — objects so big that comparing them directly is expensive (matrices, polynomials in normal form, long strings, function tables). The idea:

Take a family MM of maps from the full representation down to a small partial one. Pick one at random, hRMh \in_R M, and compare the fingerprints h(O1)=?h(O2)h(O_1) \overset{?}{=} h(O_2).

  • If h(O1)=h(O2)h(O_1) = h(O_2): report “(probably) equivalent.”
  • If h(O1)h(O2)h(O_1) \ne h(O_2): report “definitely not equivalent.”

Three requirements make this an algorithm rather than a wish:

  1. hh is efficiently computable;
  2. the comparison h(O1)=?h(O2)h(O_1) \overset{?}{=} h(O_2) is efficient (fingerprints are small);
  3. soundness in one direction:
    O1=O2  h(O1)=h(O2)(always),O_1 = O_2 \ \Rightarrow\ h(O_1) = h(O_2) \quad \text{(always)},
    O1O2  h(O1)h(O2)for *enough* hM.O_1 \ne O_2 \ \Rightarrow\ h(O_1) \ne h(O_2) \quad \text{for *enough* } h \in M.

The error is one-sided. Different objects always differ; equal objects never differ. The only mistake is a false positive — two different objects that happen to collide on this particular random hh. Requirement 3 guarantees only a small fraction of hh’s collide, so the failure probability is small. This is exactly a co-RP / Monte-Carlo flavour: an “unequal” verdict is gospel, an “equal” verdict is probably true, and we amplify by repeating with fresh hh. (Ties back to Lecture 3’s one-sided error classes.)

The whole rest of this section is: pick the right family MM for the objects at hand.


6. Freivalds — verifying a matrix product in O(n2)O(n^2)

The problem. Someone claims AB=CA \cdot B = C for n×nn \times n matrices. Checking by recomputing ABAB costs O(n3)O(n^3) naively (or O(n2.37)O(n^{2.37}) with fast multiplication — complicated and constant-heavy). Can we verify a claimed product faster than computing one? Yes — in O(n2)O(n^2), the cost of barely touching the matrices.

The fingerprint. Project everything through a random 0/10/1 vector αR{0,1}n\alpha \in_R \{0,1\}^n:

checkA(Bα)=?Cα.\text{check}\quad A(B\alpha) \overset{?}{=} C\alpha.

Computed right-to-left: BαB\alpha is a matrix–vector product (O(n2)O(n^2)), then A(Bα)A(B\alpha) another (O(n2)O(n^2)), and CαC\alpha another. Total O(n2)O(n^2) — never form ABAB.

Correctness. The fingerprint of a matrix XX is the vector XαX\alpha.

The one-line heart. D0D \ne 0 has some nonzero entry dijd_{ij}. Look at row ii: (Dα)i=kdikαk=dijαj+(terms not involving αj).(D\alpha)_i = \sum_k d_{ik}\alpha_k = d_{ij}\,\alpha_j + (\text{terms not involving } \alpha_j). Reveal every coordinate of α\alpha except αj\alpha_j first (principle of deferred decisions). Now (Dα)i=0(D\alpha)_i = 0 forces αj\alpha_j to one specific value. But αj\alpha_j is a fair coin over {0,1}\{0,1\}, so it hits that value with probability 12\le \tfrac12. Hence Pr[(Dα)i=0]12\Pr[(D\alpha)_i = 0] \le \tfrac12, so Pr[Dα=0]12\Pr[D\alpha = 0] \le \tfrac12.

 Pr[A(Bα)=CαABC]12. \boxed{\ \Pr[A(B\alpha) = C\alpha \mid AB \ne C] \le \tfrac12.\ }

Repeat with tt independent α\alpha’s to drive the error to 2t2^{-t}.

Punchline. Verifying a product is strictly easier than computing it — and the proof never needed anything about A,B,CA, B, C beyond “D0D\ne 0 has a nonzero entry.” A random linear projection cannot kill a nonzero matrix more than half the time. (Exam-critical: the “matrix match” fingerprint, exam-1 Q1 / exam-3 Q1.)


7. Polynomial identity testing & the Schwartz–Zippel lemma

Freivalds is the matrix face of a much bigger idea: two objects are equal iff a certain polynomial is identically zero, and a nonzero polynomial is rarely zero at a random point.

Univariate warm-up

To test p1(x)=?p2(x)p_1(x) \overset{?}{=} p_2(x): form q=p1p2q = p_1 - p_2. If p1p2p_1 \ne p_2 then q≢0q \not\equiv 0 has degree n\le n, so at most nn roots. Pick αRS\alpha \in_R S for a large set SFS \subseteq F:

Pr[q(α)=0p1p2]  nS.\Pr[q(\alpha) = 0 \mid p_1 \ne p_2] \ \le\ \frac{n}{|S|}.

Deterministically you’d expand to normal form — possibly exponential time. Randomly you just evaluate at one point. This single idea powers perfect matching (§14), read-once branching programs (§8), and pattern matching (§9).

Schwartz–Zippel (the multivariate engine)

Lemma (Schwartz–Zippel). Let Q(x1,,xn)F[x1,,xn]Q(x_1,\dots,x_n) \in F[x_1,\dots,x_n] have total degree dd, let SFS \subseteq F, and pick α1,,αnRS\alpha_1,\dots,\alpha_n \in_R S independently. Then

Pr[Q(α1,,αn)=0Q≢0]  dS.\Pr[\,Q(\alpha_1,\dots,\alpha_n) = 0 \mid Q \not\equiv 0\,] \ \le\ \frac{d}{|S|}.

The univariate fact (“degree dd \Rightarrow d\le d roots”) is the n=1n=1 case; this lifts it to many variables at the same rate d/Sd/|S|.

Proof — induction on the number of variables nn.

The deep point. Peeling off the highest-degree variable lets the induction split the degree budget d=(dk)+kd = (d-k) + k exactly between “the leading coefficient vanishes” and “the leading coefficient survives but the univariate slice vanishes.” Neither case can be bad more than its share, and they add up to the full degree. That clean additivity is why the bound stays d/Sd/|S| no matter how many variables.

Variant: counting roots over Zp\mathbb{Z}_p

A sibling lemma counts roots directly. If Q(x1,,xn)Q(x_1,\dots,x_n) over Zp\mathbb Z_p has per-variable degree d\le d, then it has at most ndpn1n\,d\,p^{\,n-1} roots — so a uniform random point in Zpn\mathbb Z_p^n is a root with probability nd/p\le n d / p. (Same induction: either all the QiQ_i vanish — (n1)dpn1(n{-}1)d\,p^{n-1} roots by induction — or some Qj0Q_j \ne 0 and the univariate slice contributes kpn1\le k\,p^{n-1}; together ndpn1\le n d\,p^{n-1}.)


8. Read-once branching programs (1BP equivalence)

A beautiful application: deciding whether two branching programs compute the same Boolean function, with no known efficient deterministic algorithm — but an easy fingerprint one.

1BP (read-once branching program): an acyclic graph; each internal vertex is labelled by a variable; each has two out-edges labelled 00 and 11. To evaluate on input α{0,1}n\alpha \in \{0,1\}^n, start at the root and at each vertex follow the edge labelled by the tested variable’s value. Read-once = every variable is tested at most once on any path. A 1-path runs from the root to the “1”-leaf; P(BP)P(\mathrm{BP}) is the set of 1-paths. The program represents ff if P(BP)P(\mathrm{BP}) is exactly the set of α\alpha with f(α)=1f(\alpha)=1.

Equivalence is a polynomial identity. Turn each 1-path y=x1v1x2v2y = x_1^{v_1} x_2^{v_2}\cdots into a monomial that is 11 exactly on that path’s inputs:

p(y)=j=1npj,pj={xj,vj=1,1xj,vj=0,QBP(x1,,xn)=yP(BP)p(y).p(y) = \prod_{j=1}^{n} p_j, \qquad p_j = \begin{cases} x_j, & v_j = 1,\\ 1 - x_j, & v_j = 0,\end{cases} \qquad\qquad Q_{\mathrm{BP}}(x_1,\dots,x_n) = \sum_{y \in P(\mathrm{BP})} p(y).

On any Boolean input α\alpha, exactly one path’s monomial is 11 (the path the input follows) and the rest are 00, so QBP(α)=f(α)Q_{\mathrm{BP}}(\alpha) = f(\alpha) on the whole cube. Because each QBPQ_{\mathrm{BP}} is multilinear, it is determined by its values on {0,1}n\{0,1\}^n. Hence

BP1BP2    f1=f2 on {0,1}n    QBP1=QBP2 as polynomials.\mathrm{BP}_1 \equiv \mathrm{BP}_2 \iff f_1 = f_2 \text{ on } \{0,1\}^n \iff Q_{\mathrm{BP}_1} = Q_{\mathrm{BP}_2} \text{ as polynomials}.

Now test QBP1QBP2?0Q_{\mathrm{BP}_1} - Q_{\mathrm{BP}_2} \overset{?}{\equiv} 0 with Schwartz–Zippel: evaluate both at a random point over a large field SS (going beyond {0,1}\{0,1\} to get a real probability gap). Example monomial sum:

Q(x1,x2,x3)=(1x1)x2x3+(1x1)(1x2)(1x3)+x1x2.Q(x_1,x_2,x_3) = (1-x_1)x_2 x_3 + (1-x_1)(1-x_2)(1-x_3) + x_1 x_2.

Why this is striking. We never expand the function table (2n2^n entries). We compare two programs by one random evaluation of their arithmetizations. The same “Boolean function \to multilinear polynomial” move (“arithmetization”) is the seed of the whole IP=PSPACE\mathsf{IP} = \mathsf{PSPACE} story.


9. Pattern matching — Karp–Rabin fingerprints

The problem. Text X=x1xnX = x_1\dots x_n, pattern Y=y1ymY = y_1\dots y_m (bits). Find an occurrence: a position jj with X(j):=xjxj+1xj+m1=YX(j) := x_j x_{j+1}\dots x_{j+m-1} = Y.

The fingerprint = remainder modulo a random prime. Read each mm-bit window as a number. Pick a random prime pR{primesτ}p \in_R \{\text{primes} \le \tau\} (with τ\tau a function of m,nm,n to be tuned), and compare remainders:

Op(X(j)):=X(j)modp,Op(Y):=Ymodp,report j if Op(X(j))=Op(Y).O_p(X(j)) := X(j) \bmod p, \qquad O_p(Y) := Y \bmod p, \qquad \text{report } j \text{ if } O_p(X(j)) = O_p(Y).

Why it’s fast — the rolling hash. Op(Y)O_p(Y) is computed once. Each window fingerprint is obtained from the previous one in O(1)O(1) arithmetic operations (O(logp)O(\log p) bit-ops): drop the top bit, shift, add the new bit, all mod pp:

Op(X(k)) = (2Op(X(k1))xk12m+xk+m1)modp.O_p(X(k)) \ =\ \big(2\cdot O_p(X(k-1)) - x_{k-1}\,2^{m} + x_{k+m-1}\big) \bmod p.

Total time O(n+m)O(n + m) instead of O(nm)O(nm) for naive matching.

Error analysis. A false match at jj means pX(j)Yp \mid |X(j) - Y|, where X(j)Y|X(j) - Y| is an mm-bit number, hence <2m< 2^m, hence has fewer than mm prime divisors. The number of primes τ\le \tau is τ/lnτ\approx \tau / \ln \tau (prime number theorem). So per position Pr[false match]mτ/lnτ\Pr[\text{false match}] \le \dfrac{m}{\tau/\ln\tau}, and summing over the n\le n positions:

jPr[Op(Y)=Op(X(j))YX(j)]  nmlnττ = O ⁣(nmlogττ).\sum_{j} \Pr[\,O_p(Y) = O_p(X(j)) \mid Y \ne X(j)\,] \ \le\ \frac{n m \ln \tau}{\tau} \ =\ O\!\left(\frac{nm \log \tau}{\tau}\right).

Choosing τ=n2mlog(n2m)\tau = n^2 m \log(n^2 m) makes this 2/n\le 2/n.

Monte-Carlo vs. Las-Vegas.

Punchline. A whole mm-bit window is crushed to a number mod a small prime, and the rolling update means the fingerprint of the next window is one cheap step from the current — turning O(nm)O(nm) into O(n+m)O(n+m). The error is one-sided and verifiable, so the Monte-Carlo algorithm upgrades to Las-Vegas for free.


10. Fingerprints across a conversation — interactive proofs

The fingerprint idea generalizes from “one random check” to a dialogue. An interactive protocol is a pair (P,V)(P, V):

Prover PP (dôkaz / student) Verifier VV (verifikácia / teacher)
power unbounded computation polynomial-time, randomized
sees the public messages its own random bits (PP does not)

They exchange messages; the verifier has the last word. We want:

xL  P: V(P,x)=1(a true claim has a convincing prover — *completeness*),x \in L \ \Rightarrow\ \exists P:\ V(P,x) = 1 \quad\text{(a true claim has a convincing prover — *completeness*)},
xL  P: Pr[V(P,x)=1]12(a false claim survives only by luck — *soundness*).x \notin L \ \Rightarrow\ \forall P:\ \Pr[V(P,x) = 1] \le \tfrac12 \quad\text{(a false claim survives only by luck — *soundness*)}.

IP\mathsf{IP} is the class of languages with such a protocol.

Graph non-isomorphism — the showcase

LGI={(G1,G2):τ, G1=τ(G2)}NP(witness=τ),L_{GI} = \{(G_1,G_2) : \exists \tau,\ G_1 = \tau(G_2)\} \in \mathsf{NP}\quad(\text{witness} = \tau),
LGNI={(G1,G2):τ, G1τ(G2)}coNP.L_{GNI} = \{(G_1,G_2) : \forall \tau,\ G_1 \ne \tau(G_2)\} \in \mathsf{coNP}.

For non-isomorphism we have no short certificate — how do you exhibit a proof that no permutation works? Interaction + randomness gives one:

Protocol (verifier VV moves first):

  1. VV secretly flips iR{1,2}i \in_R \{1,2\} and picks a random permutation τ\tau.
  2. VV computes H=τ(Gi)H = \tau(G_i) and sends HH to PP.
  3. PP answers with j{1,2}j \in \{1,2\} — its guess for which graph HH came from.
  4. VV accepts “G1,G2G_1, G_2 non-isomorphic” iff i=ji = j.

Theorem. If G1,G2G_1, G_2 are non-isomorphic, an honest PP convinces VV with certainty. If they are isomorphic, any (even cheating) PP convinces VV with probability 12\le \tfrac12.

Why it works — the intuition.

The deep point. Randomness lets the verifier pose a question whose answer PP knows iff the claim is true: “tell apart two graphs” is possible only when they really are different. The verifier’s hidden coin is the fingerprint PP cannot fake. This is the entry point to IP=PSPACE\mathsf{IP} = \mathsf{PSPACE} — interaction + randomness buys “proofs” that ordinary coNP\mathsf{coNP} certificates seem unable to give.


11. Hashing I — linear probing and balls-in-boxes

Fingerprints meet data structures: a hash function is a fingerprint we store things by. Before the theory, two concrete analyses.

Linear probing in space n=3mn = 3m has O(1)O(1) expected FIND

Store mm keys in a table TT of size n=3mn = 3m (load factor 13\tfrac13). On a collision at slot h(x)h(x), probe h(x)+1,h(x)+2,h(x)+1, h(x)+2, \dots until a free slot. Claim: expected probe length is O(1)O(1).

Proof gadget — a binary tree over the table. Build a complete binary tree whose leaves are the table slots. A vertex vv at height kk covers 2k2^k consecutive slots. Say a key xx hashes into vv if h(x)h(x) lands in vv’s subtree. The expected number of keys hashing into vv is

μ=mn2k=2k3.\mu = \frac{m}{n}\cdot 2^k = \frac{2^k}{3}.

Call vv dangerous if at least 2μ2\mu keys hash into it. By Chernoff (Pr[X(1+δ)μ](eδ/(1+δ)1+δ)μ\Pr[X \ge (1+\delta)\mu] \le \big(e^\delta/(1+\delta)^{1+\delta}\big)^\mu with δ=1\delta = 1):

Pr[#keys2μ]  (e4)μ.\Pr[\#\text{keys} \ge 2\mu] \ \le\ \left(\frac{e}{4}\right)^{\mu}.

Since e/4<1e/4 < 1, dangerous vertices are exponentially unlikely in their height. A long run (cluster) BB of length b{2,,2+1}b \in \{2^\ell, \dots, 2^{\ell+1}\} forces at least one of 3\sim 3 subtrees of height 2\ell-2 covering it to be dangerous, so

Pr[run length{2,,2+1}]  3(e4)22/3,\Pr[\text{run length} \in \{2^\ell, \dots, 2^{\ell+1}\}] \ \le\ 3\left(\frac{e}{4}\right)^{2^{\ell-2}/3},
E[FIND]=bbpb  32(e4)22/3=O(1).E[\mathrm{FIND}] = \sum_b b\,p_b \ \le\ 3\sum_\ell 2^\ell \left(\frac{e}{4}\right)^{2^{\ell-2}/3} = O(1).

The combinatorial core (“a long run needs a dangerous subtree”) is a capacity / pigeonhole argument: if all three covering subtrees of height 2\ell-2 were safe (each holding <2μ< 2\mu keys), they could not supply enough keys to fill a contiguous run that long — there would be a hole (empty slot) inside it, contradicting that it is one unbroken run.

Deep point. The slack load factor 13\tfrac13 is doing the work: at every height the expected fill is a third of capacity, so being double the mean (dangerous) is a large deviation, and Chernoff makes it vanish fast enough that the expected cluster length — hence expected FIND — is a constant independent of mm.

Balls in boxes (guličky a krabice)

Throw mm balls uniformly into nn boxes. Three staples:

  1. Load: XX = balls in a fixed box, E[X]=m/nE[X] = m/n.
  2. Empty boxes: Zi=1Z_i = 1 iff box ii is empty; E[Zi]=(11n)mem/nE[Z_i] = (1-\tfrac1n)^m \approx e^{-m/n}, so E[Z]=nem/nE[Z] = n\,e^{-m/n}. For m=nm = n: E[Z]n/eE[Z] \approx n/e (about a third of the boxes stay empty even with as many balls as boxes).
  3. First collision (birthday): with kk balls,
    Pr[no collision]=i=1k1(1in)i=1k1ei/n=ek(k1)/(2n).\Pr[\text{no collision}] = \prod_{i=1}^{k-1}\Big(1 - \tfrac{i}{n}\Big) \le \prod_{i=1}^{k-1} e^{-i/n} = e^{-k(k-1)/(2n)}.
    This drops below 12\tfrac12 once k(k1)/(2n)ln2k(k-1)/(2n) \ge \ln 2, i.e. knk \sim \sqrt{n}.

The n\sqrt n threshold (birthday paradox) is the single most-reused fact in hashing: collisions become likely at knk \approx \sqrt n items, which is exactly why a table that wants no collisions among mm keys needs m2\sim m^2 slots (§13).


12. Universal hash families

Setup. Universe U={0,,m1}U = \{0,\dots,m-1\}, table T={0,,n1}T = \{0,\dots,n-1\}, hash h:UTh : U \to T. For any single fixed hh there is a bad input (some set of keys that all collide). The adversary wins against any fixed function. Solution — randomize the function: keep a family H={h}H = \{h\} and pick hRHh \in_R H. This is “eliminate the adversary” (part 1) applied to the hash function itself — and it yields a good deterministic structure for a static dictionary (fix the lucky hh once).

Definitions

For mnm \ge n and a family HH of functions UTU \to T:

Property Condition (for all distinct x1,,xkx_1,\dots,x_k; hRHh \in_R H)
universal Pr[h(x)=h(y)]1n\Pr[h(x) = h(y)] \le \dfrac1n for xyx \ne y
kk-universal Pr[h(x1)==h(xk)]1nk1\Pr[h(x_1) = \cdots = h(x_k)] \le \dfrac{1}{n^{\,k-1}}
strongly kk-universal (kk-independent) y1,,yk: Pr[h(x1)=y1,,h(xk)=yk]=1nk\forall y_1,\dots,y_k:\ \Pr[h(x_1)=y_1,\dots,h(x_k)=y_k] = \dfrac{1}{n^{\,k}}

Universal = “collisions no more likely than for a truly random function.” Strongly kk-universal = “on any kk inputs the outputs look fully independent and uniform.”

Counting collisions with a 2-universal family

Let S={x1,,xm}S = \{x_1,\dots,x_m\}, Xij=1X_{ij} = 1 iff h(xi)=h(xj)h(x_i) = h(x_j), and X=i<jXijX = \sum_{i<j} X_{ij} the number of colliding pairs:

E[X]=i<jPr[h(xi)=h(xj)](m2)1nm22n.E[X] = \sum_{i<j} \Pr[h(x_i) = h(x_j)] \le \binom{m}{2}\frac1n \le \frac{m^2}{2n}.

A box holding YY keys creates (Y2)Y2/2\binom Y2 \approx Y^2/2 collisions, so by Markov Pr[Xm2/n]12\Pr[X \ge m^2/n] \le \tfrac12, giving Pr[Ym2/n]12\Pr[Y \ge m\sqrt{2/n}\,] \le \tfrac12; for m=nm = n, Pr[max load2n]12\Pr[\text{max load} \ge \sqrt{2n}\,] \le \tfrac12. Good enough for one level — but the max bin is still n\sim\sqrt n, which §13 fixes.

A concrete 2-universal family

ha,b(x)=((ax+b)modp)modn,pmn prime,h_{a,b}(x) = \big((a x + b) \bmod p\big) \bmod n, \qquad p \ge m \ge n \text{ prime},
H={ha,b:1ap1, 0bp1},H=p(p1).H = \{h_{a,b} : 1 \le a \le p-1,\ 0 \le b \le p-1\}, \quad |H| = p(p-1).

Lemma. HH is 2-universal: Pr[ha,b(x1)=ha,b(x2)]1n\Pr[h_{a,b}(x_1) = h_{a,b}(x_2)] \le \tfrac1n for x1x2x_1\ne x_2.

Proof. First, x1x2ax1+b≢ax2+b(modp)x_1 \ne x_2 \Rightarrow a x_1 + b \not\equiv a x_2 + b \pmod p (since a0a \ne 0). For fixed x1x2x_1 \ne x_2, the map $(a,b) \mapsto (u, v) = (a x_1 + b,\ a x_2 + b) \bmod p$ is a bijection onto pairs (u,v)(u, v) with uvu \ne v (solve the 2×22\times2 system — x1x2x_1 \ne x_2 makes it invertible). A collision means uv(modn)u \equiv v \pmod n with uvu \ne v. For each uu there are at most p/n1(p1)/n\lceil p/n\rceil - 1 \le (p-1)/n values vuv \ne u with vu(modn)v \equiv u \pmod n. So the number of colliding pairs is pp1n\le p\cdot\frac{p-1}{n}, and

Pr[collision]  p(p1)/np(p1) = 1n.\Pr[\text{collision}] \ \le\ \frac{p(p-1)/n}{p(p-1)} \ =\ \frac1n. \qquad \blacksquare

Strongly 2-universal families

Scalar version. U=T={0,,p1}U = T = \{0,\dots,p-1\}, pp prime, ha,b(x)=(ax+b)modph_{a,b}(x) = (ax+b)\bmod p, H={ha,b:0a,bp1}H = \{h_{a,b} : 0 \le a,b \le p-1\}, H=p2|H| = p^2.

Lemma. Pr[h(x1)=y1, h(x2)=y2]=1/p2\Pr[h(x_1) = y_1,\ h(x_2) = y_2] = 1/p^2 for distinct x1,x2x_1, x_2.

Because the linear system ax1+b=y1, ax2+b=y2(modp)a x_1 + b = y_1,\ a x_2 + b = y_2 \pmod p has a unique solution (a,b)(a,b) (Vandermonde, x1x2x_1 \ne x_2). So exactly one of the p2p^2 functions sends x1y1, x2y2x_1 \mapsto y_1,\ x_2 \mapsto y_2 — probability 1/p21/p^2.

Vector version. U={0,,pk1}U = \{0,\dots,p^k-1\}, T={0,,p1}T = \{0,\dots,p-1\}, identify u(u0,,uk1){0,,p1}ku \leftrightarrow (u_0,\dots,u_{k-1}) \in \{0,\dots,p-1\}^k, and

ha,b(u)=(i=0k1aiui+b)modp.h_{a,b}(u) = \Big(\sum_{i=0}^{k-1} a_i u_i + b\Big) \bmod p.
If u1u2u_1 \ne u_2 they differ in some coordinate ii; fixing the other aja_j, the two equations aiu1,i+b=, aiu2,i+b=a_i u_{1,i} + b = \dots,\ a_i u_{2,i} + b = \dots pin down (ai,b)(a_i, b) uniquely among p2p^2 choices, giving Pr[h(u1)=y1h(u2)=y2]=1/p2\Pr[h(u_1)=y_1 \wedge h(u_2)=y_2] = 1/p^2. Still strongly 2-universal.

Why “strongly” matters. Universal controls collisions; strongly 2-universal controls the actual output distribution on any two points — and that is what limited-independence derandomization needs (Lecture 2’s pairwise independence is exactly strong 2-universality).


13. Perfect hashing — O(1)O(1) worst-case lookups

A static dictionary SS (S=m|S| = m) wants O(1)O(1) worst-case FIND, not just expected. With 2-universal HH and chaining, the expected bin size is great but the max bin is n\sim\sqrt n — too slow in the worst case. We want a perfect hash: no collisions on SS.

First: expected bin size

Lemma. For hh from a 2-universal family and X=bin(h(x))X = |\text{bin}(h(x))|,

E[X]={m/n,xS,1+(m1)/n,xS.E[X] = \begin{cases} m/n, & x \notin S,\\ 1 + (m-1)/n, & x \in S.\end{cases}

(Indicators Xi=1X_i = 1 iff h(x)=h(si)h(x) = h(s_i); E[Xi]=1/nE[X_i] = 1/n for sixs_i \ne x, plus the certain self-term when xSx \in S.) For n=mn = m this is E[X]2E[X] \le 2 — but somewhere a bin still has n\sim\sqrt n keys. So average is fine, worst case is not.

Perfect hashing in O(m2)O(m^2) space

If nm2n \ge m^2 then Pr[h is perfect on S]12\Pr[h \text{ is perfect on } S] \ge \tfrac12.

With X=i<jXijX = \sum_{i<j} X_{ij} counting collisions, E[X](m2)/n<m2/(2n)12E[X] \le \binom m2/n < m^2/(2n) \le \tfrac12, so by Markov Pr[X1]12Pr[no collision]12\Pr[X \ge 1] \le \tfrac12 \Rightarrow \Pr[\text{no collision}] \ge \tfrac12. Find one by Las Vegas: try random hh’s; E[#tries]=2E[\#\text{tries}] = 2. Lookup is then O(1)O(1) worst case — but space is a wasteful O(m2)O(m^2).

Two-level perfect hashing in O(m)O(m) — the FKS scheme

Lemma. Two-level hashing gives perfect hashing in O(m)O(m) space.

The only worry is total second-level space ibi2\sum_i b_i^2. Pick a level-1 hh whose collision count satisfies X=i(bi2)mX = \sum_i \binom{b_i}{2} \le m (possible since E[X]m2/(2n)=m/2E[X] \le m^2/(2n) = m/2, so Pr[Xm]12\Pr[X \ge m] \le \tfrac12). Then, using b2=2(b2)+bb^2 = 2\binom b2 + b,

ibi2 = 2i(bi2)+ibi  2m+m = 3m = O(m).\sum_i b_i^2 \ =\ 2\sum_i \binom{b_i}{2} + \sum_i b_i \ \le\ 2m + m \ =\ 3m \ =\ O(m).

Punchline (FKS, Fredman–Komlós–Szemerédi). O(m)O(m) space, O(1)O(1) worst-case lookup, for a static set — optimal. The trick is fractal: the same “quadratic table \Rightarrow no collisions” idea is applied once globally (to bound bi2\sum b_i^2) and once inside each bin (to make each bin perfect). The birthday n\sqrt n threshold from §11 is why quadratic is exactly the right size at each level.


14. Matchings via matrices — Tutte and the Isolation Lemma

The grand finale: fingerprints decide whether a graph has a perfect matching, and the Isolation Lemma makes that matching unique so we can even compute it in parallel.

Tutte matrix — matching existence is a polynomial identity

For a graph GG on nn vertices, the Tutte matrix AA has an indeterminate per edge:

A(i,j)={xij,(i,j)E, i<j,xij,(i,j)E, i>j,0,(i,j)E.A(i,j) = \begin{cases} x_{ij}, & (i,j) \in E,\ i < j,\\ -x_{ij}, & (i,j) \in E,\ i > j,\\ 0, & (i,j)\notin E.\end{cases}

(It is skew-symmetric.)

Theorem (Tutte). GG has a perfect matching     det(A)≢0\iff \det(A) \not\equiv 0 (as a polynomial in the xijx_{ij}).

Why. Expand

det(A)=πSn(1)sgn(π)i=1nA(i,π(i)),\det(A) = \sum_{\pi \in S_n} (-1)^{\operatorname{sgn}(\pi)} \prod_{i=1}^n A(i, \pi(i)),
and read each permutation π\pi as a cycle cover of GG.

So det(A)≢0    \det(A) \not\equiv 0 \iff some surviving (all-even, in particular the matching) term exists     \iff GG has a perfect matching. Testing det(A)≢0\det(A) \not\equiv 0 is polynomial identity testing — by Lovász, substitute random values xijRZpx_{ij} \in_R \mathbb Z_p with p=Ω(n2)p = \Omega(n^2) and apply Schwartz–Zippel: if a matching exists, det0\det \ne 0 with high probability.

Exam-critical. “Matching exists” \equiv “this determinant is a nonzero polynomial” \equiv “a random evaluation is nonzero.” The fingerprint method decides a combinatorial property through a numeric check.

Making the minimum matching unique (toward the algorithm)

Existence is not enough if we want to output a matching — and in parallel we cannot just “pick one.” So put random weights on edges and aim for a unique minimum matching. Replace xij±2wijx_{ij} \mapsto \pm 2^{w_{ij}}:

B(i,j)={2wij,A(i,j)=xij,2wij,A(i,j)=xij,0,A(i,j)=0.B(i,j) = \begin{cases} 2^{w_{ij}}, & A(i,j) = x_{ij},\\ -2^{w_{ij}}, & A(i,j) = -x_{ij},\\ 0, & A(i,j) = 0.\end{cases}

Theorem. If GG has a unique minimum-weight perfect matching MM of weight WW, then det(B)0\det(B) \ne 0 and 22W2^{2W} is the largest power of 2 dividing det(B)\det(B).

Each π\pi contributes val(π)=iB(i,π(i))\operatorname{val}(\pi) = \prod_i B(i,\pi(i)) with val(π)=2(sum of edge weights used)|\operatorname{val}(\pi)| = 2^{(\text{sum of edge weights used})}. Odd cycles cancel. An even-cycle cover decomposes into two matchings M1,M2M_1, M_2, contributing 2W(M1)+W(M2)2^{W(M_1)+W(M_2)}. The special “doubled MM” permutation (each matched edge as a 2-cycle) contributes 22W2^{2W}. Because MM is the unique minimum, every other even permutation has W(M1)+W(M2)>2WW(M_1) + W(M_2) > 2W, so the term 22W2^{2W} is the lowest power of 22 and cannot cancel (it is alone at that level). Hence v2(detB)=2Wv_2(\det B) = 2W exactly.

Theorem (read off the edges). With MM the unique minimum matching of weight WW,

(i,j)M    det(Bij)2wij22W is odd,(i,j) \in M \iff \frac{\det(B_{ij})\, 2^{w_{ij}}}{2^{2W}} \text{ is odd},
where BijB_{ij} is the minor deleting row ii, column jj.

The minor isolates permutations sending iji \mapsto j; if (i,j)M(i,j) \in M, exactly one even cycle contributes at the minimal level 22W2^{2W} (odd ratio); if (i,j)M(i,j) \notin M, every contribution is 22W+1\ge 2^{2W+1} (even ratio). One determinant per edge tells you membership.

The Isolation Lemma — where the uniqueness comes from

But how do we guarantee a unique minimum matching? Random weights — and the reason is completely general, nothing to do with graphs:

Isolation Lemma. Let (X,F)(X, \mathcal F) be a set system, X={x1,,xm}X = \{x_1,\dots,x_m\}, F={S1,,Sk}\mathcal F = \{S_1,\dots,S_k\} with SiXS_i \subseteq X, and weights w(S)=xSw(x)w(S) = \sum_{x \in S} w(x). If each w(xi)R{1,,2m}w(x_i) \in_R \{1,\dots,2m\} independently, then

Pr[ a unique minimum-weight set in F]  12.\Pr[\,\exists \text{ a unique minimum-weight set in } \mathcal F\,] \ \ge\ \tfrac12.

(For matching: XX = edges, F\mathcal F = perfect matchings. So random edge weights isolate a unique minimum perfect matching w.p. 12\ge\tfrac12.)

Proof — the threshold trick. Reveal the weights one element at a time. Fix all weights except w(xi)w(x_i) and define the threshold

αi=(min weight of a set not containing xi)(min weight of a set containing xi, with w(xi) set to 0).\alpha_i = \big(\min\text{ weight of a set } \mathbf{not}\text{ containing } x_i\big) - \big(\min\text{ weight of a set containing } x_i,\text{ with } w(x_i) \text{ set to } 0\big).
Both terms are independent of w(xi)w(x_i). Now compare:

The first two cases force xix_i’s membership; only the tie leaves it undecided. Since αi\alpha_i does not depend on w(xi)w(x_i),

Pr[xi ambiguous]=Pr[w(xi)=αi]12m.\Pr[x_i \text{ ambiguous}] = \Pr[w(x_i) = \alpha_i] \le \frac{1}{2m}.
Union bound over the mm elements:
Pr[ ambiguous element]m12m=12.\Pr[\exists \text{ ambiguous element}] \le m \cdot \frac{1}{2m} = \frac12.
If no element is ambiguous, every element’s membership in the minimum set is forced — so the minimum-weight set is unique. Hence Pr[unique minimum]12\Pr[\text{unique minimum}] \ge \tfrac12. \blacksquare

The deep point. Uniqueness can fail only if some element lands exactly on its own threshold — and that threshold was fixed before its weight was drawn, so it is a bullseye hit with probability 1/2m\le 1/2m. The weights need only be drawn from a range of size 2m2m (linear in the ground set), independent of how many — possibly exponentially many — sets F\mathcal F contains. That is the magic: we tame an exponential family with O(m)O(m)-range random weights.


15. RNC perfect matching — the payoff

Put it together. Perfect matching RNC\in \mathsf{RNC} (randomized poly-log parallel time):

Input: G(V,E)G(V,E) containing a perfect matching.

  1. For every edge ee, choose a random weight w(i,j)R{1,,2m}w(i,j) \in_R \{1,\dots,2m\}.
  2. Build the weighted matrix BB (B(i,j)=±2w(i,j)B(i,j) = \pm 2^{w(i,j)}).
  3. Compute det(B)\det(B).
  4. Find the largest WW with 22Wdet(B)2^{2W} \mid \det(B) — the minimum matching weight.
  5. Compute the adjugate adj(B)=det(B)B1\operatorname{adj}(B) = \det(B)\cdot B^{-1} (all minors at once).
  6. For every edge, r(i,j)=det(Bij)2w(i,j)/22Wr(i,j) = \det(B_{ij})\,2^{w(i,j)} / 2^{2W}.
  7. Put (i,j)M(i,j) \in M iff r(i,j)r(i,j) is odd.

By the Isolation Lemma, step 1 yields a unique minimum perfect matching w.p. 12\ge\tfrac12; steps 3, 5 (determinant and adjugate) are in NC\mathsf{NC} (parallelizable). The only randomness is the weights.

Punchline (Mulmuley–Vazirani–Vazirani). The hard part of parallel matching is not finding a matching but agreeing on one — independent processors computing in parallel have no way to coordinate a choice among many matchings. The Isolation Lemma dissolves the problem: random weights make the minimum matching unique, so it becomes a canonical object every processor computes the same way, read off edge-by-edge from one determinant and its minors. Randomness here buys not speed but coordination.


Recurring themes from this part

Theme Where it appeared
Fingerprint = small random projection of a big object Freivalds (§6), Karp–Rabin (§9), hash functions (§12)
Equality \to “is this polynomial 0\equiv 0?” \to random evaluation Schwartz–Zippel (§7), 1BP (§8), Tutte matching (§14)
One-sided error, then verify/amplify (Monte-Carlo \to Las-Vegas) fingerprints (§5), Karp–Rabin LV (§9), perfect hashing (§13)
Eliminate the adversary, hashing edition (random hh from a family) universal families (§12)
Birthday n\sqrt n threshold sets the table size balls-in-boxes (§11) \to FKS quadratic tables (§13)
Chernoff “double the mean is rare” linear probing dangerous vertices (§11)
Combinatorial property decided by a numeric determinant Tutte matrix (§14)
Randomness for coordination, not speed (make the answer unique) Isolation Lemma \to RNC matching (§14–§15)

The one sentence tying it together:

Stop comparing the elephants — compare their shadows. A random projection (a vector, a prime, a field point, a hash) shrinks an intractable equality test to a cheap one with only one-sided error; pushed further, the same idea hashes data in O(1)O(1), decides matching through a determinant, and — via the Isolation Lemma — isolates a single canonical solution that even parallel machines can agree on.

fingerprintspolynomial-identitySchwartz-Zippelhashinguniversal-hashingperfect-hashingmatchingisolation-lemmaRNC

← Back to Randomized Algorithms