intro
each prokudin-gorskii plate stores a scene as three vertically stacked exposures, captured through blue, green, and red filters. the task is to extract those channels and recover a single color photograph by estimating a 2d translation that registers the green and red plates onto blue.
l2 norm
for each candidate shift, we compare a crop of the moving channel (green or red) against the same window of blue. l2 is the euclidean distance between those two pixel vectors: add up the squared per-pixel differences, then take a square root. a smaller distance means the channels look more alike at that displacement.
L₂(I, J) = √ Σ (I − J)²
normalized cross-correlation (ncc)
the three filter plates are different exposures, so a region that is bright in blue may be dim in red. l2 treats that as a large error even when the scene is aligned. normalized cross-correlation (ncc) first subtracts each crop’s mean and divides by its l2 magnitude (norm), then takes a dot product. this compares the shape of the brightness pattern, not the raw intensity.
the score is a correlation in [−1, 1]. we pick the displacement with the largest ncc. our goal is to maximize ncc (alternatively, minimize l2 norm).
gaussian blur
before we shrink a channel we blur it, otherwise the downsample would alias high frequencies into the coarse image and throw off the search. the kernel is a 7×7 discrete gaussian (radius 3, σ = 1), normalized so the weights sum to 1, then applied with reflect padding.
G(x, y) = (1 / 2πσ2) e−(x2 + y2) / 2σ2
image pyramid
on a ~3000px tiff, green or red can be shifted by tens or hundreds of pixels. searching that at full resolution would mean trying every (dx, dy) in a huge window, and running ncc on millions of pixels each time. instead we search a small version of the image first, then refine.
each level uses the gaussian blur above, then keeps every other pixel ([::2, ::2]) and recurses. at the coarsest level the channel is under 100px, so we do the same ±15 exhaustive search as single-scale. coming back up, we double that offset (one coarse pixel is two fine pixels) and only look ±2 around it. our wider search occurs on a scaled down image and subsequent, larger level uses the previous estimate, which is much more efficient than our brute-force search.
border crop
np.roll is circular: pixels that slide off one edge wrap around to the other. those wrapped strips would skew both l2 and ncc calculations, so i opted for the search to only score the interior (about 10% of the channel height/width is left out). after stacking, the same wrap borders are cropped off. i chose 20px for the jpegs, since the images are generally smaller, and 100px for the tiffs.
single-scale l2 norm
cathedral, monastery, and tobolsk are small enough for a ±15 exhaustive search. scored with l2 norm. offsets are (dx, dy) of g and r relative to b.
single scale + l2 norm vs pyramid + ncc
just for comparison, here is our naive search with the l2 norm metric versus a pyramid search using the normalized cross-correlation (ncc) metric. we find similar results!
multi-scale pyramid alignment
the remaining plates are the full-size tiffs. the pyramid keeps the search to a few dozen ncc evaluations per level instead of a giant ±displacement window on a 3000px channel. offsets are (dx, dy) in pixels at full resolution.
library of congress plates
three more plates pulled from the library of congress collection, run through the same pyramid + normalized cross-correlation (ncc) pipeline.
failure: emir
emir is the one plate the pipeline did not recover. green landed at (24, 49), which is in a reasonable range, but red jumped to (-593, -442), which is quite off. you can see red emir's "ghost" to the top left of the forefront.
the likely cause is that emir’s filter plates have very different intensity patterns. a bright white turban in one plate may be darker in another; saturated clothing can also change contrast across filters. ncc compares pixel brightness patterns, so it can mistake unrelated large bright/dark regions for a match at a coarse pyramid level.
that wrong coarse shift is then doubled at each finer level. since the refine search only looks ±2 pixels around that estimate, it cannot travel hundreds of pixels back to the correct position. edges could help because they represent intensity changes instead of absolute brightness. the turban’s outline, the robe seams, the silhouette, and the door-frame boundaries should still show up in every channel even when their brightness differs.
