$ grovecode

← all posts

Crunching an RGB Image into Two Channels and Three Colors

July 23, 2026  · #compression #gradient-descent #linear-algebra #javascript

An RGB image stores three numbers per pixel. What if you only kept two?

Not two of the channels — that just throws away a color. Instead, keep two scalar channels X and Y per pixel, plus three colors A, B, C shared by the whole image, and rebuild every pixel as an affine blend:

color(x, y) = A + B · X(x, y) + C · Y(x, y)

A, B, C are RGB triples — nine numbers total. X and Y are per-pixel values in [0, 1]: a two-channel “texture”. So a megapixel image costs two channels per pixel plus a tiny nine-number palette, instead of three channels per pixel.

The geometry

Every pixel is a point in RGB space. The formula A + B·X + C·Y traces out a parallelogram: start at corner A, move along edge B as X goes 0→1, along edge C as Y goes 0→1. Fitting the model means finding the flat 2D parallelogram that hugs the cloud of pixel colors as tightly as possible — and X, Y are just each pixel’s coordinates inside that parallelogram.

That’s a rank-2 affine approximation. There’s a closed-form optimum (it’s PCA: A is the mean color, B and C are the top two principal directions), but it’s more fun to watch it get discovered.

Watch it crunch

The widget below starts from a completely random guess — random X, Y texture, random colors — and refines everything at once with gradient descent (Adam), shrinking the reconstruction error step by step. It runs live in your browser at 64×64.

// dual-channel crunch — color = A + B·X + C·Y

PAUSED
iteration
0
MSE
—
PSNR
—
original
reconstruction
X channel
Y channel
X only (Y=0): A + B·X
Y only (X=0): A + C·Y
error vs. iteration

A few things to look for as it converges:

  • The reconstruction sharpens from noise into a recognizable image, and the error curve drops fast, then plateaus at the floor (~28 dB PSNR — the best any two-channel affine fit can do for this image).
  • The X only and Y only panels show each channel’s contribution in isolation (the other channel held at zero). Add those two layers together and you get the full reconstruction.
  • The palette at the bottom shows the three fitted colors and the four corners of the parallelogram (A, A+B, A+C, A+B+C). A is oriented to be the darkest, near-black corner, with B and C adding intensity.

Hit replay a few times. Because any random start is equally valid, you’ll land on different — but equally good — solutions each run. Sometimes a corner lands on pure white and another color carries negative components to tint back toward the true hue. All of them bottom out at the same error floor.

Why two channels are enough for [0, 1]

You might worry that clamping X, Y to [0, 1] (so they’re storable as an 8-bit texture) costs quality. It doesn’t. The model has a built-in gauge freedom: rescaling X, Y and folding the scale into B, C gives an identical reconstruction. So you take the optimal fit and rescale the channels into [0, 1] for free — same pixels, same error.

The one thing that would cost you is forcing A to pure black ([0, 0, 0]): that drags the parallelogram off its optimal plane and raises the error floor. “Dark A” is free; “black A” is not.

Takeaways

  • color = A + B·X + C·Y is a rank-2 affine fit — a parallelogram threaded through the pixel cloud in RGB space.
  • Random init plus plain gradient descent reliably finds the optimum; you don’t need the closed form to get there, though PCA gives it in one shot.
  • Constraining the channels to [0, 1] is a free change of coordinates, which is exactly what makes the result storable as an ordinary two-channel texture.