Entropy Atrophy

Kudos to the Carnegie Mellon alum for the closest-fit solution to xkcd's externally-controlled April Fools comic and Skein hash collision contest (always read the alt text). I was far too hardware-deprived to be nerd-sniped by this one, but there were plenty others who jumped right on it, all motivated by challenge rather than "money."

That's encouraging because it seems information theory isn't taught much anymore (my own alma mater has long since dropped "ICS" for "CS"). Although we now need it most (in our big data and security-starved era), our collective entropy-sophy has atrophied.

In areas like security policy and algorithm design, the cold reality of the pigeonhole principle is too often forgotten. We often regard hashes as magic, forgetting they're just bit-twiddling mapping functions and that when the domain is bigger than the range, there will be collisions. Simple truths like this are ignored in spots ranging from the XBox to ReiserFS.

The winner of xkcd's contest was still 384 bits shy of a total collision, but every imperfect hash has plenty of clashes. So be careful out there to win that battle between bit length and processing power, at least while GPUs and quantum computers develop.

BTW, it seems Wikipedia got enough donations from the effort to be good sports about the xkcd-hacking.