Blackbox ((install)) • Recommended & Fast

We are now in a position where we must trust the oracle, but we are forbidden from looking behind the curtain. Historically, enlightenment thinkers believed that explanation preceded trust . We believed the sun would rise because Newton explained gravity. We believed a surgeon was competent because we saw their diploma.

Ironically, we call this device the "black box" (it’s actually bright orange). It is the ultimate witness. It swallows a storm of inputs—airspeed, altitude, button presses, screams—and produces a perfectly linear story of cause and effect. blackbox

In 1977, two Boeing 747s collided on a foggy runway in the Canary Islands. It remains the deadliest accident in aviation history. In the aftermath, investigators didn’t rely on eyewitness testimony or pilot memory. They dove into a twisted, fire-scorched device buried in the tail of the aircraft: the Flight Data Recorder . We are now in a position where we

The old black box—the flight recorder—was built to survive a fire. It tells us exactly why we crashed. We believed a surgeon was competent because we

The machine was brilliant. It was also dangerously stupid. And no one could see the error until a person almost died. This opacity is crashing headlong into Western jurisprudence. The 14th Amendment guarantees "equal protection under the law." But what happens when a judge uses a black box algorithm to set bail? If the algorithm is biased against a zip code, how can a defense attorney cross-examine it?

In 2016, ProPublica investigated an algorithm called COMPAS, used in US courts to predict recidivism. The black box returned a "risk score." ProPublica found it was twice as likely to falsely label Black defendants as future criminals than white defendants. The company that made the algorithm denied the bias. Because the box was black, both sides could claim the math supported them.

It works. It works terrifyingly well. But it is mute.