Review of Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy by Cathy O'Neil

The profile of the term “Big Data” has risen recently. Yet, like so many buzzwords, people often don’t fully grasp the significance of the term. “Big Data” is more than the nebulous connotation of corporations collecting our information, and perhaps packaging and selling it—although it is that. It is, in fact, about how corporations quantify everything we do, even the information we don’t realize we’re leaking out into the world, and then use that data to make decisions on our behalf, or for us, or about us, without even informing us of how they reached those decisions. It’s creepy. And it is already everywhere in our lives.

Weapons of Math Destruction is Cathy O’Neil’s impassioned plea not to let this spread further, and indeed, for us to take a hard look at how we are using Big Data algorithms already. I was drawn to this book because it’s at the intersection of two things I love: mathematics and social justice. As a mathematician, I love learning about how mathematics interacts with our society. People scoff at the utility of math, particularly the higher-level, “pure” math, yet ultimately that is what powers the digital devices we use every day and allows us to do things like make video calls across continents. You don’t have to understand the math to appreciate its power. Similarly, as someone passionate about social justice, I am sympathetic to O’Neil’s argument here. To be clear, you don’t have to have either of these qualities in order to find Weapons of Math Destruction informative or valuable; that’s just to indicate where I’m coming from.

Some of the topics O’Neil covers will be familiar, in part or all, to readers. You might already have heard about predictive policing using models of criminal activity, and the way in which it leads to a self-fulfilling prophecy (the more heavily you police an area, the more crime you uncover, the more you police the area…). You’ve probably heard about how credit scores are notoriously far-flung now yet frustratingly opaque. I also appreciate how O’Neil mentions lower-tech WMDs, like the college ranking system, which started off as something not based on computer algorithms at all. The various and sundry examples given here reinforce the fact that WMDs exist throughout our society rather than in any particular field.

More broadly, O’Neil’s overall thesis is that we can’t fix this with more technology. This is not a problem of not having “enough data” or not having “good enough models” or programs or whatever. Fundamentally this is a social problem. I appreciate that she outright states that mathematics is not the solution here, nor is it being used in a neutral way. Often people like to pretend that math and science, because they are so-called “hard” disciplines (as in their rigour, not their difficulty) are objective or neutral. That’s not the case, as O’Neil demonstrates here.

O’Neil makes the companion point, though, that models are not in and of themselves negative social forces. A model is not a WMD until it is deployed improperly or its flaws are ignored. To some extent, we are stuck, now that we have the technological capability to do this. Although, on the surface, this might seem like a contradiction of her thesis, it’s actually just the logical conclusion: O’Neil reminds us that the only way we can fix these mathematical tools is through social pressure, i.e., as a society we have to decide how we want these data-crunching algorithms to operate.

So, Weapons of Math Destruction admirably fulfills its purpose: it will educate you in more detail about algorithms, big data, and the decisions that they make about our lives. It could have been fuller and longer, sure. O’Neil’s writing style is pretty basic and leaves something to be desired (or developed further). Yet that just means it’s a quick read. It has the word “math” in the title, and sure, there is some discussion of math (mostly statistics)—but I promise you there is nary an equation to be found in these pages, and while that might be disappointing to me, I suspect it won’t be to many other readers.