The Book of the Week is “Super Crunchers” by Ian Ayres, published in 2007. This is a book about how projections based on vast quantities of numerical data in various areas of life are spurring innovations and controversy.
Improvements have been made in health, education, welfare, politics, marketing and other aspects of the day-to-day existence of humans because technological advances have greatly facilitated large volumes of number-crunching; however, not without heated debates.
People who are “experts” in specific disciplines whose projections can be quantified, are being obsolesced by machines that make predictions better than they can. For instance, software has been created to project the duration of celebrity marriages. Such duration has been found to have an inverse relationship with Google web traffic. Horror.
When this ebook was published, Farecast.com (Now Bing Travel), a company known for its online airfare search engine– processed its information with a five-terabyte database– “… fifty billion prices that it purchased from ITA Software, a company that sells price data to travel agents, websites, and computer reservation services.” The sheer amount of data minimizes bias. Such “randomization” lets researchers “… run the equivalent of a controlled test without having to laboriously match up and control for dozens or hundreds of potentially confounding variables.”
A hue and cry was heard at teaching hospitals when internet users acquired the ability to diagnose themselves by Googling their symptoms. Around the same time, software was created by medical professionals concerned about the high percentage of misdiagnoses. Such software allowed medical-school students to make diagnoses with the use of a statistical algorithm in a database of diseases, syndromes, disorders, symptoms, causes, drug side effects, clinical findings, lab results and patient histories. The data consisted of “…word patterns in journal articles that were most likely to be associated with each disease.” The computer was more accurate than the medical school professors.
One profession in which jobs are not threatened by large-scale data processing, is psychoanalysis. It’s inferential and subjective– hard to quantify. In financial services, ego and feelings interfere with securities trading and the granting of loans. But computer programs’ regression equations are completely impartial. So they do better than humans at making predictions that make money. Even when a combination of a human and a machine are used to determine whether to grant parole to convicts (based on the probability they’ll go back to committing crimes after being released from prison), the machine alone makes better decisions in a larger percentage of cases.
Read the book to learn why number-crunching software is: inappropriate for making major one-time decisions; making some teachers into robots; good at predicting Supreme Court decisions; sometimes poorly understood by healthcare professionals; raising privacy concerns, and much more.
147 thoughts on “Super Crunchers”
Comments are closed.