The tyranny of algorithms - II

Once upon a time, everybody loved the internet. It would make us freer, richer, smarter. It would make us better citizens, better consumers, better humans. Most people continue to think of tech companies as businesses that sell some product or service at a profit. Amazon just wants to sell you stuff, Facebook wants to show you advertising, and Apple wants you to buy the latest iPhone product. Your phone, your computer, and various apps and programs etc. come from just a few Silicon Valley overlords. Their companies want very much to shape what you do. They hope to automate the choices that we make everyday. Unrestrained power, no matter how well-meaning or alluring, is something of which we should always be wary.

Google, Facebook, Amazon use technology and data to sidestep traditional restrictions on monopoly power. Credit raters, search engines, major banks etc. collect data about us and convert it into scores, rankings, risk calculations, and watch lists which impact us daily. They use it to make important decisions about us and to influence the decisions we make for ourselves. A bad credit score may prevent a customer from accessing any loans but he will never understand exactly how it was calculated. A predictive analytics firm may score someone as a “high cost” or “unreliable” worker, yet never tell her about the decision. As Frank Pasquale writes in The Black Box Society:

Continuing unease about black box scoring reflects long-standing anxiety about misapplications of natural science methods to the social realm. A civil engineer might use data from a thousand bridges to estimate which one might next collapse; now financial engineers scrutinize millions of transactions to predict consumer defaults. 

But unlike the engineer, whose studies do nothing to the bridges she examines, a credit scoring system increases the chance of a consumer defaulting once it labels him a risk and prices a loan accordingly. Moreover, the “science” of secret scoring does not adopt a key safeguard of the scientific method: publicly testable generalizations and observations.  As long as the analytics are secret, they will remain an opaque and troubling form of social sorting.

The proprietary algorithms by which digital economy companies collect and analyse data of their customers are immune from scrutiny. Internet companies collect more and more data on their users but fight regulations that would exercise some control over the resulting digital dossiers. The conclusions they come to — about the productivity of employees, or the relevance of websites, or the attractiveness of investments — are determined by complex formulas devised by lots of engineers and guarded by many lawyers. The law is very protective of secrecy in the world of commerce but is increasingly silent when it comes to the privacy of persons.

Are these algorithmic applications fair? Why, for instance, does YouTube (owned by Google) so regularly beat out other video sites in Google’s video search results? How does one particular restaurant or auto stock make it to the top of the hit list while another does not? It’s anyone’s guess, as long as the algorithms involved are kept secret. Without knowing what Google actually does when it ranks sites, we cannot assess when it is acting in good faith to help users, and when it is biasing results to favor its own commercial interests. The same goes for status updates on Facebook, trending topics on Twitter, or recommendations on Amazon. Franklin Foer says in World Without Mind:

Computer scientists have an aphorism that describes how algorithms relentlessly hunt for patterns. They talk about torturing the data until it confesses. Yet this metaphor contains unexamined implications. Data, like victims of torture, tells its interrogators what it wants to hear. 

Google is the dominant way we navigate the Internet which makes it the primary lens through which we experience the world. Google has steadily added to the roles it plays in people’s lives – it hosts e-mail, it owns the free blog-hosting service Blogger, offers online software such as a word processor, spreadsheets, presentation software, and a calendar service, has its own Web browser called Chrome, its Google Books project has scanned millions and millions of volumes and has made many of them available online at no cost, it has Google Maps, Street View, and Google Earth, etc. As we shift more of our Internet use to Google-branded services such as Gmail and YouTube, Google is on the verge of becoming indistinguishable from the Web itself.

This gives it enormous power to set agendas and alter perceptions. But we should be cautious about putting our faith in the goodwill of an enterprise whose mission is “to organize the world’s information and make it universally accessible and useful,” We see so clearly how it makes our lives better, our projects easier, and our world smaller that we fail to consider the costs, risks, options, and long-term consequences of our over-reliance on it. Its biases (valuing popularity over accuracy, established sites over new, etc.) are built into algorithms and they have an affect how we value and perceive things. These may be legitimate choices by Google but one has to recognize that these are choices, not science.