The tyranny of algorithms - II

Once upon a time, everybody loved the internet. It would make us freer, richer, smarter. It would make us better citizens, better consumers, better humans. Most people continue to think of tech companies as businesses that sell some product or service at a profit. Amazon just wants to sell you stuff, Facebook wants to show you advertising, and Apple wants you to buy the latest iPhone product. Your phone, your computer, and various apps and programs etc. come from just a few Silicon Valley overlords. Their companies want very much to shape what you do. They hope to automate the choices that we make everyday. Unrestrained power, no matter how well-meaning or alluring, is something of which we should always be wary.

Google, Facebook, Amazon use technology and data to sidestep traditional restrictions on monopoly power. Credit raters, search engines, major banks etc. collect data about us and convert it into scores, rankings, risk calculations, and watch lists which impact us daily. They use it to make important decisions about us and to influence the decisions we make for ourselves. A bad credit score may prevent a customer from accessing any loans but he will never understand exactly how it was calculated. A predictive analytics firm may score someone as a “high cost” or “unreliable” worker, yet never tell her about the decision. As Frank Pasquale writes in The Black Box Society:

Continuing unease about black box scoring reflects long-standing anxiety about misapplications of natural science methods to the social realm. A civil engineer might use data from a thousand bridges to estimate which one might next collapse; now financial engineers scrutinize millions of transactions to predict consumer defaults. 

But unlike the engineer, whose studies do nothing to the bridges she examines, a credit scoring system increases the chance of a consumer defaulting once it labels him a risk and prices a loan accordingly. Moreover, the “science” of secret scoring does not adopt a key safeguard of the scientific method: publicly testable generalizations and observations.  As long as the analytics are secret, they will remain an opaque and troubling form of social sorting.

The proprietary algorithms by which digital economy companies collect and analyse data of their customers are immune from scrutiny. Internet companies collect more and more data on their users but fight regulations that would exercise some control over the resulting digital dossiers. The conclusions they come to — about the productivity of employees, or the relevance of websites, or the attractiveness of investments — are determined by complex formulas devised by lots of engineers and guarded by many lawyers. The law is very protective of secrecy in the world of commerce but is increasingly silent when it comes to the privacy of persons.

Are these algorithmic applications fair? Why, for instance, does YouTube (owned by Google) so regularly beat out other video sites in Google’s video search results? How does one particular restaurant or auto stock make it to the top of the hit list while another does not? It’s anyone’s guess, as long as the algorithms involved are kept secret. Without knowing what Google actually does when it ranks sites, we cannot assess when it is acting in good faith to help users, and when it is biasing results to favor its own commercial interests. The same goes for status updates on Facebook, trending topics on Twitter, or recommendations on Amazon. Franklin Foer says in World Without Mind:

Computer scientists have an aphorism that describes how algorithms relentlessly hunt for patterns. They talk about torturing the data until it confesses. Yet this metaphor contains unexamined implications. Data, like victims of torture, tells its interrogators what it wants to hear. 

Google is the dominant way we navigate the Internet which makes it the primary lens through which we experience the world. Google has steadily added to the roles it plays in people’s lives – it hosts e-mail, it owns the free blog-hosting service Blogger, offers online software such as a word processor, spreadsheets, presentation software, and a calendar service, has its own Web browser called Chrome, its Google Books project has scanned millions and millions of volumes and has made many of them available online at no cost, it has Google Maps, Street View, and Google Earth, etc. As we shift more of our Internet use to Google-branded services such as Gmail and YouTube, Google is on the verge of becoming indistinguishable from the Web itself.

This gives it enormous power to set agendas and alter perceptions. But we should be cautious about putting our faith in the goodwill of an enterprise whose mission is “to organize the world’s information and make it universally accessible and useful,” We see so clearly how it makes our lives better, our projects easier, and our world smaller that we fail to consider the costs, risks, options, and long-term consequences of our over-reliance on it. Its biases (valuing popularity over accuracy, established sites over new, etc.) are built into algorithms and they have an affect how we value and perceive things. These may be legitimate choices by Google but one has to recognize that these are choices, not science. 

The tyranny of algorithms - I

[T]here is one world in common for those who are awake, but [when] men are asleep each turns away into a world of his own. — Heracleitus

It was assumed that the internet would level the playing field and invite new competition into markets that had always had high barriers to entry. Thoreau remarked that our inventions are but improved means to an unimproved end. Those who think they see clearly the direction in which a new technology will take us, especially the inventors of that technology, are blind to unforeseen consequences. In Technopoly, Neil Postman says that we are currently surrounded by people who see only the positives of a technology and are blind to its negatives. ‘They gaze on technology as a lover does on his beloved, seeing it as without blemish and entertaining no apprehension for the future. They are therefore dangerous and are to be approached cautiously.’

To illustrate his point about there being winners and losers in the development and spread of any technology, Postman writes about computer technology. It has increased the power of large-scale organizations like the armed forces, or airline companies or banks or tax-collecting agencies. But to what extent has computer technology been an advantage to the masses of people like steelworkers, vegetable-store owners,  garage mechanics, musicians, bricklayers, etc.? He writes:

Their private matters have been made more accessible to powerful institutions. They are more easily tracked and controlled; are subjected to more examinations; are increasingly mystified by the decisions made about them; are often reduced to mere numerical objects. They are inundated by junk mail. They are easy targets for advertising agencies and political organizations. 

[SNIP]

It is to be expected that the winners will encourage the losers to be enthusiastic about computer technology. That is the way of winners, and so they sometimes tell the losers that with personal computers the average person can balance a checkbook more neatly, keep better track of recipes, and make more logical shopping lists. They also tell them that their lives will be conducted more efficiently. But discreetly they neglect to say from whose point of view the efficiency is warranted or what might be its costs. 

Should the losers grow skeptical, the winners dazzle them with the wondrous feats of computers, almost all of which have only marginal relevance to the quality of the losers' lives but which are nonetheless impressive. Eventually, the losers succumb, in part because they believe. . . that the specialized knowledge of the masters of a new technology is a form of wisdom. The masters come to believe this as well . . . The result is that certain questions do not arise. For example, to whom will the technology give greater power and freedom? And whose power and freedom will be reduced by it? 

About 15-20 years ago, the Internet was promising a new era of transparency, in which open access to information would result in extraordinary liberty. Law professor Glenn Reynolds predicted that “an army of Davids” would overthrow smug, self-satisfied elites. But the powerful players in the worlds of business, finance, and search deployed strategies of obfuscation and secrecy to consolidate power and wealth. The fates of individuals, businesses, and even our financial systems are at the mercy of hidden databases and dubious correlations generated by ‘unbiased’ algorithms. It has always been the case that those who have control over the workings of a particular technology accumulate power and inevitably form a kind of conspiracy against those who have no access to the specialized knowledge made available by the technology.

Technology gurus of Silicon Valley are creating a new universal narrative that legitimises the authority of algorithms that manipulate huge amounts of data. Every day people take in huge amounts of data through emails, phone calls and articles; process the data; and transmit back new bits through more emails, phone calls and articles. According to Yuval Noah Harari, the motto of these data mavericks says: “If you experience something — record it. If you record something — upload it. If you upload something — share it.” Internet companies will claim that the more you tell them, the more they can help you.

A key trend in how the Internet is developing today is the drive toward the personalization of our online experience. Everything we click, read, search, and watch online is increasingly the result of some complicated optimization effort (which is immune from scrutiny), whereby our previous clicks, searches, “likes,” purchases, and interactions determine what appears in our browsers and apps. Internet companies want that endless array of data points to develop detailed profiles of the people who use them. Pattern recognition is the name of the game — connecting the dots of past behavior to predict the future. Every business wants a data advantage that will let it target its ideal customers. Since it costs us nothing we don't think twice about it.

But it is a myth that sharing data has no costs. Those who cultivate competence in the use of a new technology become an elite group that are granted undeserved authority and prestige by those who have no such competence. For every discount or shortcut big data may offer, it’s probably imposing other, hidden costs or wild goose chases. Your data is a source of huge profit to other people, but often at your expense. When we click on an ad promising a discount, there’s probably a program behind the scenes calculating how much more it can charge us on the basis of our location or whether we’re using a Mac or PC.  Recommendation engines at Amazon and YouTube affect an automated familiarity, gently suggesting offerings they think we’ll like. What finance firms do with money, leading Internet companies do with attention. They  direct it toward some ideas, goods, and services, and away from others.

Harari says that this reliance on Big Data to make important decisions will make authority shift from humans to algorithms. Big Data could then empower Big Brother. Given enough bio-metric data and enough computing power, external algorithms could know us better than we know ourselves, and then governments and corporations could predict our decisions, manipulate our emotions and gain absolute control over our lives. If and when artificial intelligence (AI) surpasses human intelligence, it may be given control of weapon systems and crucial decisions, with potentially calamitous consequences.