What they conceal is vital - I

I don't read the newspaper and I am not a mathematician.  John Allen Paulos is a mathematician and reads plenty of newspapers so he is ideally suited to write A Mathematician Reads the Newspaper. His general reaction  can be summed up by a quote by John Lennon that he gives at the beginning of the book: "I read the news today. Oh boy."(Though he is  not as dismissive as Jim Hacker.)

But he admits that newspapers are still much better than T.V. news which are much more superficial. (An instance of bias in T.V. news that I remember is that if I had watched  only NDTV, I would never have known about the massive floods in Pakistan a couple of years ago which was the main news in the international channels. But I would have known about the fashion show in Delhi 'that everyone is talking about'.)

Apart from reviewing some of the mathematical errors that colour news reports, some psychological factors like availability and anchoring that affect the reportage are also discussed. Paulos cautions against believing the results of polls since those results can easily be skewed by the questions that are asked as demonstrated by Humphrey Appleby in an episode of 'Yes Prime Minister'. He also mentions some advertising shenanigans like giving erroneous graphs and figures. ( How would you like some 'Splenda'?)

There are many advertisements for superficial products like face creams, hair gels, fashion accessories, etc. with dazzling and irrelevant visuals and dialogues that exaggerate the attributes of the products.Then there are the fashion pages. Paolos comments wryly about clothes that seem unwearable by anyone but a model - 'Always risible are the claims of the "top designers" that these glitzy, outlandish concoctions are for the busy working woman.' 'These glitzy ads featuring glamorous models gushing over diamond jewellery and mouthing obviously untrue bromides like 'It doesn't matter where you are born' remind me of what Bassanio said before choosing the lead casket in The Merchant of Venice:

So may the outward shows be least themselves:
The world is still deceived with ornament.
In law, what plea so tainted and corrupt,
But, being seasoned with a gracious voice,
Obscures the show of evil? In religion,
What damned error, but some sober brow
Will bless it and approve it with a text,
Hiding the grossness with fair ornament?
There is no vice so simple but assumes
Some mark of virtue on his outward parts:
How many cowards, whose hearts are all as false
As stairs of sand, wear yet upon their chins
The beards of Hercules and frowning Mars;
Who, inward search'd, have livers white as milk;
And these assume but valour's excrement
To render them redoubted! Look on beauty,
And you shall see 'tis purchased by the weight;
Which therein works a miracle in nature,
Making them lightest that wear most of it:
So are those crisped snaky golden locks
Which make such wanton gambols with the wind,
Upon supposed fairness, often known
To be the dowry of a second head,
The skull that bred them in the sepulchre.
Thus ornament is but the guiled shore
To a most dangerous sea; the beauteous scarf
Veiling an Indian beauty; in a word,
The seeming truth which cunning times put on
To entrap the wisest.

Non-linear systems are very sensitive to initial conditions, often called the Butterfly effect. Sociological phenomena are non-linear phenomena with many interacting variables that have positive and negative feedback loops and are hazardous to predict. Paulos writes:

You should observe that the accuracy of social forecasts and predictions is vastly greater if the predictions are short-term rather than long-term; if they deal with simple rather than complex phenomena; with pairs of closely associated variables rather than many subtly interacting ones; if they're hazy anticipations rather than precise assertions; and if they are not colored by the participants' intentions. Note how few political and economic predictions meet the conditions of these "ifs" - those are the ones to take seriously.

In a long-term study Phillip Tetlock found that the confident predictions made by charismatic experts are often false. He describes his study in this talk. Another relevant talk is by Nassim Nicholas Taleb about Black Swan events.

PS: There is a BBC program that investigates the numbers in the news.

PPS: The 11 Ways That Consumers Are Hopeless at Math

Debugging gone awry

My previous post reminded me of another incident involving computers that I was involved in. It is not with pleasure that I look back at the changed computer, the floppy disk, the sleepless night, the copying disaster, the note on the door, the woebegone expression of Sisyphus...I see that you are fogged so I will start at the beginning which, I never tire of reminding you, was also told by the king of Alice's Adventures in Wonderland to Alice during her adventures in Wonderland.

During a computer course at IIMA, we  had to write some program. Different groups had different projects. We had completed writing most of the program and only the  tiresome debugging process was left. It was at this time that Fate and I had an entanglement which is, as Salman Rushdie says in Midnight's Children, '...at the best of times a dangerous sort of involvement'. In Leave it to Psmith, P.G. Wodehouse writes:

The fact that many writers in their time have commented at some length on the mysterious manner in which Fate is apt to perform its work must not deter us now from a brief survey of this latest manifestation of its ingenious methods.

In the normal course of events, there would have been an uneventful few hours in the computer room where we were working. But as luck would have it, the computer room where we were working developed some problem and we had to shift to another computer room in a neighbouring building to complete the project.

This meant that we had to copy the incomplete program to a floppy disk, copy it back from the floppy disk to the hard disk of the new computer and finally copy the completed program back onto the floppy disk. (Those of less ancient vintage will have to visit The Museum of Endangered Sounds to hear 'the strained buzzing of a floppy disk drive'.) At that time we had to type in DOS commands to achieve these tasks. Typing 'copy c: a:' meant 'copy from hard disk to floppy disk' and typing 'copy a: c:' meant 'copy from floppy disk to hard disk'. (I hope I remember the commands right.This whole post is because of getting the command wrong.)

We got the first two activities right and the incomplete program was in the hard disk of the new computer by about 11 p.m. Over the next 5 hours or so. we were hard at work dotting the i's and crossing the t's of the code. Sometime after 4 a.m., we decided that the program was functioning reasonably well and that we couldn't improve it further. All that was left  was to copy  the completed program onto the floppy. You would have thought that a couple of MBAs-to-be from WIMWI would be able to manage that, right? (In the management cases that we were given at IIMA, the protagonist was often a graduate of WIMWI - Well-known Institute of Management in Western India.)

Blame it on lack of sleep, blame it on over confidence, blame it on what you will, but we muffed it up big time. We quickly typed in 'copy a: c:' and pressed ENTER. (Note the diplomatic 'we'.) The result was obvious: the partially completed program on the floppy disk was copied onto the hard disk and the completed program on the hard disk disappeared into the ether. We didn't know about this disaster at the time and congratulated each other on a job well done.

I returned to the hostel to catch up on some sleep before classes for the day began.Something attempted, something done had earned a night's repose. Actually it was almost morning and the early birds were getting ready to hassle the hapless worms. (The old saw about the early bird just goes to show that the worm should have stayed in bed. - Heinlein) My friend had some other work and said that he will return to the hostel later.

When I woke up after a couple of hours sleep, I saw a Post-it note attached to the door with a message from my friend asking me if I could please come to his room ASAP. (Since this is a family blog, I have refrained from mentioning the friendly salutations that preceded this request which seemed to indicate that he was a trifle agitated.) I sauntered across to his room harbouring nothing more than mild curiosity.

He wasted no time in telling me the truth, the whole truth and nothing but the truth - our efforts of the night had gone waste. I think he had tried to show the program to someone and found to his horror that it was a total mess. He put two and two together and guessed that we had been prized chumps. I was like the cartoon character who runs past the edge of a cliff at full speed, keeps running for some time, suddenly looks down, finds that there is nothing beneath him and falls straight down. In the Wodehouse novel Mike and Psmith, there is a description of the teacher Mr. Downing when he is overwrought:

In all times of storm and tribulation there comes a breaking point, a point where the spirit definitely refuses to battle any longer against the slings and arrows of outrageous fortune. Mr.Dowing could not bear up against this crowning blow.  He went down beneath it.  In the language of the ring, he took the count. It was the knockout.

I similarly took the count. There followed a period of silence of the type that I have often seen writers describe as being pregnant. Then, as such disasters are usually the fault of the other guy, we appraised each other of our perception of the other's intellectual capabilities, the sort of discussion that is known in diplomatic circles as 'a frank exchange of views'.

After thus venting our frustrations, we took the only decision that was on the table: do the damn thing again. As I returned to my room,I had the expression that Sisyphus must have had when he sat at the bottom of the mountain glaring sourly at that accursed rock that he had to roll back up the slopes.

We headed back to the computer lab in the afternoon after classes. Sometime before midnight, we finished fixing the program and the moment of reckoning arrived. We carefully typed 'copy c: a:' and asked folks around to have a look. After we received the all clear, we pressed the ENTER key as if it was the proverbial nuclear button. Then we checked the floppy disk. Everything seemed to be all right. And what sighs of relief there were my countrymen!

Oscar Wilde said, "Experience is simply the name we give our mistakes." I gained so much experience writing the program that I never wrote one again. It sounds amusing now but at the time, it was anything but funny.

Selling 'solutions'

Sometime back, I tuned in to CNBC after many years. There were familiar and new faces talking about dimly familiar things like volatility of beta stocks and open interest positions, things which are now outside my complexity horizon. One of the last times I had listened to it, there were huge celebrations about the Sensex having crossed 20,000  and there were breathless discussions about how long it will take for the Sensex to reach 30,000 or maybe even 40,000. So it went to 10,000.

It was reminiscent of the title of a book that I have not read and prompts me to make a Hirohito comment: The expectations were a trifle optimistic! (When announcing Japan’s  surrender in 1945, Emperor Hirohito famously explained his decision as follows: “The war situation has developed not necessarily to Japan’s advantage". I got that from a post  in Paul Krugman's blog.)

I wondered how much they really knew about what they were talking. Perhaps they were groping in the dark like I was when I was trying to sell computers. During the campus interview at IIMA, I got a job in sales in Wipro Infotech. My knowledge of the innards of a computer was not much more than that of the boss in Dilbert comics. (In one Dilbert comic strip, the boss was giving specifications about a computer to be purchased for his office. When he was asked, 'What about the RAM?', he replied, 'Make it red.' In How the Mind Works, Steven Pinker writes:                                        

And then along came computers: fairy-free, fully exorcised hunks of metal that  could not be explained without the full lexicon of mentalistic taboo words. "Why isn't my computer printing?" Because the program doesn't know you replaced your dot-matrix printer with a laser printer. It still thinks it is talking to the dot-matrix printer and is trying to print the document by asking the printer to acknowledge its message.  But the printer doesn't understand the message; it's ignoring it because it expects its input to begin with '%!' The program refuses to give up control while it polls the printer, so you have to get the attention of the monitor so that it can wrest control back from the program. Once the programme learns what printer is connected to it, they can communicate ." The more complex the system and the more expert the users, the more their technical conversation sounds like the plot of a soap opera.

I was assured that all these problems will be sorted out during a training session in Bangalore. During training, I listened to the speeches of the great and the good, learned some useful trade lingo (eg. 'we sell solutions not boxes') and grappled with disk striping, RAID technology , handshaking protocols, superpiplining and other fearsome beasts. After this ego restructuring exercise, I was let loose on an unsuspecting Mumbai market. The man who has a quote for all occasions (a woman who knew nothing about Shakespeare went to see a production of Hamlet and came out at the interval saying, 'It is full of  quotations!') explains my predicament via Portia in The Merchant of Venice:

 If to do were as easy as to know what were good
 to do, chapels had been churches and poor men's
 cottages princes' palaces.

In The Periodic Table, Primo Levi writes of the the time when he had to do the work of Customers' Service (CS):

... When it falls to me to work in CS, at the office or traveling, I do it unwillingly, with hesitation, compunction, and little human warmth.  Worse: I tend to be brusque and impatient with customers who are impatient and brusque, and to be mild and yielding with suppliers who, being in their turn CSs, prove to be just that, yielding and mild.  In short, I am not a good CS, and I fear that by now it is too late for me to become one.

I was similarly like a fish out of water in the matter of conning (oops, convincing) people that I was selling them a fantastic product. (I also had to be careful not to be as convincing as Amitabh Bachchan.) Mark Twain said, 'All you need in this life is ignorance and confidence, and then success is sure.' I had the ignorance all right but I was lacking a bit in the confidence department.

I reached out to hard-nosed EDP managers and tried to convince them  that Wipro products were a bargain buy. I read various computer magazines and  vomited out the stuff in them. I used liberally the technical terms in them to, as George Orwell wrote, ' give an appearance of solidity to pure wind.' My sales talk would have been described by one of  my school teachers as 'a diarrhoea of words and a constipation of thoughts'. (I have conveniently forgotten whether the compliment was for my essay or someone else's.)

I was surprised that my attempts to imitate Gratiano of The Merchant of Venice in speaking 'an infinite deal of nothing' went largely unchallenged. This  was the time when it really sunk in that an IIMA degree has  significant social cachet. I  could rely on the halo effect to shield me from awkward questions. This was also the time when I had frequent episodes of the Imposter Syndrome. (I would have been more relaxed if I had known at that time that Darwin also had similar periods.)

It wasn't a surprise when I quit my job some months later and joined the financial services sector. This job was more up my alley - sitting in  an A/C room and manipulating numbers in an Excel spreadsheet. (I was not a Quant so the world economy was not in any danger.)

PS: Here is another type of quant. I suppose this is the kind of guy known as a datasexual.

Religious superstitions: to criticize or not to criticize? That is the question

“It’s really interesting that wherever religion is on the front foot, it bears down in a very impressive [way] on people. Look at the Taliban. Wherever they’re on the back foot, they suddenly become very friendly, very concessive and very tolerant. And that’s where they should be, very firmly on the back foot.” - A.C. Grayling

As far as I know, all cultures that have been discovered so far have had some form of religion. Many people find comfort from their religious beliefs especially in times of great personal tragedies. I know many people who were able to deal with the misfortunes that befell them only because of their faith. Only some privileged people have the time to ponder over questions about god. Most people are too busy trying to make ends meet to bother about such questions. In this episode of Point of Inquiry about cheating, Dan Ariely points out some surprising benefits that religion may be providing.

Maybe there is a human need for rituals and ceremonies.Witness the elaborate ceremonies around the Olympic flame. I tend to get bored with the opening and closing ceremonies of sporting events but I am obviously a mutant. Most people seem to enjoy these events. Graduation ceremonies will lose something if students did not wear those funny hats and coats. We are all irrational at various times. Actually the world would be quite a boring place if everyone was rational all the time. (Here is an interesting lecture by Robert Sapolsky on the evolution of religious rituals.)

The relationship between religiosity and intelligence is confusing. Einstein thought that religion was a psychological 'prop'. But take Francs Collins. You can't get much more smart than him but he has weird views about religion. Or consider the case of the Hitchens brothers - two very smart people with opposite views about god. Christopher Hitchens was as strident as they come in criticizing religion while his brother Peter Hitchens is a believing Christian who abhors his brother's views. Or take the case  of religious experiences of astronauts. (But there are doubts about how spontaneous some of these experiences were.)

It is fallacious to think that reason can always trump belief. Religion makes emotional appeals to fear, hope, tradition, etc. and talking about the double helix or the Big Bang often doesn't produce results. Man is a social animal and it  is indisputable that religion provides plenty of opportunities for like-minded people to meet and interact with each other. Most people are content being what Kierkegaard called an 'automatic cultural man' who is described in The Denial of Death by Earnst Becker (a typical example is depicted in Nissim Ezekiel's poem The Professor):

...man as confined by culture, a slave to it, who imagines that he has an identity if he pays his insurance premium, that he has control of his life if he guns his sports car or works his electric toothbrush....For Kierkegaard "philistinism" was triviality, man lulled by the daily routines of his society, content with the satisfactions that it offers him: in today's world the car, the shopping center, the two-week summer vacation.  Man is protected by the secure and limited alternatives his society offers him, and if he does not look up from his path he can live out his life with a certain dull security:

Devoid of imagination, as the Philistine always is, he lives in a certain trivial province of experience as to how  things go, what is possible, what usually occurs..... Philistinism tranquilizes itself in the trivial...

Does all this mean that we should close our eyes when we encounter religious superstitions and treat them with respectful silence since the majority of them seem to be harmless? Is it nobler in the mind to suffer them with a patient shrug or by opposing them reduce their virulence? Carl Sagan ponders this question in The Demon-Haunted World:

Clearly there are limits to the uses of skepticism.  There is some cost-benefit analysis which must be applied, and if the comfort, consolation and hope delivered by mysticism and superstition is high, and the dangers of belief comparatively low, should we not keep our mis-givings to ourselves? But the issue is tricky.  Imagine that you enter a big-city taxicab and the moment you get settled in, the driver begins a harangue about the supposed iniquities and inferiorities of another ethnic group.  Is your best course to keep quiet, bearing in mind that silence conveys assent? Or is it your moral responsibility to argue with him, to express outrage, even to leave the cab - because you know that every silent assent will encourage him next time, and every vigorous dissent will cause him next time to think twice? Likewise, if we offer too much silent assent about mysticism and superstition - even when it seems to be doing a little good - we abet a general climate in which skepticism is considered impolite, science tiresome, and rigorous thinking somehow stuffy and inappropriate. Figuring out a prudent balance takes wisdom.

Meera Nanda also makes several good points. It is not my contention that everybody should become a Dawkins or a Hitchens.It may be the case that just like there needs to be a balance between risk takers and followers, there perhaps needs to be a balance between those who believe in god and those who don't in order to organise masses of people to achieve a common goal. (I may be wrong here. The Scandinavian countries have some of the lowest rates of religious belief in the world but they consistently top the Human Development Index.)

It would be disingenuous to suggest that Dawkins and Co. are not aware of what Sam Harris calls 'The Fireplace Delusion'.  Dawkins has often said that even when he is debating in front of believers, he is not trying to  convince them. He is hoping that his message that religion does not deserve special privileges is heard by people who are sitting on the fence, people 'who didn't even know there was a fence to sit on' as he put it in the BBC series The Life Scientific. As Eric Macdonald says, '...while it may be true that Dawkins, in Spufford’s words, knows “sod-all about religion,”* it is also true that most religious believers know even less.' And that is because believers accept unquestioningly what their religious leaders say.

Why does the 'best culture' in the world consistently have such an abysmal rank in the Human Development Index? (Years of good economic growth have not had much impact on these figures. Something is rotten in the State of Denmark. I am sick of listening to gasbags going ga-ga over hot air.) What role does religion play in perpetuating inequality? Is religion a cause or an effect of poverty? Can religion be confined to the private realm or is its very nature such that it will intrude into the public sphere? These are questions that are worth discussing instead of always tiptoeing  carefully around the elephant in the room.

Atheists are not wasting their time. Somebody needs to ask the uncomfortable questions and push the envelop. Believers dislike the New Atheists because they are gadflies who keep pushing them out of their comfort zone and face the fact that the Emperor has no clothes. They don't parrot 'what everybody knows' which often has to be treated cautiously. They have helped shift the Overton Window. Converts' Corner is evidence that their arguments are having an impact.

Perennial deference to the prevailing zeitgeist doesn't produce change. As Salman Rushdie said in this interview with CNN-IBN, I am tired of religion constantly asking for privileges. Anything goes under the garb of 'right to religion' and firm voices need to be raised against pious thuggery instead of pusillanimous capitulation which is generally the case.Ayaan Hirsi Ali puts it bluntly, " At the heart of that alternative are the ideals of the rule of law and freedom of thought, worship, and expression. For these values there can and should be no apologies, no groveling, no hesitation."  If you wear the right religious uniform you can get away with anything  and this bluff needs to be called.