Sunday, March 4, 2012

Newell's twenty questions

In an earlier post, I had brought up the contrasting attitudes towards what constitutes physics, attributed to Rutherford and Feynman, respectively.
That which is not physics is stamp collecting. ~Ernest Rutherford.
In the haughty perspective of Rutherford, the primary concern of science was to construct explanations (theories) for observed phenomena (stamps), and it was only the physicist who would fit this role. Indeed, under this view, it would seem that all other sciences were sources of observations for physics to explain.
Physicists often have the habit of taking the simplest explanation of any phenomenon and calling it physics, leaving the more complicated examples to other fields. ~Richard Feynman.
In the more humble of perspective of Feynman, it seems that the fields outside of physics patiently investigate and characterize anomalies lying outside the realm of the most common instance of a given phenomenon. Their role could thus be viewed as fulfilling the business of collecting and describing rare stamps. Rare stamps when fed back into the activity of theory building (Rutherford's physics), would enable the development of simpler theories with greater predictive power.

The blind men and the elephant

Image courtesy

To understand the state of a field in progress, it is worth considering the Indian parable of the blind men and the elephant. The story goes that six blind men decide to understand how an elephant looks by touching it. Each manages to touch only a small part of the elephant. As a result each develops strong beliefs about the nature of the elephant. They liken the tusk to a spear, the trunk to a snake, the tail to a rope, the feet to a tree, the ears to a fan, and the torso to a wall. Since they are unable to perform further experiments, they end up debating the nature of the elephant ad nauseum.

The gist of the parable is that (1) partial and noisy observations of a system result in erroneous conflicting hypotheses, and (2) the hypotheses are at a stalemate because of the lack of right tools to perform further experiments.

Enter Newell

With this background, it is perhaps useful to pause and consider what Alan Newell had to say back in 1973. Newell is known, among many things, for advocating the mind as an information processing system, along with his advisor and Nobel laureate Herbert Simon. In a valedicatory talk titled, You cannot play 20 questions with nature and win, he attempted to characterize the field of experimental psychology, and as we shall see, his ideas are broadly applicable to the state of cognitive neuroscience today. It is interesting to note that at the time of this lecture, Ed Posner, the founder of the Neural Information Processing Systems (NIPS) conference was in the audience.

To understand the essence of his ideas, let us consider the metaphor of nature as a jigsaw puzzle.

Imagine that you have the pieces of a jigsaw puzzle face down. You don't know what the puzzle looks like, and you don't know how many pieces there are. To flip each piece, you need to do very rigorous experiments and verify the results carefully, multiple times. Each piece may be likened to a phenomenon of the mind, such as auditory short term memory, or one of its properties, such as how long a certain type of information resides in auditory short term memory. The act of spotting each new piece may be likened to an observation of the phenomenon, and the act of flipping it may be likened to the careful elucidation of its properties. Each of these acts is performed by a number of scientists working together or independently, over a number of years. Sometimes new phenomena are discovered; at other times, new properties are elucidated. At still other times, a deeper understanding of the phenomenon is gained, as Feynman explains here with a chess analogy.

Newell articulated beautifully that the journey from empirical exploration to unified theory is a complex one. Consider the following paragraph:

I stand by my assertion that the two constructs that drive our current experimental style are (1) at a low level, the discovery and empirical exploration of phenomena [...] and (2) at the middle level, the formulation of questions to be put to nature that center on the resolution of binary oppositions. At a high level of grand theory, we may be driven by quite general concerns: to explore development; to discover how language is used; to show that man (sic!) is a processor of information; to show that he (sic!) is solely analysable in terms of contingencies of reinforcement responded to. But it is through the mediation of these lower two levels that we generate our actual experiments and give our actual explanations. Indeed, psychology with its penchant for being explicit about its methodology has created special terms, such as "orienting attitudes" and "pretheoretical dispositions," to convey the large distance that separates the highest levels of theory from the immediate decisions of day to day science.

Newell's attitude was as follows. One half of him was very excited about the fact that multiple new pieces are being discovered all the time. But another half of him was concerned that if the trend of upturning new jigsaw pieces was to be extrapolated into the future, the field would be nowhere closer to seeing how the pieces fit.

Consider this statement:

Science advances by playing twenty questions with nature. The proper tactic [hyperlink, mine*] is to frame a general question, hopefully binary, that can be attacked experimentally. Having settled that bits-worth, one can proceed to the next. The policy appears optimal--one never risks much, there is feedback from nature at every step, and progress is inevitable. Unfortunately, the questions never seem to be answered, the strategy does not seem to work.
*This is a reference to another outstanding opinion piece from the 1960s called Strong Inference by John Platt. He stresses the importance of alternate hypotheses and systematically ruling out one of the two, with examples from theoretical physics and molecular biology.

After commending a selection of outstanding individual studies for example, he states:

What I wanted was for these excellent pieces of the experimental mosaic to add up to the psychology that we all wished to foresee. They didn't, not because of a lack of excellence locally, but because most of them seemed part of a mosaic of psychological activity that didn't seem able to cumulate.

If you replace experimental psychology with cognitive neuroscience, and 1973 with 2012, Newell's assessment would still ring true.

In a subsequent post, I will get around to analyzing Newell's recommendations to get unstuck, and how they could apply to the state of modern cognitive neuroscience.