next up previous
Next: Plausible reasoning Up: Beyond the binding Previous: The imagination perception

If a picture is worth a thousand words, how many pictures is a word worth?

There has been an ongoing debate between British associationists and logical positivists in the philosophy world, on whether our minds use pictures or words for thinking. The same discussion continues today in the form of the imagery debate. Symbolic vs non-symbolic AI discussion is also a mirror of the same philosophical tradition.

The advocates of thinking with pictures say that a picture is worth a thousand words. There will always be details of a picture left that a symbolic description just fails to capture. There are always more inferences you can draw from a picture than from its description.

Other philosophers argued on the other hand, that it was impossible to represent disjunctions (a tall or a Chinese man), and negations (a room without a giraffe) with pictures. In fact, in general, it is impossible to represent sets of things. One can never visualize the concept of a triangle. Each time you try, you will see a specific triangle.

Another argument for symbols being the natural currency of thinking, comes from the way people store things in their long term memory. Without looking at a penny, try drawing the picture of one, seen from the head side. Typically people remember that there is a portrait of Abraham Lincoln from profile, somewhere it says ``in God we trust''. Maybe you even remember that the year and the word ``liberty'' are also on this side. Did you finish the drawing? Now find a real penny and compare the results. The mistakes people typically make are drawing Lincoln looking at left rather than right, putting ``in God we trust'' to the bottom rather than top, switching the places of the year and ``liberty''. No one draws a penny with the upper left side faded out, as you would expect from the degradation of a pixel representation.

This is a powerful argument for symbolic representations. However, it does not say anything about using images for doing inference, the imagination-perception loop. You may be reconstructing the image from its symbolic description, and this does not hinder the advantage of using images for certain kinds of inference. Furthermore, if you didn't have any pictorial information in your memory, you wouldn't be able to draw anything. This shows that our long term memory has to carry pictorial representations in addition to verbal ones. In fact, it can be argued that each representational system has its own long term memory. They each experiences the world within its own language, remember things from the past and learn from regularities within its own framework.

The answer to the question in the title of this section is not 1/1000. In fact it is a thousand also. This is an unusual situation. We are used to seeing ``many to one'' mappings. But typically this means the ``one'' set is much smaller than the ``many'' set. Imagine a situation where the relationship is symmetric. To completely capture the information in a picture, you need a lot of words. To completely capture what is represented by a particular concept, you need a lot of pictures. This is the power of multiple representational systems. One cannot substitute for the other. What you can say in a few bits in one, require infinite space in the other, and vice versa [Rao, 1996].



next up previous
Next: Plausible reasoning Up: Beyond the binding Previous: The imagination perception



Deniz Yuret
Tue Apr 1 21:26:01 EST 1997