Mar 10, 2009
Originally posted on Half an Hour, March 10, 2009.
The Network Phenomenon: Empiricism and the New Connectionism
Stephen Downes, 1990
(The whole document in MS-Word)
TNP Part V Previous Post
VI. The Problems of Perception
A. The veracity of Experience
Let me turn now to the problems concerning perception which were outlined above. These problems were then introduced as problems concerning the theory-ladenness of perceptions. In order to address this question, it is useful to begin with more traditional problems of perception. These divide roughly into two categories: first, can our perceptions be mistaken, that is, is the world really the way we perceive it to be; and second, can we be mistaken about our perceptions, that is, do we really perceive what we think we perceive? By means of a discussion of these questions I put into place the theoretical structure necessary to consider in more detail a response to concerns regarding theory-laden perceptions.
Philosophers have historically attached the veracity of experience. Plato, in the Republic, employs an analogy of shadows on the wall of a cave to make this point. [40] The way the world appears to us is represented by the shadows, but of course, if we want to see the way the world really is, we must leave the cave. Descartes, in Meditations, begins his sceptical argument with an attack on the senses. We can be fooled by the senses, argues Descartes, and if we can be fooled by them, then we should not place our trust in them. [41] Contemporary empiricists, for example Ayer, considered this to be a problem. [42]
There are two approaches to this argument. First, we can argue that the premise, that we are fooled by the senses, is false. Second, we can argue that the argument is irrelevant from the point of view of cognition; even if we are fooled by the senses, we nonetheless rely on the senses.
The first response is provided by Austin. [43] He considers examples of cases in which we are "fooled" by the senses and argues that we are not in fact fooled. For example, a stick appears bent when placed in water. We know that the stick is not bent., therefore, it appears as though we are fooled by our senses. However, nobody is actually fooled by this experience. For one thing, when we perceive a stick in water, we perceive not only the stick, but also the water. Since we expect a straight stick to appear bent when in water, we are not fooled into believing that the stick is bent. And for another thing, even had we never seen a stick in water, we would have seen the stick before and after it was placed in water. Our evaluation of the shape of the stick is based on all of these observations, not only the observation of the stick in the water.
Both of these responses make the same point. We cannot evaluate the veracity of any experience by considering only that experience in isolation. We must consider that experience within the totality of experiences, and experience as a whole as occurring within a given context, for a given person. And when we consider our experiences as a whole it turns out that we are not fooled by out experiences. What happens is that when we have apparently conflicting experiences, we revise our understanding of what we have seen in order to explain the experiences. This process of revision is easily explainable in terms of a connectionist system; it is a process of readjusting weights in order to reduce error.
The reason why it is arguable that questions regarding the veracity of experience are not relevant to a study of cognition is that in a certain sense it cannot be argued that we are wrong about our experiences. Now here it is important to be careful, for I do not wish to assert some sort of infallibility thesis. What I mean to say is that questions concerning the correctness or incorrectness of experience have no bearing on whether or not a given experience is experienced. Let me outline the argument before giving details. The infallibility thesis states that we cannot be wrong when we think we are experiencing, say, red. This can be criticized on the grounds that our perception of what we experience can be distinct from what we actually experience. [44] In response, on the one hand, we can say that the experience is whatever it is, and what we think about it is irrelevant, or on the other hand, what we think about the experience is what it is, and what the actual experience is is irrelevant. Arguments concerning the veracity of experience make sense only if we insist that there be some one-to-one correspondence between the actual experience and what we think we experience, and there is no reason to suppose that such a correspondence is the case.
Now let me proceed with this argument in more detail. Suppose I proceed by stipulation and assert that whatever constitutes the activation of input units in a given network is by definition experience. Either this input is in some way isomorphic with or representative of some external reality, or it is not. Sceptics about experience (such as Plato and Descartes) assert that it is not. However, whether or not the input is isomorphic with or representative of some external reality has no bearing on whether the input is input. No matter what the relation between the input and the external work, there is some state of activation of input units, and that state is called "experience", and the argument that this state of activation is somehow "wrong" does not change the fact that it exists.
The infallibility thesis rests on the assertion that one cannot be wrong about the content of one's own perceptions. In one sense, this is meaningless; if one's perceptions are a part of oneself, then how can a person be wrong about them? In order to make the argument against the infallibility thesis work, it is necessary to separate the person from the experience, or at the very least, to separate one's consciousness from the experience. Some philosophers, for example Hume in the Treatise, argue that this cannot be done. If, however, we can somehow separate the two, then it is in principle possible for a person to be wrong about his perceptions. But here we must step carefully: what does it mean to separate a person or a consciousness form that person's experience?
We must have two "experiences": one which is the actual input, and one which is a representation of that actual input. Arguments against the infallibility thesis are arguments that the representation of the experience is not isomorphic with or representative of the actual experience, which has some sort of "reality" of its own. But if this "reality" is outside our consciousness, then it cannot be considered to be a part of experience; our experience per se is our consciousness of this reality, and about that we cannot be mistaken. And our consciousness of an experience must be whatever it is, no matter what the "reality" of the experience is. (Please notice that I am arguing by analogy of form here; this argument has exactly the same form as the argument concerning experience versus reality a few paragraphs above.)
In my opinion, the major questions concerning perception and experience are questions regarding what counts as a perception or an experience. Above, I stipulated that the activation of input units is what counts as experience. This needs to be qualified. For if what I have just asserted immediately above is true, then since we are not consciously aware of the actual input activation, then the actual input activation cannot be considered to be experience. What we need to do is distinguish different areas of the network. Let me, quite arbitrarily, divide the network into two parts: that part of the brain the processes of which are consciously accessible to us; and that part of the processes which is not accessible to us. Then I can employ the definition of "experience" given above in the following consistent manner. There are two types of experience: on the one hand, "real experience", which is the input activation to the network as a whole; and "conscious experience" which is the input of the consciously accessible part of the network.
In human brains, my divisions work roughly as follows. The activation of, say, rods and cones on the retina in the eye constitute "real" experience. We are not consciously aware of these activations. These activations spread through the network and eventually reach the cerebral cortex. Then, since the activations from the initiative activations first in the V-IV section of the cerebral cortex, the activation of V-IV cells constitutes "conscious experience". The precise division of conscious and non-conscious areas, as well as input and non-input areas, is subject to empirical investigation. It is worth noting that at the other end, perhaps, say, the cerebellum, we are again unaware of states of activation.
B. The Nature of Experience
Now let me turn to the questions concerning the nature of experience. It is arguable, as I have mentioned above, that there are no "pure" perceptions, that is, that there is no means of distinguishing between the experiential and the non-experiential components of experience. What we believe about the world, or what theories govern our beliefs about the world, are a part of and indistinguishable from what we see or observe. In order to conduct this discussion, I will review the arguments concerning the theory-ladenness of perception and then, drawing on the lessons of the previous section, provide a reasonable and empirical explanation of such phenomena.
There are two ways to argue for the theory-ladenness of perception. First, it is arguable that one's beliefs or other cognitive state can affect perception. And second, it is arguable that there are innate constraints which affect perception.
The examples given above are examples of the first. Consider, for example, Hanson's argument. he suggests that when Ptolemy, who believes that the Sun revolves around the Earth, and Kepler, who believes that the Earth revolves around the Sun, look to the east at dawn, they see two different things. Ptolemy sees the Sun rising above the horizon, while Kepler sees the horizon turning to face the Sun. [45] This is an instance of one's beliefs about the world affecting the way we see the world. Churchland [46] makes a similar point quite vividly. If we view the horizon as level, the stars just look like, well, stars. If, however, if we (in the northern hemisphere) tilt our heads, and if we are aware that the planets orbit the Sun on a certain plane, then we can see this plane and further we can see that we are on the side of a planet on this plane. This really is a most vivid experience.
On the other hand, one may argue that there are innate constraints which affect the way we perceive the world and which constitute some item of knowledge about the world. According to Marr, for example, in order to construct three dimensional representations of the world from two dimensional input, as is for example received at the retina, we must be guided by two constraints: the uniqueness constraint and the continuity constraint. [47] In order to construct three dimensional representations, we need to match a given input from one eye to a given input from the other. The uniqueness constraint asserts that one thing cannot be in two places at the same time, thus, we match the input from one eye to the one and only input from the other. The continuity constraint asserts that we can't see through objects, so adjacent inputs tend to be considered to be at the same depth.
It is possible to create three-dimensional representations by employing these two constraints and a network as described above. The matrix consists of possible matchings between input from each eye. The two constraints represent inhibitory and excitatory connections in this matrix. For any given input from a given eye, the possible machine matches with inputs from the other eye are inhibited with respect to each other; we want to pick only one such match. Matches which represent the same depth excite each other; we think that objects have relatively continuous surfaces. The reason why these constraints must be innate, it is argued, is that there is nothing in the input from the eyes which, even given the learning processes described above, would result in this pattern of connectivity. This it must be build-in. See figure 12.
Let me now respond to these considerations. If anything like back propagation is correct, then we have a reasonable explanation for both types of theory-ladenness. We continue to define experiences as above, that is, as the activation of input units. This initial activation in turn causes the activation of other units. Patterns and association are detected by the system. Once these patterns are detected, they provide grounds for correcting various input activations. Input to the eye is "clamped", that is, it is fixed and cannot be changed. So real experience is unchangeable. However, conscious experience is not clamped. It is initiated by the activation of real input units, and this input is filtered and modified by the pattern of connections between the real input units and the conscious input units. This pattern of connectivity can be altered via back propagation. Thus, our conscious experience can be altered by previous experience. See figure 13.
This is an extremely powerful theory. First, it explains why we are not fooled by appearances. As mentioned above, we are not fooled because we take into consideration the totality of our experiences. Since previous experience creates a pattern of connectivity in the brain, this previous experience is effectively stored and plays an active role in our processing of subsequent experience. So when we see, say, a bent stick, we take into account the totality of our experiences of that stick and, as Austin suggests, are not fooled by the illusion. Second, it explains how beliefs and other cognition can affect what we consciously see. Once again, since beliefs are in pact patterns of connectivity, these patterns, via back propagation, can affect the connections between real input and conscious input, and hence, affect the states of activation of conscious input neurons.
There is empirical evidence that something like back-propagation may be at work here, Past experience plays a role in present perception. We can pick out objects when thy resemble objects we have seen before, such as for example picking out a dalmatian in a birch forest. We can also pcik out objects, such as birds flying in the dark, by their motions. [48]
Third, it explains Marr's uniqueness and continuity constraints. The particular patterns of connectivity described emerge because, if they did not, visual input would clash with tactile input. we know, for example, that surfaces are solid because we can't put our hands through them. Our experience of touching walls and the like creates a certain pattern of connectivity. In order to minimize error, it is necessary to adjust the connections in the three-d vision matrix to fix this pattern.
This last claim is something that can be tested empirically. In essence, what I am asserting is that other senses affect how we see. We can either proceed with biological experiments, testing the effect of sensory deprivation on visual processing, or we can proceed computationally, testing for improvements in visual processing as a consequence of the addition of other sensory modalities.
This theory can also explain why it is impossible to distinguish the theoretical component of conscious experience from the visual component. Suppose "i" is a unit at the level of conscious input. Then "i" is receiving excitatory or inhibitory input from both visual cells, such as those in the retina, and from higher level cognitive cells in the form of back propagation. These input activations are summed, so that is the retinal cells are sending a signal of '4' and the cognitive cells are sending a signal of '5', the net input to "i" is '9'. This is what shows up in "i"; it increases activation by 9. Once, however, the input is summed, there is no means of determining which part of the sum is the retinal input and which part is the cognitive input. Given only '9', we can't tell whether it is '3'+'6', '5'+'4', or whatever.
C. A Theory of Perception
I have, in the course of responding to these various objections, sketched the outline of a theory of perception, which I would now like to make clear.
Our conscious experience just is the activation of a certain set of cells. These cells are the input cells to whatever network constitutes the conscious part of the brain, that is, the part of the activities of which we are aware. Input to these input cells comes from two sources: on the one hand, from input cells in the various of our sensory modalities, for example, retinal cells; and on the other hand, via back propagation from cells which are at higher levels, which I have called cognitive cells. These inputs are summed to produce activation, and that activation is what we call conscious experience.
Input activation produces what we call a representation. Above, I have explicitly endorsed what is called the "picture" theory of representation. The term "picture" is a bit misleading, since as input activation may be augmented and corrected from back propagation from previously formed associations and other sensory modalities, and there is no requirement that a given representation be isomorphic with or represent input activation. It is not necessary, and rather unlikely, that representations are composed of symbols and sentences. Human beings do not appear to be constrained by the sort of principles which constrain symbolic representations, rather, what appears to be the case is that representations are constructed and manipulated according to their relevant similarity with other representations.
I would like to briefly address the categorization of the different regions of human neural networks which I have identified above. I suggest that there are no fixed delineations regarding what counts as the conscious network or what counts as the input part of the conscious network. It seems reasonable to believe that the exact boundaries of these regions may vary from person to person, and indeed, may vary over time in a single person. Therefore, in a precise sense, there is no set of necessary and sufficient conditions which define what counts as an "experience" and a "representation". Let me describe a well-known phenomenon which substantiates this claim.
Professional athletes, for example, skiers, practice their sport by "imagining" or "visualizing" perfect performances. [49] It is worth noting, first, that the are able to do this, and second, that their performance improves as a result. For the most part, further, this is not an ability they have had from birth; they must be trained, and then practice, mental imagery. What I believe they are doing is learning to stimulate activation of conscious input cells according to previously learned association. In a sense, this requires extending the conscious region of the brain, for not everybody has the ability to vividly "picture" a sequence of events.
In my own case, I am quite convinced that this is a skill which can be learned, for over the course of the last two or three years I have practiced it myself. When I started, I had no such skills, yet now, I am able to evoke vivid mental images. I am able to control the subject matter of these images, but I prefer to allow them to occur at random. It really is an extraordinary experience and I recommend it for fun and relaxation. [50]
TNP Part VI Next Post
[40] Plato, Republic, p. 168.
[41] Rene Descartes, The Philosophical Works of Descartes (Haldane and Ross, eds.) Vol. 1, p. 145.
[42] A.J. Ayer, The Problem of Knowledge, p. 37.
[43] J.L. Austin, Sense and Sensibilia, ch. III.
[44] Here I am thinking of arguments such as Armstrong's SEEG argument. Pollock also argues this way. See John Pollock, Contemporary Theories of Knowledge, p.34.
[45] N.R. Hanson, Patterns of Discovery, p. 5. See also Thomas Kuhn, The Structure of Scientific Revolution, p. 150.
[46] Paul Churchland, Scientific Realism and the Plasticity of Mind, Ch. 1.
[47] See D. Marr and T. Poggio, "A Computational Theory of Human Stereo Vision", Science 194, pp.283-7, 1976. Also see P. Johnson-Laird, The Computer and the Mind, pp. 86-87.
[48] See Johnson-Laird, The Computer and the Mind, pp. 100-110. He describes this as
'top-down' processing. Typically, when people think of top-down processing, they think of pre-established or innate knowledge. But there is no reason to suppose this; it could just as easily and more consistently be the result of back propagation.
[49] Arnold Lazarus, In the Mind's Eye, p. 30.
[50] It is worth noting that when I started this, I tried to visualize a circle as though it were really in front of me. I had almost no success for about six months, Quite by accident, however, I once visualized a baseball, which of course appears as a circle. Since then, I have had no trouble producing circles. I start with a circle I have actually seen, then allow it to become an outline form.
The Network Phenomenon: Empiricism and the New Connectionism
Stephen Downes, 1990
(The whole document in MS-Word)
TNP Part V Previous Post
VI. The Problems of Perception
A. The veracity of Experience
Let me turn now to the problems concerning perception which were outlined above. These problems were then introduced as problems concerning the theory-ladenness of perceptions. In order to address this question, it is useful to begin with more traditional problems of perception. These divide roughly into two categories: first, can our perceptions be mistaken, that is, is the world really the way we perceive it to be; and second, can we be mistaken about our perceptions, that is, do we really perceive what we think we perceive? By means of a discussion of these questions I put into place the theoretical structure necessary to consider in more detail a response to concerns regarding theory-laden perceptions.
Philosophers have historically attached the veracity of experience. Plato, in the Republic, employs an analogy of shadows on the wall of a cave to make this point. [40] The way the world appears to us is represented by the shadows, but of course, if we want to see the way the world really is, we must leave the cave. Descartes, in Meditations, begins his sceptical argument with an attack on the senses. We can be fooled by the senses, argues Descartes, and if we can be fooled by them, then we should not place our trust in them. [41] Contemporary empiricists, for example Ayer, considered this to be a problem. [42]
There are two approaches to this argument. First, we can argue that the premise, that we are fooled by the senses, is false. Second, we can argue that the argument is irrelevant from the point of view of cognition; even if we are fooled by the senses, we nonetheless rely on the senses.
The first response is provided by Austin. [43] He considers examples of cases in which we are "fooled" by the senses and argues that we are not in fact fooled. For example, a stick appears bent when placed in water. We know that the stick is not bent., therefore, it appears as though we are fooled by our senses. However, nobody is actually fooled by this experience. For one thing, when we perceive a stick in water, we perceive not only the stick, but also the water. Since we expect a straight stick to appear bent when in water, we are not fooled into believing that the stick is bent. And for another thing, even had we never seen a stick in water, we would have seen the stick before and after it was placed in water. Our evaluation of the shape of the stick is based on all of these observations, not only the observation of the stick in the water.
Both of these responses make the same point. We cannot evaluate the veracity of any experience by considering only that experience in isolation. We must consider that experience within the totality of experiences, and experience as a whole as occurring within a given context, for a given person. And when we consider our experiences as a whole it turns out that we are not fooled by out experiences. What happens is that when we have apparently conflicting experiences, we revise our understanding of what we have seen in order to explain the experiences. This process of revision is easily explainable in terms of a connectionist system; it is a process of readjusting weights in order to reduce error.
The reason why it is arguable that questions regarding the veracity of experience are not relevant to a study of cognition is that in a certain sense it cannot be argued that we are wrong about our experiences. Now here it is important to be careful, for I do not wish to assert some sort of infallibility thesis. What I mean to say is that questions concerning the correctness or incorrectness of experience have no bearing on whether or not a given experience is experienced. Let me outline the argument before giving details. The infallibility thesis states that we cannot be wrong when we think we are experiencing, say, red. This can be criticized on the grounds that our perception of what we experience can be distinct from what we actually experience. [44] In response, on the one hand, we can say that the experience is whatever it is, and what we think about it is irrelevant, or on the other hand, what we think about the experience is what it is, and what the actual experience is is irrelevant. Arguments concerning the veracity of experience make sense only if we insist that there be some one-to-one correspondence between the actual experience and what we think we experience, and there is no reason to suppose that such a correspondence is the case.
Now let me proceed with this argument in more detail. Suppose I proceed by stipulation and assert that whatever constitutes the activation of input units in a given network is by definition experience. Either this input is in some way isomorphic with or representative of some external reality, or it is not. Sceptics about experience (such as Plato and Descartes) assert that it is not. However, whether or not the input is isomorphic with or representative of some external reality has no bearing on whether the input is input. No matter what the relation between the input and the external work, there is some state of activation of input units, and that state is called "experience", and the argument that this state of activation is somehow "wrong" does not change the fact that it exists.
The infallibility thesis rests on the assertion that one cannot be wrong about the content of one's own perceptions. In one sense, this is meaningless; if one's perceptions are a part of oneself, then how can a person be wrong about them? In order to make the argument against the infallibility thesis work, it is necessary to separate the person from the experience, or at the very least, to separate one's consciousness from the experience. Some philosophers, for example Hume in the Treatise, argue that this cannot be done. If, however, we can somehow separate the two, then it is in principle possible for a person to be wrong about his perceptions. But here we must step carefully: what does it mean to separate a person or a consciousness form that person's experience?
We must have two "experiences": one which is the actual input, and one which is a representation of that actual input. Arguments against the infallibility thesis are arguments that the representation of the experience is not isomorphic with or representative of the actual experience, which has some sort of "reality" of its own. But if this "reality" is outside our consciousness, then it cannot be considered to be a part of experience; our experience per se is our consciousness of this reality, and about that we cannot be mistaken. And our consciousness of an experience must be whatever it is, no matter what the "reality" of the experience is. (Please notice that I am arguing by analogy of form here; this argument has exactly the same form as the argument concerning experience versus reality a few paragraphs above.)
In my opinion, the major questions concerning perception and experience are questions regarding what counts as a perception or an experience. Above, I stipulated that the activation of input units is what counts as experience. This needs to be qualified. For if what I have just asserted immediately above is true, then since we are not consciously aware of the actual input activation, then the actual input activation cannot be considered to be experience. What we need to do is distinguish different areas of the network. Let me, quite arbitrarily, divide the network into two parts: that part of the brain the processes of which are consciously accessible to us; and that part of the processes which is not accessible to us. Then I can employ the definition of "experience" given above in the following consistent manner. There are two types of experience: on the one hand, "real experience", which is the input activation to the network as a whole; and "conscious experience" which is the input of the consciously accessible part of the network.
In human brains, my divisions work roughly as follows. The activation of, say, rods and cones on the retina in the eye constitute "real" experience. We are not consciously aware of these activations. These activations spread through the network and eventually reach the cerebral cortex. Then, since the activations from the initiative activations first in the V-IV section of the cerebral cortex, the activation of V-IV cells constitutes "conscious experience". The precise division of conscious and non-conscious areas, as well as input and non-input areas, is subject to empirical investigation. It is worth noting that at the other end, perhaps, say, the cerebellum, we are again unaware of states of activation.
B. The Nature of Experience
Now let me turn to the questions concerning the nature of experience. It is arguable, as I have mentioned above, that there are no "pure" perceptions, that is, that there is no means of distinguishing between the experiential and the non-experiential components of experience. What we believe about the world, or what theories govern our beliefs about the world, are a part of and indistinguishable from what we see or observe. In order to conduct this discussion, I will review the arguments concerning the theory-ladenness of perception and then, drawing on the lessons of the previous section, provide a reasonable and empirical explanation of such phenomena.
There are two ways to argue for the theory-ladenness of perception. First, it is arguable that one's beliefs or other cognitive state can affect perception. And second, it is arguable that there are innate constraints which affect perception.
The examples given above are examples of the first. Consider, for example, Hanson's argument. he suggests that when Ptolemy, who believes that the Sun revolves around the Earth, and Kepler, who believes that the Earth revolves around the Sun, look to the east at dawn, they see two different things. Ptolemy sees the Sun rising above the horizon, while Kepler sees the horizon turning to face the Sun. [45] This is an instance of one's beliefs about the world affecting the way we see the world. Churchland [46] makes a similar point quite vividly. If we view the horizon as level, the stars just look like, well, stars. If, however, if we (in the northern hemisphere) tilt our heads, and if we are aware that the planets orbit the Sun on a certain plane, then we can see this plane and further we can see that we are on the side of a planet on this plane. This really is a most vivid experience.
On the other hand, one may argue that there are innate constraints which affect the way we perceive the world and which constitute some item of knowledge about the world. According to Marr, for example, in order to construct three dimensional representations of the world from two dimensional input, as is for example received at the retina, we must be guided by two constraints: the uniqueness constraint and the continuity constraint. [47] In order to construct three dimensional representations, we need to match a given input from one eye to a given input from the other. The uniqueness constraint asserts that one thing cannot be in two places at the same time, thus, we match the input from one eye to the one and only input from the other. The continuity constraint asserts that we can't see through objects, so adjacent inputs tend to be considered to be at the same depth.
It is possible to create three-dimensional representations by employing these two constraints and a network as described above. The matrix consists of possible matchings between input from each eye. The two constraints represent inhibitory and excitatory connections in this matrix. For any given input from a given eye, the possible machine matches with inputs from the other eye are inhibited with respect to each other; we want to pick only one such match. Matches which represent the same depth excite each other; we think that objects have relatively continuous surfaces. The reason why these constraints must be innate, it is argued, is that there is nothing in the input from the eyes which, even given the learning processes described above, would result in this pattern of connectivity. This it must be build-in. See figure 12.
Let me now respond to these considerations. If anything like back propagation is correct, then we have a reasonable explanation for both types of theory-ladenness. We continue to define experiences as above, that is, as the activation of input units. This initial activation in turn causes the activation of other units. Patterns and association are detected by the system. Once these patterns are detected, they provide grounds for correcting various input activations. Input to the eye is "clamped", that is, it is fixed and cannot be changed. So real experience is unchangeable. However, conscious experience is not clamped. It is initiated by the activation of real input units, and this input is filtered and modified by the pattern of connections between the real input units and the conscious input units. This pattern of connectivity can be altered via back propagation. Thus, our conscious experience can be altered by previous experience. See figure 13.
This is an extremely powerful theory. First, it explains why we are not fooled by appearances. As mentioned above, we are not fooled because we take into consideration the totality of our experiences. Since previous experience creates a pattern of connectivity in the brain, this previous experience is effectively stored and plays an active role in our processing of subsequent experience. So when we see, say, a bent stick, we take into account the totality of our experiences of that stick and, as Austin suggests, are not fooled by the illusion. Second, it explains how beliefs and other cognition can affect what we consciously see. Once again, since beliefs are in pact patterns of connectivity, these patterns, via back propagation, can affect the connections between real input and conscious input, and hence, affect the states of activation of conscious input neurons.
There is empirical evidence that something like back-propagation may be at work here, Past experience plays a role in present perception. We can pick out objects when thy resemble objects we have seen before, such as for example picking out a dalmatian in a birch forest. We can also pcik out objects, such as birds flying in the dark, by their motions. [48]
Third, it explains Marr's uniqueness and continuity constraints. The particular patterns of connectivity described emerge because, if they did not, visual input would clash with tactile input. we know, for example, that surfaces are solid because we can't put our hands through them. Our experience of touching walls and the like creates a certain pattern of connectivity. In order to minimize error, it is necessary to adjust the connections in the three-d vision matrix to fix this pattern.
This last claim is something that can be tested empirically. In essence, what I am asserting is that other senses affect how we see. We can either proceed with biological experiments, testing the effect of sensory deprivation on visual processing, or we can proceed computationally, testing for improvements in visual processing as a consequence of the addition of other sensory modalities.
This theory can also explain why it is impossible to distinguish the theoretical component of conscious experience from the visual component. Suppose "i" is a unit at the level of conscious input. Then "i" is receiving excitatory or inhibitory input from both visual cells, such as those in the retina, and from higher level cognitive cells in the form of back propagation. These input activations are summed, so that is the retinal cells are sending a signal of '4' and the cognitive cells are sending a signal of '5', the net input to "i" is '9'. This is what shows up in "i"; it increases activation by 9. Once, however, the input is summed, there is no means of determining which part of the sum is the retinal input and which part is the cognitive input. Given only '9', we can't tell whether it is '3'+'6', '5'+'4', or whatever.
C. A Theory of Perception
I have, in the course of responding to these various objections, sketched the outline of a theory of perception, which I would now like to make clear.
Our conscious experience just is the activation of a certain set of cells. These cells are the input cells to whatever network constitutes the conscious part of the brain, that is, the part of the activities of which we are aware. Input to these input cells comes from two sources: on the one hand, from input cells in the various of our sensory modalities, for example, retinal cells; and on the other hand, via back propagation from cells which are at higher levels, which I have called cognitive cells. These inputs are summed to produce activation, and that activation is what we call conscious experience.
Input activation produces what we call a representation. Above, I have explicitly endorsed what is called the "picture" theory of representation. The term "picture" is a bit misleading, since as input activation may be augmented and corrected from back propagation from previously formed associations and other sensory modalities, and there is no requirement that a given representation be isomorphic with or represent input activation. It is not necessary, and rather unlikely, that representations are composed of symbols and sentences. Human beings do not appear to be constrained by the sort of principles which constrain symbolic representations, rather, what appears to be the case is that representations are constructed and manipulated according to their relevant similarity with other representations.
I would like to briefly address the categorization of the different regions of human neural networks which I have identified above. I suggest that there are no fixed delineations regarding what counts as the conscious network or what counts as the input part of the conscious network. It seems reasonable to believe that the exact boundaries of these regions may vary from person to person, and indeed, may vary over time in a single person. Therefore, in a precise sense, there is no set of necessary and sufficient conditions which define what counts as an "experience" and a "representation". Let me describe a well-known phenomenon which substantiates this claim.
Professional athletes, for example, skiers, practice their sport by "imagining" or "visualizing" perfect performances. [49] It is worth noting, first, that the are able to do this, and second, that their performance improves as a result. For the most part, further, this is not an ability they have had from birth; they must be trained, and then practice, mental imagery. What I believe they are doing is learning to stimulate activation of conscious input cells according to previously learned association. In a sense, this requires extending the conscious region of the brain, for not everybody has the ability to vividly "picture" a sequence of events.
In my own case, I am quite convinced that this is a skill which can be learned, for over the course of the last two or three years I have practiced it myself. When I started, I had no such skills, yet now, I am able to evoke vivid mental images. I am able to control the subject matter of these images, but I prefer to allow them to occur at random. It really is an extraordinary experience and I recommend it for fun and relaxation. [50]
TNP Part VI Next Post
[40] Plato, Republic, p. 168.
[41] Rene Descartes, The Philosophical Works of Descartes (Haldane and Ross, eds.) Vol. 1, p. 145.
[42] A.J. Ayer, The Problem of Knowledge, p. 37.
[43] J.L. Austin, Sense and Sensibilia, ch. III.
[44] Here I am thinking of arguments such as Armstrong's SEEG argument. Pollock also argues this way. See John Pollock, Contemporary Theories of Knowledge, p.34.
[45] N.R. Hanson, Patterns of Discovery, p. 5. See also Thomas Kuhn, The Structure of Scientific Revolution, p. 150.
[46] Paul Churchland, Scientific Realism and the Plasticity of Mind, Ch. 1.
[47] See D. Marr and T. Poggio, "A Computational Theory of Human Stereo Vision", Science 194, pp.283-7, 1976. Also see P. Johnson-Laird, The Computer and the Mind, pp. 86-87.
[48] See Johnson-Laird, The Computer and the Mind, pp. 100-110. He describes this as
'top-down' processing. Typically, when people think of top-down processing, they think of pre-established or innate knowledge. But there is no reason to suppose this; it could just as easily and more consistently be the result of back propagation.
[49] Arnold Lazarus, In the Mind's Eye, p. 30.
[50] It is worth noting that when I started this, I tried to visualize a circle as though it were really in front of me. I had almost no success for about six months, Quite by accident, however, I once visualized a baseball, which of course appears as a circle. Since then, I have had no trouble producing circles. I start with a circle I have actually seen, then allow it to become an outline form.
