mculloch-pitts_A Logical Calculus of Ideas Immanent in Nervous Activity

BULLETIN OF MATHEMATICAL BIOPHYSICS VOLUME 5, 1943 A LOGICAL CALCULUS OF THE IDEAS IMMANENT IN NERVOUS ACT IV ITY WARREN S. MCCULLOCH AND WALTER PITTS FROM THE UNIVERSITY OF ILLINOIS, COLLEGE OF MEDICINI~, DEPARTMENT OF PSYCHIATRY AT THE ILLINOIS NEUROPSYCHIATRIC INSTITUTE, AND THE UNIVERSITY OF CHICAGO Because of the "all-or-none" character of nervous activity, neural events and the relations among them can be treated by means of propo- sitional logic. I t is found that the behavior of every net can be described in these terms, with the addition of more complicated logical means for nets containing circles; and that for any logical expression satisfying certain conditions, one can find a net behaving in the fashion it describes. I t is shown that many particular choices among possible neurophysiologi- cal assumptions are equivalent, in the sense that for every net behav- ing under one assumption, there exists another net which behaves un- der the other and gives the same results, although perhaps not in the same time. Various applications of the calculus are discussed. I. Introduction Theoret ica l neurophys io logy rests on cer ta in card ina l assump- t ions. The nervous sys tem is a net of neurons, each hav ing a soma and an axon. The i r ad junct ions, or synapses, are a lways between the axon of one neuron and the soma of another . A t any ins tant a neuron has some threshold, which exci tat ion must exceed to in i t iate an im- pulse. This, except fo r the fac t and the t ime of its occurrence, is de- te rmined by the neuron, not by the excitat ion. F rom the point of ex- c i tat ion the impulse is p ropagated to all par ts of the neuron. The velocity a long the axon var ies direct ly w i th its d iameter , f rom less than one meter per second in th in axons, which are usual ly short , to more than 150 meters per second in th ick axons, which are usual ly long. The t ime for axonal conduct ion is consequent ly of l i tt le impor - tance in determin ing the t ime of a r r iva l of impulses at points un- equal ly remote f rom the same source. Exc i ta t ion across synapses oc- curs predominant ly f rom axonal te rminat ions to somata. I t is stil l a moot point whether this depends upon i r rec iproc i ty of indiv idual syn- apses or mere ly upon preva lent anatomica l conf igurat ions. To sup- pose the la t ter requires no hypothes is ad hoc and expla ins known ex- ceptions, but any assumpt ion as to cause is compat ib le wi th the cal- culus to come. No case is known in which exci tat ion through a s ingle synapse has elicited a nervous impulse in any neuron, whereas any 115 116 LOGICAL CALCULUS FOR NERVOUS ACTIVITY neuron may be excited by impulses arr iving at a sufficient number of neighboring synapses within the period of latent addition, which lasts less than one quarter of a millisecond. Observed temporal summation of impulses at greater intervals is impossible for single neurons and empirically depends upon structural properties of the net. Between the arrival of impulses upon a neuron and its own propagated im- pulse there is a synaptic delay of more than half a millisecond. Dur- ing the first part of the nervous impulse the neuron is absolutely re- fractory to any stimulation. Thereafter its excitability returns rap- idly, in some cases reaching a value above normal from which it sinks again to a subnormal value, whence it returns slowly to normal. Fre- quent activity augments this subnormality. Such specificity as is possessed by nervous impulses depends solely upon their time and place and not on any other specificity of nervous energies. Of late only inhibition has been seriously adduced to contravene this thesis. Inhibition is the termination or prevention of the activity of one group of neurons by concurrent or antecedent activity of a second group. Until recently this could be explained on the supposition that previous activity of neurons of the second group might so raise the thresholds of internuncial neurons that they could no longer be ex- cited by neurons of the first group, whereas the impulses of the first group must sum with the impulses of these internuncials to excite the now inhibited neurons. Today, some inhibitions have been shown to consume less than one millisecond. This excludes internuncials and requires synapses through which impulses inhibit that neuron which is being stimulated by impulses through other synapses. As yet ex- periment has not shown whether the refractoriness is relative or ab- solute. We will assume the latter and demonstrate that the difference is immaterial to our argument. Either variety of refractoriness can be accounted for in either of two ways. The "inhibitory synapse" may be of such a kind as to produce a substance which raSses the threshold of the neuron, or it may be so placed that the local disturb- ance produced by its excitation opposes the alteration induced by the otherwise excitatory synapses. Inasmuch as position is already known to have such effects in the case of electrical stimulation, the first hy- pothesis ~s to be excluded unless and until it be substantiated, for the second involves no new hypothesis. We have, then, two explanations of inhibition based on the same general premises, differing only in the assumed nervous nets and, consequently, in the time required for inhibition. Hereafter we shall refer to such nervous nets as equiva- lent in the extended sense. Since we are concerned with properties of nets which are invariant under equivalence, we may make the physical assumptions which are most convenient for the calculus. WARREN S. MCCULLOCH AND WALTER PITTS 117 Many years ago one of us, by considerations impertinent to this argument, was led to conceive of the response of any neuron as fac- tually equivalent to a proposition which proposed its adequate stimu- lus. He therefore attempted to record the behavior of complicated nets in the notation of the symbolic logic of propositions. The "all- or-none" law of nervous activity is sufficient to insure that the activ- ity of any neuron may be represented as a proposition. Physiological relations existing among nervous activities correspond, of course, to relations among the propositions; and the utility of the representa- tion depends upon the identity of these relations with those of the logic of propositions. To each reaction of any neuron there is a corre- sponding assertion of a simple proposition. This, in turn, implies either some other simple proposition or the disjunction or the con- junction, with or without negation, of similar propositions, according to the configuration of the synapses upon and the threshold of the neuron in question. Two difficulties appeared. The first concerns facilitation and extinction, in which antecedent activity temporarily alters responsiveness to subsequent stimulation of one and the same part of the net. The second concerns learning, in which activities concurrent at some previous time have altered the net permanently, so that a stimulus which would previously have been inadequate is now adequate. But for nets undergoing both alterations, we can sub- stitute equivalent fictitious nets composed of neurons whose connec- tions and thresholds are unaltered. But one point must be made clear: neither of us conceives the formal equivalence to be a factual expla- nation. Per contra!--we regard facilitation and extinction as depen- dent upon continuous changes in threshold related to electrical and chemical variables, such as after-potentials and ionic concentrations; and learning as an enduring change which can survive sleep, anaes- thesia, convlusions and coma. The importance of the formal equiva- lence lies in this: that the alterations actually underlying facilitation, extinction and learning ~in no way affect the conclusions which follow from the formal treatment of the activity of nervous nets, and the relations of the corresponding propositions remain those of the logic of propositions. The nervous system contains many circular paths, whose activity so regenerates the excitation of any participant neuron that reference to time past becomes indefinite, although it still implies that afferent activity has realized one of a certain class of configurations over time. Precise specification of these implications by means of recursive func- tions, and determination of those that can be embodied in the activity of nervous nets, completes the theory. 118 LOGICAL CALCULUS FOR NERVOUS ACTIVITY H. The Theory: Nets Without Circles We shall make the following physical assumptions for our cal- culus. 1. The activity of the neuron is an "all-or-none" process. 2. A certain fixed number of synapses must be excited within the period of latent addition in order to excite a neuron at any time, and this number is independent of previous activity and position on the neuron. 3. The only significant delay within the nervous system is syn- aptic delay. 4. The activity of any inhibitory synapse absolutely prevents excitation of the neuron at that time. 5. The structure of the net does not change with time. To present the theory, the most appropriate symbolism is that of Language II of R. Carnap (1938), augmented with various notations drawn from B. Russell and A. N. Whitehead (1927), including the Principia conventions for dots. Typographical necessity, however, will compel us to use the upright 'E' for the existential operator in- stead of the inverted, and an arrow ('-~') for implication instead of the horseshoe. We shall also use the Carnap syntactical notations, but print them in boldface rather than German type; and we shall intro- duce a functor S, whose value for a property P is the property which holds of a number when P holds of its predecessor; it is defined by ' S (P ) (t) .=-. P (Kx) . t ~ x') ' ; the brackets around its argument will often be omitted, in which case this is understood to be the nearest predicate-expression [Pr] on the right. Moreover, we shall write S~Pr for S (S (Pr ) ) , etc. The neurons of a given net ~ may be assigned designations 'c1', 'c~', . . . , 'c~'. This done, we shall denote the property of a number, that a neuron c~ fires at a time which is that number of synaptic de- lays from the origin of time, by 'N' with the numeral i as subscript, so that N~ (t) asserts that c~ fires at the time t. N~ is called the action of e~. We shall sometimes regard the subscripted numeral of 'N' as if it belonged to the object-language, and were in a place for a func- toral argument, so that it might be replaced by a number-variable [z] and quantified; this enables us to abbreviate long but finite dis- junctions and conjunctions by the use of an operator. We shall era- ploy this locution quite generally for sequences of Pr; it may be se- cured formally by an obvious disjunctive definition. The predicates 'NI', 'N.~', . . . , comprise the syntactical class 'N'. WARREN S. MCCULLOCH AND WALTER PITTS 119 Let us define the peripheral afyerents of ~ as the neurons of ~( with no axons synapsing upon them. Let N1, - - . , Np denote the ac- tions of such neurons and N~+I, N~+~, . . . , N~ those of the rest. Then a solution of $~ will be a class of sentences of the form S~: Np+l (zl) .=. Pr~ (N1, N~, . . . , Np, z~), where Pr~ contains no free variable save zl and no descriptive symbols save the N in the argument [Arg], and possibly some constant sentences [sa] ; and such that each S~ is true of ~ . Conversely, given a Pr l ( lp~, lp12, . . . , ~p~p, zl , s), containing no free variable save those in its Arg, we shall say that it is realizable in the narrow sense if there exists a net $~ and a series of N~ in it such that Nl(z~) .----. Pr I (N~, N~, . . . , z l , sa~) is true of it, where sa~ has the form N(0) . We shall call it realizable in the extended sense, or simply realizable, i f for some n S~(Pr l) (p l , . " , pp, z~, s) is realizable in the above sense, c~ is here the realizing neuron. We shall say of two laws of nervous excitation which are such that every S which is realizable in either sense upon one supposition is also re- alizable, perhaps by a different net, upon the other, that they are equivalent assumptions, in that sense. The following theorems about realizability all refer to the ex- tended sense. In some cases, sharper theorems about narrow re~liz- ability can be obtained; but in addition to greater complication in statement this were of little practical value, since our present neuroo physiological knowledge determines the law of excitation only to ex- tended equivalence, and the more precise theorems differ according to which possible assumption we make. Our less precise theorems, however, are invariant under equivalence, and are still sufficient for all purposes in which the exact time for impulses to pass through the whole net is not crucial. Our central problems may now be stated exactly: first, to find an effective method of obtaining a set of computable S constituting a solution of any given net; and second, to characterize the class of realizable S in an effective fashion. Materially stated, the problems are to calculate the behavior of any net, and to find a net which will behave in a specified way, when such a net exists. A net will be called cyclic if it contains a circle: i.e., if there ex- ists a chain c~, c~§ ... of neurons on it, each member of the chain synapsing upon the next, with the same beginning and end. I f a set of its neurons a~, c~ , . . . , cp is such that its removal from ~ leaves it without circles, and no smaller class of neurons has this property, the set is called a cyclic set, and its cardinality is the order of ~. In an important sense, as we shall see, the order of a net is an index of the complexity of its behavior. In particular, nets of zero order have especially simple properties; we shall discuss them first. 120 LOGICAL CALCULUS FOR NERVOUS ACTIVITY Let us define a temporal propositional expression (a TPE), des- ignating a temporal propositional function (TPF), by the following recursion: 1. A ~p~ [z~] is a TPE, where p~ is a predicate-variable. 2. I f S~ and S~ are TPE containing the same free individual variable, so are SSI, S~vS~, SI.S~ and S~. ~ S~. 3. Nothing else is a TPE. THEOREM I. Every net of order 0 c~n be solved in terms of temporal proposi- tional expressions. Let c~ be any neuron of g~ with a threshold t~ > 0, and let c~1, c~, ... , c~, have respectively n~, n~:, ... , n~p excitatory synapses upon it. Let cj~, ci:, --. , c~q have inhibitory synapses upon it. Let ~ be the set of the subclasses of (n~l, n~:, .~., n~,) such that the sum of their members exceeds 6~. We shall then be able to write, in accord- ance with the assumptions mentioned above, N~(z~) . - - .S t I~,~ ~ Nj~(z~). ~,~ ~H N~ (z~)} (1) where the 'F~' and 'II' are syntactical symbols for disjunctions and conjunctions which are finite in each case. Since an expression of this form can be written for each c~ which is not a peripheral affer- ent, we can, by substituting the corresponding expression in (1) for each Ns,, or N~, whose neuron is not a peripheral afferent, and re- peating the process on the result, ultimately come to an expression for N~ in terms solely of peripherally afferent N , since ~ is without circles. Moreover, this expression will be a TPE, since obviously (1) is; and Jt follows immediately from the definition that the result of substituting a TPE for a constituent p (z) in a TPE is also one. THEOREM II. Every TPE is realizable by a net of order zero. The functor S obviously commutes with disjunction, conjunction, and negation. I t is obvious that the result of substituting any S~, realizable in the narrow sense (i.n.s.), for the p(z) in a real~izable ex- pression $1 is itself realizable i.n.s. ; one constructs the realizing net by replacing the peripheral afferents in the net for $1 by the realizing neu- rons in the nets for the S~. The one neuron net realizes pl(zl) i.n.s., WARREN S. MCCULLOCH AND WALTER PITTS 121 and Figure 1-a shows a net that realizes Spl (zl) and hence SS2, i.n.s., if S: can be realized i.n.s. Now if S~ and $3 are realizable then SmS~ and S~S~ are realizable i.n.s., for suitable m and n . Hence so are S~S~ and S~"Sa. Now the nets of Figures lb, c and d respectively realize S (p~ ( zl ) v p~ ( z~ ) ), S (p~(zD. p~ ( z~ ) ), and S (p~ ( z~ ) . oo p~ ( z~ ) ) i.n.s. Hence S ~+'+~ (S, v S~), S ~§ (81 . $2), and S ~§ (S1. o~ S2) are realizable i.n.s. Therefore S, v $2 $1 . S~ S~ . oo $2 are realizable if $1 and S:~ are. By complete induction, all TPE are realizable. In this way all nets may be regarded as built out of the fundamental elements of Figures la, b, c, d, precisely as the temporal propositional expres- sions are generated out of the operations of precession, disjunction, conjunction, and conjained negation. In particular, corresponding to any description of state, or distribution of the values true and false for the actions of all the neurons of a net save that which makes them all false, a single neuron is constructible whose firing is a neces- sary and sufficient condition for the validity of that description. More- over, there is always an indefinite number of topologically different nets realizing any TPE. THEOREM I I I . Let there be given a complex sentence S~ built up in any manner out of elementary sentences of the form p (z~ - zz) where zz is any nu- meral, by any of the propositional connections: negation, dis]unction, con]unction, implication, and equivalence. Then $1 is a TPE wad only if it is false when its constituent p (zl - zz) are all assumed false--i.e., replaced by false sentences - - or that the last line in its truth-table contains an 'F',--or there is no term in its Hilbert dis- ]unctive normal form composed exclnsively of negated terms. These latter three conditions are of course equivalent (Hilbert and Ackermann, 1938). We see by induction that the first of them is necessary, since p(z l - zz) becomes false when it is replaced by a false sentence, and $1 v S~, S~ . S~ and $1 . ~ $2 are all false if both their constituents are. We see that the last condition is sufficient by remarking that a disjunction is a TPE when its constituents are, and that any term 81.8 .~ . . . . S~. ~ S,,~+~. ~176 . . . . ~ 8 . can be written as (8~.8.~ . . . . Sin) .~ (S~,vS~§ . . . . vS , ) , which is clearly a TPE. The method of the last theorems does in fact provide a very con- venient and workable procedure for constructing nervous nets to or- 122 LOGICAL CALCULUS FOR NERVOUS ACTIVITY der, for those cases where there is no reference to events indefinitely fa r in the past in the specification of the conditions. By way of ex- ample, we may consider the case of heat produced by a transient cooling. I f a cold object is held to the skin for a moment and removed, a sensation of heat will be felt; if it is applied for a longer time, the sensation will be only of" cold, with no preliminary warmth, however transient. It is known that one cutaneous receptor is affected by heat, and another by cold. I f we let N1 and N~ be the actions of the respec- tive receptors and N~ and N4 of neurons whose activity implies a sen- sation of heat and cold, our requirements may be written as N~(t ) : ~- : N l ( t -1 ) .v .N~(t -3 ) . ~ N~(t -2 ) N4( t ) . =--. N~(t -2 ) . N~(t -1 ) where we suppose for simplicity that the required persistence in the sensation of cold is say