IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 20, NO. 5, SEPTEMBER/OCTOBER 1990 1229

A Model of Participatory Learning

RONALD R. YAGER

Abstract -A model of learning called participatory learning is intro- duced. This model allows the representation of learning in environments in that which is already believed to play a role in the process of learning turther information. A central part of this system is the compatibility between observations and belief. The idea of arousal that occurs when data is repeatedly seen that contradicts our beliefs is also discussed.

I. INTRODUCXION The current resurgence of interest in neural network research

has brought to the forefront the issue of adaption and learning. We suggest a model of learning that captures many of the salient features of human learning. The model suggested also has relevance to much of the AI work knowledge in acquisition via Kelly’s theory of personal constructs [l].

In many environments, learning is a bootstrap process in the sense that you learn and revise your beliefs in the framework of what you already know or believe. We shall call such an environ- ment a participatory learning environment. This name empha- sizes that our current knowledge of we are trying to learn participates in the process of learning about itself. A prototypi- cal example of this participatory environment is that of trying to convince a scientist to discard an old theory for a new one. In this situation, we must relate and explain this new theory in terms of the old theory, the faults of the old theory must lie within itself. Thus the old theory must participate in the learn- ing and believing in any new theory.

Fig. 1 highlights these ideas. What is important to note is that the current beliefs in addition to providing, via the ordinary (lower) feedback loop, a standard against which the observations a compared they very directly affect the process used for learn- ing via the upper feedback loop. This upper feedback loop is used to indicate that current beliefs and theories effect how we accept and process other information. This upper feedback loop corresponds to the participatory nature of the model.

Because of the structure of Fig. 1, what will be shown to be a central characteristic of this idea of participating learning is that an exogenous observation has the greatest impact in causing learning or belief revision when it is compatible with our current belief system. In particular, observations too distant (conflicting) with our current beliefs are discounted. We shall see that participatory learning is optimal in situations in which we are just trying to change a small part of our current belief system. The structure of a participatory learning system (PLS) is such that it is most receptive to learning when confronted with observations that essentially say “what you know is correct except for this little part.” On the other a PLS when confronted with observations that essentially say “you’re all wrong, this is the situation” responds by discounting what is being told to it. In its nature, it is a conservative learning system and hence very stable. This stability provided by the upper feedback link in Fig. 1.

Informally, we can say that a participatory learning environ- ment uses sympathetic observations to modify itself. Unsympa- thetic observations are discounted as being erroneous. Funda- mental to the PLS is the participation of our current beliefs. In particular, our current beliefs form our knowledge. Hence, if an observation occurs that is incompatible with our current belief

Manuscript received January 28, 1989; revised April 28, 1990. The author is with the Machine Intelligence Institute, Iona College, New

IEEE Log Number 9037414. Rochelle, NY 10801.

Le- Process ObsenraQns

Fig. 1

(could not be generated by it) our system says, since our beliefs are true and what we see conflicts with our beliefs something must be wrong with the observation. In a sense the PLS can be seen very much in the spirit of nonmonotonic reasoning [21, 131. In this correspondence the following is true.

Nonmonotonic Participatory Reasoning Learning

I. Certain Knowledge 11. Default Knowledge 11. Observations

In the previous table, correspondence priority is given in both systems to the I items at the expense of the I1 items.

We shall begin by investigating one formal model of this participatory learning idea.

I. Belief Structure

11. PARTICIPATORY LEARNING PROCEDURE In this section, we shall formally introduce a kind of participa-

tory learning algorithm. It is important to emphasize that this is not the only model of participatory learning nor is it meant to be exactly representative of the human learning process. The model presented captures the essential features of this important learning style.

Assume k = 1; . . ,q indicates a collection of nodes or vari- ables. Let V, be a valuation associated with each node. It is our objective to learn these valuations. We shall assume that V, E [0,1]. We shall also assume that our knowledge about these valuations comes in a sequence of observational vectors, D(1), D(2); . ., each of dimension q. Thus D,( j ) E [0,1] is the manifestation of the kth valuation in the j th observation. We are using the D vectors as means of learning about the valua- tions, the V’s. We shall say that our learning process is partici- patory if the usefulness of each of the observations, D(j), in contributing to the learning process depends upon its accep- tance by the current estimate of the V’s as being a valid observation. Implicit in this characterization is the idea that for an observation D(j) to be useful in learning about the V’s, it must in some sense agree with the current estimate of the V’s. We shall let V,(j) be our estimate for the node Vk after j observations and let V( j ) indicate the vector of these q values. Then our idea of participatory learning means that for D ( j ) to be useful (or used) in helping estimate the V’s, D( j ) should in some sense agree with V(j). What appears obvious about this type of learning is its very conservative nature. Participatory learning is saying that I am willing to learn from you if you are not too different from me.

A formal mechanism we shall use for updating our estimate (or belief) is a smoothing like algorithm [4]:

where V( j + l), V( j ) and D( j ) are q-vectors corresponding respectively to the new belief, the old belief and the current observation. Furthermore a E [O, 11 is the base or primary learn-

0018-9472/90/0900-1229$01.00 01990 IEEE

1230 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 20, NO. 5, SEPTEMBER/OCTOBER 1990

ing rate and p , which is the current compatibility level, is also required to sarisfy pI E [O, 11.

In participatory type learning system p, is dependent upon the compatibility between current belief and the current obser- vation, thus we require

P, = F ( D ( j ) , V ( j ) ) ,

where p, = 0 indicates complete incompatibility and p, = 1 com- plete compatibility.

In the above PLS if the current observation, D(j ) , is com- pletely contradictory with our current belief system V( j ) then

P J = o

and

V( j + 1) = V( j).

Thus in the case of complete contradiction the system discounts the current observation, i.e., it is not open to any learning from the current observation. Effectively, it is saying this is not a valid observation.

At the other extreme is the situation in which p, = 1. In this case the observation is in complete agreement with the current belief system. In this case with a = 1 we get

V( j + 1) = V( j) +1*(D( j ) - V( j ) )

V ( j + l ) = D ( j ) ,

and thus our system is the most open to the new information. Since p, is a measure of the agreement or compatibility of the

current observation D( j ) with the current belief V( j ) , a possible formulation for p, is

1 4 P,='-- C lDk(j)-Vk(.i)l .

k = l

Previously, p, is defined as is the complement of the average absolute difference between each observation and its corre- sponding current belief. (Alternatively, we could use tbe square of the differences.)

Using the above formulation for p, we get for each individual node, m = 1,. 1 a , q

Example: Assume tion for V be

q = 4 and a = 1. Let the current observa-

Hence

r -0.5 i

and 1 4

pi = 1 - -[OS +0.3+0.1+0.3] = 0.7.

Thus the new update belief V( j + 1) is

V( j + 1) = V ( j ) +0.7)D( j ) - V ( j ) I 0.7

0.6

It should be strongly noted that while in the case when p j = 1 we induce an environment where the system is most conducive to learning no learning occurs in this case. For we note that for pj to equal 1 it must be the case

1 4 - I D k ( j ) - V k ( j ) l ) = O . 9 k - 1

This requirement implies that for all k = 1; . ., 9

Dk( i) - Vk( i) = 0

and therefore

Dk( j ) = Vk(j) .

Thus

v( j + l) = Vk( i ) + al( D k ( j ) - Vk( j ) ) .

Vk( j + 1) = V k ( j ) .

As we shall subsequently see the most dramatic belief revision occurs when the vectors D( j ) and V( j ) are close but not exactly the same.

Considering the basic learning algorithm, for a any note k

Vk ( j + = Vk ( j ) + f j ( Dk ( j ) - vk (i)) the most dramatic learning for this node occurs when

= pjl(Dk(j) - V k ( j ) ) la

assumes its maximal value. Without loss of generality, we shall assume a = 1. Since

1 4 p j = 1 - - I D k ( j ) - V k ( j ) l

4 k = l

and denoting

I D k W - Vk(j) l=Ek(j)

we get

Rewriting this as

and denoting E k ( j ) as U and Crn+.kEm(j) as b we can express the aforementioned as

where U E [0, a , ] and b E [O, b,] where

a 1 =max[l- Vk(j),Vk(j)l

b,= C mm[(l-Vm(i) ,Vm(j)] .

Under the assumption that q > 2 it can be shown that the

m # k

IEEE TRANSACTIONS O N SYSTEMS, MAN, AND CYBERNETICS, VOL. 20, NO. 5 , SEPTEMBER/OCTOBER 1990 1231

maximum of Ak occurs at

C e m ( i ) = o m # k

and

€ k ( j ) = a ] = max [ l - V k ( j ) ? vk( j ) ]

at this point

A k = a l ( l - u l / q ) .

Thus the maximal learning for a particular node occurs when all the other nodes are in agreement and the node in question is observed to be furthest from its current value. The essential idea we see is that maximal learning occurs in a step-by-step fashion.

It is interesting to note that in environments in which q is large we are in a better position to learn in the face of some conflicts than one in which q is small. A possible interpretation of this fact is that in environments in which we have a strong highly interrelated theory we are more confident and thus more willing to accept disagreement.

In the above we see that the primary rate a is essentially “modulated” by the compatibility. In models such as the above where there are no participatory considerations, no p, term, a is usually made to be a small value, close to 0 (see [4] for a comprehensive discussion of the selection of a). The reason for the low selection of a is to avoid great swings due to spurious values of D which are far from V . This small value of a while protecting against the influence of bad readings have the down side effect of slowing down the learning process. The introduc- tion of the participatory term, p,, allows us to use higher values of a. The reason for this is that the learning rate in this model is dynamic, p, acts in a way to lower the learning rate when large deviations occur. On the other hand when the compatibility is large p, is such that it allows the full force of learning to occur. Thus we see that in this model that we can speed up the learning process.

It is possible to provide a more general formulation of the participatory learning algorithm capturing the essential features. In this more general formulation we let

V(J + 1 ) = V ( i ) + P, (D( i ) - V i ) ) where p, is some, perhaps nonlinear, function of p, such that

1) P, E tO71I

2 ) do, / d P , > 0.

The essential property is captured in condition 2 that stipulates that as pJ, the compatibility of our beliefs and observations, increases the learning rate p, increases.

111. FORMULATION OF THE COMPATIBILITY

In the previous section, we introduced a measure of compati- bility, p,,, between the current belief system V ( j ) and the current observation. In particular we suggested

1 4

p = I - - c I D k ( j ) - v k ( j ) l k = l

1 4 PJ = - ( ‘ - I D k ( i ) - V k < j > I ) -

k = l

In this section, we shall discuss more general alternative formulations of this measure. In order to simplify the notation, we shall drop the sequence number parameter, j . Thus Vk and Dk shall indicate the current belief and current observation of the variable k and V and D the associated vectors.

In a more general sense the term

1 - ID, - Vkl

can be seen to be a measure of similarity between Dk and Vk. In the following, we shall use the term s k to indicate the measure of similarity between vk and Dk, where

s k E[O,l].

Here again s k = 0 indicates complete dissimilarity and s k = 1 indicates the closest similarity. Formally we can introduce 3. function Gk, such that

s k = G k ( V k ? D k ) .

That is Gk maps pairs (vk,Dk) into a degree of similarity. We note the introduction of Gk allows us to free the V k ’ s and the D k ’ s from being in the unit interval. Secondly, it provides the facility for the two vk and Dk to have an sk = 1 even if they are not exactly equal. Thirdly it allows for different perceptions of similarity for different nodes (different k’s). We shall let S be the vector of similarities. Thus formally we can see that

p = F(S) = F( SI ,s,;. . , s,).

Essentially F is seen to be an aggregation function. A funda- mental property that must be associated with F is that it is monotonic, this requires that if

a J > b , V j = l ; * . , q

then

F ( a l , a z , . . ’ 9 a , ) > F ( bI7b2,. . . , b , ) .

A second property is that

F( 1,l; . . , 1 ) = 1.

This condition implies that if each element is similar, there is overail similarity.

Parenthetically, we note that our original measure p is simply

1 4 p = - s k .

I q k = l

More generally, we recall that in a participatory learning (or revision) algorithm the parameter p measures the degree to which our observed data D agrees with our current belief. In formulating such an aggregation function, we have some latitude in defining how restrictive we are in conceptualizing what we mean by the idea of agreement. That is, do we require that all the D k ’ s be in agreement with the V k ’ s , all the S k ’ s are 1, or are we willing to accept a situation in which most of the nodes are similar? To provide the most latitude in representing this con- cept, we introduce a class of aggregation operators called OWA (ordered weighted averaging) operators to formulate the F function. This operator was originally introduced by Yager 151 in the framework of multiple criteria decision function. We note that this operator was also used by Yager as an aggregation operator in neural networks [6].

Definition: A mapping F from

1 4 4 I I = [o, 13 is called an OWA operator of dimension q if associated with it is a weighting vector W.

1232

such that

2) c j w , = l

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 20, NO. 5, SEPTEMBER/OCTOBER 1990

and where

F ( S,,S,;. . ,S , ) = w , E , + w2B2 + . . . + wqE,

where E, is the jth largest element in the collection S , , SZ; . ., S,. It is important to emphasize that the weights in an OWA operator are associated with a particular ordered position rather than a particular s k .

Example: Assume F is a OWA operator of size 4 with

r0 .21

If S, = 0.6, S, = 1 , S, = 0.3 and S4 = 0.5 then the 6’s are B, = 1, B, = 0.6, E , = 0.5, and E, = 0.3 and hence

p = (0.2)(1) + (0.3)(0.6) +(0.1)(0.5) + (0.4)(0.3) = 0.55. It is important to note that our original formulation for p is a

The two extrema1 OWA operators are special OWA operator in which = l / q for all j .

Kl w*= 0

1 1 1

w*=l:J. The W , operator provides for the situation in which agreement between I/ and D requires that ‘‘all’’ the Vk’s and Dk’s individu- ally agree. It is a most restrictive aggregation. The W * operator provides overall agreement of at least one of the Dk’s agree with its V,. It is a most lax aggregation.

The elements of W can be associated with an indication of how open we are to learning. More formally, we can associate with a given OWA operator, W, a measure

PAR(W)=l-((;-I) 5 ((q-l)Wk) k = l

indicating the degree of participation required. It can be easily shown that

PAR E [O, 11.

We note PAR(W) = 1 indicates a requirement for complete agreement of all the individual nodes before learning while PAR ( W ) = 0 indicates a willingness to learn with minimal par- ticipation agreement.

As shown in [4], appropriate selection of the weights allow us to emulate environments in which we can specify the degrees of agreement required from the constituent nodes. Essentially the closer the W is to the W * the stronger the requirements for openness to learning, the more conservative the learning pro- cess.

IV. BACKGROUND AROUSAL PROCESS In the earlier section, we introduced a participatory learning

model

Around Mechanjsm

Arousal Rae Compatibilty Value

Observuabns ::I‘ - t

I . Fig. 2.

where p, = F(s1; . ., sk) measures the degree of compatibility of the current observation with the current belief system. As we have indicated with this participatory model, a high compatibil- ity between the current belief and the current observation enhances the environment for learning. This enhancement is manifested by p,. A number of questions can be raised about the previous structure. One concern is that no facility is provided for measuring the confidence we have in the current belief struc- ture, we are acting as if we have complete faith in our current theory, the values of the nodes, the Vk’s. A second concern is that such a system may prove to be to stable, that is the initial observations formulate a theory and the PLS has difficulty moving from this initial position. In this section, we shall intro- duce the facility for an opposing phenomenon, a background arousal mechanism. In particular, we feel that in this type of learning environment we are trying to emulate if over a long period we see a procession of low p,’s, incompatibility of belief and observations, the system should become more open to learning. That is, while in the immediate mode a low p, causes an aversion to learning a continued stream of low p,’s should make us more susceptible to learning. This background process we are trying to capture is essentially saying, that if we see a long string at low compatibilities between our belief and the data, we may come to believe that our belief structure is wrong, not the data. This is seen as a type of arousal.

In Fig. 2 we show an extended version of PLS. In this system the arousal mechanism monitors the performance of the lower system by observing the compatibility of the current model with the observations. This information is then feedback to the lower system in terms of an arousal rate that will subsequently be used to effect the learning process. As we shall subsequently see the higher the arousal rate the less confident we are with our current model and the more willing we are to let even conflict- ing observations change our beliefs.

We shall let a, E [0,1] be our arousal index. The higher a, the more aroused we are. The dynamics of the updation of the arousal index are specified in the following formula

a,+ 1 = a, + P ( < 1 - P,+ 1 ) - a , ) . (1)

It is the function of the block labeled arousal mechanism to process this arousal updation. From the above formula we see that if p,+ I = 1, high compatibility, then

a, , 1 = a, - Pa,

a,, I = a, + P ( 1 - a , )

the arousal index decreases. On the other hand, p, + , = 0 implies

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 20, NO. 5, SEPTEMBER/OCTOBER 1990

r------ Obsemtiolu - +-

musal Index

Cnm

(Arousal Mecherusm) . , ,'

Behf Enur Arousal Index

\'

Behf S p o m

Learmng Process

' Fig. 3.

and thus causes an increase in the arousal index. The constant p E [O, 11 modulates the rate of change of arousal, the closer P is to one the quicker the system is to sense the background process. P can be seen as some kind of sensitivity characteristic the larger its value the more sensitive the system is to respond- ing to deviations from the predicted. An alternative characteri- zation of P can be seen as the complement of the degree of conservatism of the system. P may be a function of the amount of information seen.

The arousal index can be seen as the complement of the confidence in the belief structure currently held.

If we denote a,,, =a, + Aa

where

A a = p ( l - p ) - pa

we see that Aa>O i f p < l - a Aa<O i f p > l - a .

The arousal index can be seen to be inversely related to p. As implied in Fig. 2 the effect of the arousal index is to

intercede in the learning process. We shall now show how the arousal index can be used to implement the phenomenon whereby if a long procession of low p,'s occurs we say that maybe it is not the observations that are wrong but that our belief system, the V's, need revision. That is a high arousal index will cause us to lose confidence in the current theory and make it more susceptible to modification.

A version of the participatory learning system that enables us to appropriately include the background arousal index is

V ( j + l ) = V ( j ) + a ( p , ) ' - ~ J ( D ( j ) - V ( j ) ) . (2)

In this PLS we see that if a, = 0 then we have our original learning model. On the other hand we see that as our arousal index increases the term (p,)'-'~ increases. In particular we note that when a, = 1, complete arousal, then

1-a ,=O

and

and therefore

V( j + 1) = V( j ) + a( D( j ) - V ( j ) ) .

In this case we have completely suppressed the effect of pi.

1233

More generally, we can indicate the learning model as

where in the aforementioned

Wa,, P,) = a ( p , ) I - ' I .

An alternate form for G is

G ( a , , p , ) = A(p ,a ) ' - ' ' .

In this formulation, h a is our learning rate in a completely compatible environment, p, = 1, a, = 0. While A rate is our learning when completely aroused. An alternative formulation for G is

G(@,,P,) = A(a, + P,a - a,p, a)

G ( a , , p , ) = A

G ( a , , p , ) = h a

Here we note that when arousal is complete a, = 1,

when arousal is lowest a, = 0 and p, = 1

We can see that a complete version of a PLS system with arousal index can be implemented with iterative use of (1) and (2):

a, + 1 = a, + P (( 1 - P, + 1 ) - a, ).

~ ( i + 1) = + a ( p , ) l - - O ~ (W) - W ) ) In using the model of (1) and (2) we would initialize V(0) with a value indicating our best guess for V and initialize a, with value indicating the compliment of our confidence in this initial guess for V. In particular a, should be close to one.

The following Fig. 3 provides a more general view of the overall system. The association of the arousal mechanism with the concept of a critic was clarified by a recent presentation by Werbos [7].

In Fig. 3 we clearly see the relationship between the learning process and the belief system as one in which the learning process is imbedded in the current belief system, this accounts for the participatory nature of the learning process. While not explicitly noted in the figure the learning process acts on the belief system by updating the belief systems parameters. The

~

1234 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 20, NO. 5, SEPTEMBER/OCTOBER 1990

second salient feature of the model is the role of the critic. It is first noted that the critic is independent of the belief system. The critic effects the learning process, and through it the belief system, by providing an objective valuation of the performance of the belief system throughout the arousal index, which as we noted is related inversely to the confidence we have in the current belief system. The independence of the critic from the belief system is crucial to receiving of an unbiased view of the performance of the system. The only memory retained by the critic is the current arousal index. It should also be noted that while the belief system is always active, it’s our world uiew, the critic is generally dormant. The critic only becomes active when an observation about the environment becomes manifest. The stimulation of the critic can be seen as instigating the next learning phase.

At this point we would like to speculate on an additional role for the critic. In this role the critic can be used to build up a data base of exceptional situations. In particular when presented with an observation that greatly deviates (or deviates in a peculiar way) from the anticipated value the critic may store this pair in a special memory assigned to the model. A natural model for storing this information can be an associative memory [8]. This memory of exceptions can perhaps be used to mediate an additional learning mechanism. Since the basic learning mecha- nism we have previously discussed this secondary learning mech- anism can play a fundamental role in situations in which large jumps in belief systems occur, so called world view changes. One such giant change is the introduction of new nodes or variables an idea suggested by Werbos [7].

V. CONCLUSION

current model.

In this work we have introduced a new concept of participa- tory learning and described a particular manifestation of it. The key idea in this type of learning is that the currently held belief system plays a central role in the learning process. We feel that the participatory learning paradigm can play a central role in adaptive neural networks. It should be carefully noted that this type of learning is not restricted to the type of quantitative system we have used to exemplify it in this paper but is also relevant to symbolic learning environments of the kind used in artificial intelligence [9].

A second feature of the system presented here is the crucial role of the independent critic in evaluating the performance of the current belief system and in turn mediating the learning Process by providing an unbiased estimate for the confidence in

REFERENCES G. A. Kelly, The Psychology of Personal Constructs. New York: Norton, 1955. P. Besnard, An lntroduction to Default Logic. Heidelberg: Springer- Verlag, 1989. R. R. Yager, “A generalized view of non-monotonic knowledge: A set theoretic perspective,” lnt. J . Gen. Syst., vol. 14, pp. 251-265, 1988. R. G. Brown, Smoothing, Forecasting and Prediction of Discrete Time Series. R. R. Yager, “On ordered weighted averaging aggregation operators in multicriteria decision making,” IEEE Trans. Syst. Man Cybern., vol. 18, pp. 183-190, 1988. -, “On the aggregation of processing units in neural networks,” Proc. lEEE First lnt..Conf. Neural Networks, San Diego, vol. 11, pp. 327-333, 1987. P. Werbos, “Neural networks for control and the link to fuzzy logic,” NASA Workshop on Neural Networks and Fuzzy Logic, Johnson Space Center, Houston, TX, 1990. T. Kohonen, Self Organization and Associatiiv Memory. Heidelberg: Springer-Verlag. 1984. T. M. Mitchell, J. G. Carbonell, and R. S. Michalski, Machine Learning: A Guide to Current Research. Boston: Kluwer Academic Publishers, 1986.

Englewood Cliffs, NJ: Prentice-Hall, 1963.

Generalized Entropy and Minimum System Complexity

IOANNIS PANDELIDIS

Abstract -The problem of characterizing information processing sys- tems in terms of their entropic complexity is examined. Automatic control systems and the design/manufacturing process are seen as such systems, so that the results of such research are intended primarily for this area. The concept of generalized entropy is introduced in order to unify the treatment of continuous and discrete signals and the corre- sponding information processing systems. Given a level of entropic complexity of the reference and disturbance inputs to a system, we derive minimum entropic complexity requirements for that system. It is assumed that the entropic system complexity that is defined in this paper is monotonically related to cost. Based on this assumption and a previously proposed design/manufacturing axiom, the system having minimum entropic complexity is derived.

I. I N T R O D U C r l O N

The concept of entropy has been shown to be a fundamental concept that seems to defy disciplinary boundaries in its applica- tion. Entropy has played a fundamental role in thermodynamics, information theory, statistical mechanics and has had an impact on a host of other disciplines such as biology and computer science. In the field of design and manufacturing, some re- searchers have proposed the concept of entropy as a measure of complexity [l]. In the axiomatic theory of design [2] the mini- mization of entropy is one of the two proposed axioms. One of the difficulties with the concept of entropy however is some incongruence in the treatment of discrete type random variables (DTRV) and the treatment of continuous type random variable (CTRV) that has not allowed for a unified treatment of the continuous (analog) and discrete signals that are both present in a typical design/manufacturing environment.

The entropy is a measure of the uncertainty associated with a particular experiment involving the possible outcomes A, of the experiment. After the experiment is performed the uncertainty is removed and thus the uncertainty is equal to the information we have gained from that experiment. The entropy of a partition is thus seen to be a measure of information.

A new concept, that of the generalized information (entropy), will be introduced. This concept includes the original concept of entropy as a special case but is capable of relating the continu- ous time signals with the discrete time signals in a uniform way. Some properties of the generalized entropy will be presented in Section 111. In Section IV we shall deal with the problem of deriving minimum system complexity requirements for systems that have specified information rate limits for the target set, the disturbances, and the acceptable error.

In another article [3], an operational theory of design/manu- facturing (D/M) was presented. The design/manufacturing process was described in terms of input/output mappings and a D/M axiom was proposed that asserted that the optimal D/M process minimizes the total expected cost at each iteration. The corrolaries of the D/M axiom coupled with the assumption that the entropic complexity of a system is monotonically related to the cost will be presented in Section V.

Manuscript received August 29, 1988; revised February 15, 1990. The author was with the Department of Mechanical Engineering, Univer-

sity of Maryland, College Park, MD 20742. He is now with the Engineering Center of the University of Maryland.

IEEE Log Number 9035402.

0018-9472/90/0900-1234$01.00 01990 IEEE