an analysis of the entropy of mixing for granular materials
TRANSCRIPT
Powder Technology 266 (2014) 90–95
Contents lists available at ScienceDirect
Powder Technology
j ourna l homepage: www.e lsev ie r .com/ locate /powtec
Short communication
An analysis of the entropy of mixing for granular materials
Zongyu Gu ⁎, J.J.J. ChenDepartment of Chemical and Materials Engineering, The University of Auckland, Private Bag 92019, Auckland 1142, New Zealand
⁎ Corresponding author. Tel.: +64 226357788, +64 99E-mail addresses: [email protected] (Z. Gu),
(J.J.J. Chen).
http://dx.doi.org/10.1016/j.powtec.2014.06.0290032-5910/© 2014 Published by Elsevier B.V.
a b s t r a c t
a r t i c l e i n f oArticle history:Received 13 March 2014Received in revised form 15 May 2014Accepted 16 June 2014Available online 23 June 2014
Keywords:Granular materialsParticle processingMixingEntropy of mixingShannon entropyPhase space
Many existing definitions of the entropy of mixing for granular materials involve a “total entropy” that iscalculated from the “local entropies” of all cells in the domain of the mixture. The total entropy has been usedas ameasure of mixedness bymany authors, but they have virtually never presented the values of the individuallocal entropies. For comparison purposes, we introduced a parallel definition of the entropy of mixing byconsidering an alternative “total entropy” that is based on the “per-species entropies” of all types of particlesin the mixture. Using a simple mixing model in continuous variables, we showed that the contributions of theindividual “local entropies” and the “per-species entropies” to their respective totals showdifferent trends duringa specific, idealisedmixing process: while the total entropy constantly increases following either definition, onlythe local entropies show non-monotonic changes; on the other hand, only the per-species entropies reach themaximum possible value of the Shannon entropy at the steady state of mixing.We rationalised these differencesby considering the changes in the probabilities associated with each individual entropic term, in the contextof the properties of the Shannon entropy, and confirmed these ideas by studying the two-dimensionalphase-space trajectories of the individual entropic terms for the mixing process considered.
© 2014 Published by Elsevier B.V.
1. Introduction: definition and some properties of the Shannonentropy
Since the concept of information entropy was put forward byShannon in 1948 [1], many authors [2–10] have defined measuresof mixedness based on the Shannon entropy for use in their study ofgranular mixing. We will first state the mathematical definition of theShannon entropy. Consider a discrete probability distribution withn outcomes, with respective probabilities of occurrence given byp1, …, pn; knowing exactly one of the n events is bound to occur,we define the Shannon entropy [1] of the distribution as
H ¼ −Xn
k¼1
pk lnpk ð1Þ
Some important properties of the Shannon entropy [1] follow fromthis definition. Firstly, any change towards equalisation of p1, p2, …, pnincreases H, that is, if p1 b p2 and we increase p1 and decrease p2 by anequal amount so that p1 and p2 are more nearly equal, then the entropyincreases. Secondly, when all the probabilities aremade exactly equal,Hreaches itsmaximumpossible value of ln n. Thus,Hwill keep increasingand ultimately reach its global maximum if and only if its associatedprobabilities constantly tend towards equalisation.
2. The “local entropy” concept
Consider a mixture of ns different types of equal size particles,or “species”. The entire domain that the particles occupy is dividedinto nc fixed regions called “cells”. Then most entropy-based mixingindices in the literature [2–10] are defined in a fashion similar to whatfollows: for a certain state of the mixture, one first calculates the “localentropy” for every individual cell using
Si ¼ −Xns
j¼1
pj=i lnpj=i ð2Þ
where i is the counter for cells, j the counter for species, and pj/i the con-ditional probability that a random particle picked from themixture is ofspecies j, given that the particle is in cell i; in orderwords, pj/i is the frac-tion of particles in cell i that are of species j. Next, onefinds the “total en-tropy” of the entire mixture, given by
Stot ¼Xnci¼1
piSi ð3Þ
where pi is the probability that a randomparticle picked from the entiremixture is from cell i, that is, the size of cell i divided by the size of thedomain. We can think of Stot as a weighted arithmetic mean of all Si,where the contribution of each Si is proportional to the size of cell i.
Although authors who proposed definitions similar to the abovehave closely examined Stot during mixing, very little about the
Table 1Initial conditions of the mixing process considered in terms of pj/i.
pj/i Cells
i = 1 (cell 1) i = 2 (cell 2) i = 3 (cell 3)
Species j = 1 (blue particles) 1/18 0 11/18j = 2 (red particles) 0 1/18 5/18j = 3 (green particles) 17/18 17/18 1/9
91Z. Gu, J.J.J. Chen / Powder Technology 266 (2014) 90–95
individual Si terms has been reported. We will now consider a specif-ic example to examine the individual contributions of Si to Stot.Consider a mixture of different types of particles that are only
0.0
0.2
0.4
0.6
0.8
1.0
0 1 2 3 4 5
pj/
i
i=1, j=1 i=1, j=2 i=1, j=3
0.0
0.2
0.4
0.6
0.8
1.0
0 1 2 3 4 5
pj/
i
i=3, j=1 i=3, j=2 i=3, j=3
0.0
0.2
0.4
0.6
0.8
1.0
0 1 2 3 4 5
pj/
i
i=1, j=2 i=2, j=2 i=3, j=2
a
c
e
Fig. 1. (a). Solutions for the simulatedmixing process in terms of pj/i; cell 1, i=1. (b). Solutions fmixing process in terms of pj/i; cell 3, i= 3. (d). Solutions for the simulated mixing processterms of pj/i; red particles, j = 2. (f). Solutions for the simulated mixing process in terms
distinguishable by their colour: blue, red, or green, for which j isequal to 1, 2, or 3 respectively. Let pj be the probability that arandom particle picked from the entire mixture is of species j, andarbitrarily set pj = 1 = 2/9, pj = 2 = 1/9, and pj = 3 = 2/3 for thepurpose of illustrating the calculations. We divide the mixture intothree equal size cells so that pi = 1/3 for i = 1, 2, 3. Now considera mixing process where the initial conditions are given by Table 1in terms of pj/i. Note it is possible to set pj, pi, and the initial valuesof pj/i to any other self-consistent values.
We now require a simple method to cause mixing of the particles.There are many methods of simulating granular mixing; some arevery sophisticated, such as the Discrete Element Method approach
0.0
0.2
0.4
0.6
0.8
1.0
0 1 2 3 4 5
pj/
i
i=2, j=1 i=2, j=2 i=2, j=3
0.0
0.2
0.4
0.6
0.8
1.0
0 1 2 3 4 5
pj/
i
i=1, j=3 i=2, j=3 i=3, j=3
b
d
f
0.0
0.2
0.4
0.6
0.8
1.0
0 1 2 3 4 5
pj/
i
i=1, j=1 i=2, j=1 i=3, j=1
or the simulatedmixing process in terms of pj/i; cell 2, i=2. (c). Solutions for the simulatedin terms of pj/i; blue particles, j=1. (e). Solutions for the simulated mixing process inof pj/i; green particles, j = 3.
0
0.2
0.4
0.6
0.8
1
1.2
0 1 2 3 4 5
En
tro
py
, S
/nat
i=1 i=2 i=3 tot
Fig. 2. Local and total entropies (Si and Stot) during the simulated mixing process.
92 Z. Gu, J.J.J. Chen / Powder Technology 266 (2014) 90–95
(see [5,9]). However, in this exercise, a much simpler mixingmodelwillsuffice for illustration purposes—we will assume here that the totalnumber of particles is sufficiently large that all probabilities can bemodelled as continuous variables. The mixture will thus homogeniseif the pj/i terms obey the following system of Ordinary DifferentialEquations (ODEs)
dp! j=i
dt¼
−k k 0k −2k k0 k −k
24
35 p! j=i for j ¼ 1;2;3 ð4Þ
where t is the time since mixing begins and k is a rate constant that hasthe dimension of the reciprocal of time; the vector p! j=i in Eq. (4) denotes
p! j=i ¼pj= i¼1ð Þpj= i¼2ð Þpj= i¼3ð Þ
24
35 for j ¼ 1;2;3 ð5Þ
It can be shown that the nine ODEs described in Eqs. (4) and (5)automatically conserve the number of particles of each species, as wellas the number of particles in each cell. When subject to the initialconditions in Table 1, the solutions to the ODEs are as seen in Fig. 1,where we have introduced τ = kt as a dimensionless quantity thatrepresents the progress of mixing.
With such data now available, we can readily calculate three Si andone Stot following Eqs. (2) and (3); plotting all these entropic termstogether gives Fig. 2.
As previously pointed out, authors who proposed the entropy-basedmeasures have only been reporting values of Stot (the solid line in Fig. 2)in their studies, but not the individual Si terms. From Fig. 2 it appearsthat, at least with this particular mixing process, even though Stotkeeps increasing as mixing continues, it is possible for the individual Siterms to undergo non-monotonic changes—see the curve for Si = 3 inFig. 2. This implies that, under such a definition of the entropy ofmixing,the degree of mixedness associated with a particular cell can be de-creasing even when that of the entire mixture is always increasing,
Table 2Initial conditions of the mixing process considered in terms of pj/i.
pj/i Cells
i = 1 (cell 1) i = 2 (cell 2) i = 3 (cell 3)
Species j = 1 (blue particles) 1/12 0 11/12j = 2 (red particles) 0 1/6 5/6j = 3 (green particles) 17/36 17/36 1/18
and the local entropy may reach its maximum point before mixinghas reached the steady state.
We can readily explain this non-monotonic trend in the localentropy if we consider the properties of the Shannon entropy stat-ed in Section 1. We see that in Eq. (2), for a fixed cell i, the “localentropy” Si is associated with the three pj/i of that cell. As mixingcontinues, these probabilities do not keep becoming more equalised;instead, they tend towards the overall fractions of species in themixture, that is, pj/i(τ → ∞) = pj, different for each j in our case.As a result, each Si does not have to be increasing at all timesduring mixing.
3. The new “per-species entropy” concept as a parallel
Wewill now introduce an alternative definition of the total entropyof mixing that is analogous to that defined in Section 2. Recall that thelatter is found by first computing Si, that is, assigning an entropy toevery cell, and then computing Stot, the total entropy of the mixture,by taking a weighted arithmetic mean of the Si terms. For our paralleldefinition, we recognise that particles can be categorised not only bycells, but equally well by species; thus, it is possible to assign an entropyto every species first—let us call this the “per-species entropy”, Rj—andthen take a weighted arithmetic mean of Rj to find the corresponding“total”, which we may denote with Rtot (the letter R is used instead ofS here to help differentiate between the two different treatments).Mathematically, this parallel definition says that
Rj ¼ −Xnci¼1
pi= j lnpi= j ð6Þ
where pi/j is now the converse of pj/i in Eq. (2): pi/j is the conditionalprobability that a random particle picked from the mixture is in cell i,given that the particle is of species j; or, it is the fraction of particlesof species j that are in cell i. It follows that the corresponding “totalentropy” should be expressed as
Rtot ¼Xns
j¼1
pjRj ð7Þ
where pj is simply the fraction of particles of species j in the mixture.Now we can examine Rj and Rtot during the same mixing process
described above. Note that the values of pi/j in Eq. (6) can be readilyobtained from the already available data for pj/i in Fig. 1 using thedefinition of conditional probability twice, with
pi= j ¼ pj=i � pi=pj ð8Þ
Table 2 and Fig. 3 display the initial conditions and the data of thesame mixing process in terms of pi/j, as opposed to Table 1 and Fig. 1where those are expressed in terms of pj/i. Subsequently, we cancalculate the per-species entropies and their corresponding totalentropy during mixing. These are plotted together in Fig. 4.
It appears in Fig. 4 that, for the very same mixing process, allthe per-species entropies as well as the total entropy increasemonotonically during mixing. This suggests that the degree ofmixedness associated with any particular species always increaseswith mixing.
Such results are expected because in Eq. (6), for a fixed species j, the“per-species entropy” Rj is associated with the three pi/j of that species.These probabilities indeed tend towards equalisation during mixing asparticles of each species are distributed more and more evenly amongthe three equal size cells. With pi/j(τ → ∞) = pi = 1/3, Rj will continueto increase and gradually converge to its maximum possible value,that is, Rj(τ → ∞) = ln nc = ln 3 for j = 1, 2, 3.
0.0
0.2
0.4
0.6
0.8
1.0
0 1 2 3 4 5
pi/
j
i=1, j=1 i=1, j=2 i=1, j=3
a b
c d
e f
0.0
0.2
0.4
0.6
0.8
1.0
0 1 2 3 4 5
pi/
j
i=2, j=1 i=2, j=2 i=2, j=3
0.0
0.2
0.4
0.6
0.8
1.0
0 1 2 3 4 5
pi/
j
i=3, j=1 i=3, j=2 i=3, j=3
0.0
0.2
0.4
0.6
0.8
1.0
0 1 2 3 4 5
pi/
j
i=1, j=1 i=2, j=1 i=3, j=1
0.0
0.2
0.4
0.6
0.8
1.0
0 1 2 3 4 5
pi/
j
i=1, j=2 i=2, j=2 i=3, j=2
0.0
0.2
0.4
0.6
0.8
1.0
0 1 2 3 4 5
pi/
j
i=1, j=3 i=2, j=3 i=3, j=3
Fig. 3. (a). Solutions for the simulatedmixing process in terms of pi/j; cell 1, i=1. (b). Solutions for the simulatedmixing process in terms of pi/j; cell 2, i=2. (c). Solutions for the simulatedmixing process in terms of pi/j; cell 3, i= 3. (d). Solutions for the simulated mixing process in terms of pi/j; blue particles, j=1. (e). Solutions for the simulated mixing process interms of pi/j; red particles, j = 2. (f). Solutions for the simulated mixing process in terms of pi/j; green particles, j = 3.
93Z. Gu, J.J.J. Chen / Powder Technology 266 (2014) 90–95
4. Phase-space representations of individual entropic terms
Because we have chosen ns = nc = 3, the individual contributionsof Si to Stot or Rj to Rtot can also be illustrated graphically in a two-dimensional manner, as shown in Fig. 5. Two of the three probabilitiesassociated with either Si or Rj are shown on the two axes (the thirdprobability is found by subtracting the first two from unity), so thatevery point in the right-angle triangular area corresponds to a uniquecombination of such probabilities; one may think of Fig. 5(a) or (b) asthe phase-space representation of Figs. 2 or 4, as the former no longershows τ explicitly. Fig. 5, suitable for an entropic term involving threeprobable outcomes, is an analogue of Fig. 6, which Shannon used toillustrate entropies involving only two outcomes [1].
To illustrate, consider the trajectory of Si = 3 in Fig. 5(a). The state ofcell 3 at τ= 0 is represented by point “A”. The coordinates of point “A”on the diagram are p(j = 1)/(i = 3) = 11/18 and p(j = 2)/(i = 3) = 5/18,corresponding to the initial conditions seen in Table 1; on the otherhand, the value of Si = 3 at τ = 0 is shown by the contour line passingthrough point “A” as being equal to approximately 0.9. During theprocess of mixing from τ = 0 to τ → ∞, the trajectory of Si = 3 crossesdifferent contours in such a way that Si = 3 initially increases andthen decreases for this particular local entropy term. Note that thisnon-monotonic change of a local entropy term is a consequence of thechosen initial conditions. Fig. 5(b), on the other hand, has the trajectoryof every per-species entropy constantly moving up the gradient of Rj,leading to monotonic changes in all Rj terms.
0.0
0.2
0.4
0.6
0.8
1.0
1.2
0 1 2 3 4 5
En
tro
py
, R
/nat
j=1 j=2 j=3 tot
Fig. 4. Per-species and total entropies (Rj and Rtot) during the simulated mixing process.
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
p( j=1 ) / i
p (j=2
)/i
1.0986
0.200.40
0.500.60
0.700.80
0.90
1.001.04
1.071.09
(i = 1)
(i = 2)(i = 3)
A
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
p( j=1 ) / i
p (j=2
)/i
i = 1i = 2i = 3τ = 0τ → ∞
0.8
0.9
1 0.200.40
0.500.60
0.8
0.9
1
j = 1
a
b
94 Z. Gu, J.J.J. Chen / Powder Technology 266 (2014) 90–95
We can also confirm that the three pj/i for each i in Fig. 5(a) donot completely equalise as is expected in the case of the three pi/jfor each j in Fig. 5(b). Consequently, as is consistent with obser-vations from Fig. 3(d)–(f), the three pi/j for each j all tend towards1/3, making all Rj converge to the global maximum of the functionat ln3. The value of all Si at steady state is, by contrast, smallerthan ln3.
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
p( i=1 ) / j
p (i=2
)/j
1.0986
0.700.80
0.90
1.001.04
1.071.09
(j = 1)
(j = 2)(j = 3)
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
p( i=1 ) / j
p (i=2
)/j
j = 2j = 3τ = 0τ → ∞
Fig. 5. (a). Graphical illustration of the contributions of Si to Stot or Rj to Rtot duringmixing; Si as a function of p(j=1)/i and p(j=2)/i. (b). Graphical illustration of the contributionsof Si to Stot or Rj to Rtot during mixing; Rj as a function of p(i = 1)/j and p(i = 2)/j.
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
En
tro
py
(n
at)
Probability, p
Fig. 6. Shannon entropy of an event with two probable outcomes [1].
5. Conclusions and further remarks
We have provided an analysis of two parallel definitions of theentropy of mixing, underscoring the fact that the individual contri-butions of entropies behave differently depending on whetherthese individual entropic terms are based on probabilities firstgrouped by species (yielding the local entropies), or by cells (yieldingthe per-species entropies).
In addition, we would like to include some comments on theabove findings. Firstly, we have shown that whether an entropicterm may potentially exhibit non-monotonic changes depends onwhether the associated probabilities will constantly equalise duringmixing—the reason that, in our example, the three Rj changemonotonically and converge to lnnc is simply because the cellshave an equal size such that all pi/j tend towards to 1/nc. That is tosay, we would have observed the opposite trends for the two defini-tions if the cells are of different sizes (i.e., not all pi are equal),or every species actually makes up a same fraction of the mixture(i.e., all pj are equal). Nevertheless, one can argue that because it ismuch more convenient to have equal size cells but more commonto have species with different overall compositions, so our findingsindeed correspond to the more typical cases.
Secondly, it is important to note that our case study, which as-sumes an idealised mixture and uses a simplified model for mixing,is of illustrative nature only. We have assumed that particles areonly distinguishable by colour but not by size or weight, and haveassumed that the number of particles is sufficiently large so that allprobabilities can be treated as continuous variables. In practicalcases, a number of complications—such as the segregation of parti-cles of different sizes, and the statistical fluctuations of cell countscaused by the finite number of particles in a cell—may potentiallyrender the trends in the individual entropic terms less obviousthan what we have presented here.
With these provisos, our theory is expected to be applicable tocases with arbitrary numbers of cells and species as long as oneknows exactly which probabilities are associated with each entropicterm considered.
95Z. Gu, J.J.J. Chen / Powder Technology 266 (2014) 90–95
References
[1] C.E. Shannon, A mathematical theory of communication, Bell Syst. Technol. J. 27(1948) 379–423 (623–656).
[2] I. Inoue, K. Yamaguchi, 固体混合の解析と確率過程 (Analysis of solid mixing andstochastic process), Res. Assoc. Powder Technol. 5 (1968) 1010–1018 In Japanese.
[3] K. Ogawa, S. Ito, A definition of quality of mixedness, J. Chem. Eng. Jpn 8 (1973)148–151.
[4] F.S. Lai, L.T. Fan, Application of a discrete mixing model to the study of mixing ofmulticomponent solid particles, Ind. Eng. Chem. Process. Des. Dev. 14 (1975) 403–411.
[5] M.A.I. Schutyser, J.T. Padding, F.J. Weber, W.J. Briels, A. Rinzema, R. Boom, Discreteparticle simulations predicting mixing behavior of solid substrate particles in arotating drum fermenter, Biotechnol. Bioeng. 75 (2001) 666–675.
[6] G.J. Finnie, N.P. Kruyt, M. Ye, C. Zeilstra, J.A.M. Kuipers, Longitudinal and transversemixing in rotary kilns: a discrete element method approach, Chem. Eng. Sci. 60(2005) 4083–4091.
[7] K. Alemaskin, I. Manas-Zloczower, M. Kaufman, Entropic analysis of colorhomogeneity, Polym. Eng. Sci. 45 (2005) 1031–1038.
[8] S. Masiuk, R. Rakoczy, The entropy criterion for the homogenisation process in amulti-ribbon blender, Chem. Eng. Process. 45 (2006) 500–506.
[9] M.M.H.D. Arntz,W.K. den Otter, W.J. Briels, P.J.T. Bussmann, H.H. Beeftink, R.M. Boom,Granularmixing and segregation in a horizontal rotating drum: a simulation study onthe impact of rotational speed and fill level, AIChE J. 54 (2008) 3133–3146.
[10] A. Guida, A.W. Nienow, M. Barigou, Shannon entropy for local and globaldescription of mixing by Lagrangian particle tracking, Chem. Eng. Sci. 65(2010) 2865–2883.