entropy and informationpaulcockshott.co.uk/publication-archive/talks... · 2019-03-11 · entropy...
TRANSCRIPT
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/Contribution
Entropy and InformationIn science and political economy
Paul Cockshott1
1School of Computer Science,University of Glasgow
October 2, 2012
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/Contribution
Outline
1 Motivation : Information Work and MoneyWhat is informationSmith and BabbageBut What is Work?
2 Understanding entropyMaxwellBoltzmannShannon
3 Our Results/ContributionInformation can be applied to industrial processesCopying technologiesParallel transmission
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/Contribution
Outline
1 Motivation : Information Work and MoneyWhat is informationSmith and BabbageBut What is Work?
2 Understanding entropyMaxwellBoltzmannShannon
3 Our Results/ContributionInformation can be applied to industrial processesCopying technologiesParallel transmission
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/ContributionBut What is Work?
What is information
How does it relate to wealth and work and energy
How does it help us understand the economy
We will look at Adam Smith, James Watt, Charles Babbage, KarlMarx, James Clerk Maxwell etc
Labor was the first price, the original purchase-moneythat was paid for all things. (Smith)
The cost of any article may be reduced in its ultimateanalysis to the quantity of labour by which it wasproduced. (Babbage)
Economy of time, to this all economy ultimately reducesitself. . . . [E]conomy of time, along with the planneddistribution of labour time among the various branches ofproduction, remains the first economic law on the basisof communal production. (Marx)
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/ContributionBut What is Work?
From these thinkers we get the common idea that value = worktimeAt about the same time as one Adam Smith was professor of MoralPhilosophy in Glasgow, and was setting out a coherent formulationof the labour theory of value, Dr Black of the department ofNatural Philosophy along with a technician, one James Watt, werelaying the foundations for a proper understanding of heat andtemperature. These two exercises have more in common thanmight be imagined. Reflection upon it, brings out how conceptsfrom engineering science, from the practice of material production,paralleled and became the foundation for classical economics.
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/ContributionBut What is Work?
Engines
Figure: The steam engines of Newcomen and Watt
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/ContributionBut What is Work?
Table
Moral Philosophy Natural Philosophy
Price in gold Temperature on anguineas of whisky mercury thermometer of whisky
Specific labour Specific heat ofcontent of gold mercury
Value of whisky Heat content of the whisky
Labour required to Thermal energy ofmake whisky hot whisky.measured in hours measured in foot pounds
or horse-power seconds
Ability to work or Ability to work orlabouring power of horse-power of thedistillery workers distillery engine ( raising barrels ? )
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/ContributionBut What is Work?
Ideas
Heat → WorkWork → ValueBut animals too can work, does the labour of a horse create valueor only human work.How does human and animal work differ?Since Watt, artificial power has eliminated the horse, but thehuman worker survives. Why?
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/ContributionBut What is Work?
The concept of Purpose
A spider conducts operations that resemble those of aweaver, and a bee puts to shame many an architect inthe construction of her cells. But what distinguishes theworst of architects from the best of bees is this, that thearchitect raises his structure in the imagination before heerects it in reality. At the end of every labour process weget a result that already existed in the imagination of thelabourer at its commencement. He not only effects achange of form in the material on which he works, but healso realises a purpose of his own that gives the law tohis modus operandi, and to which he must subordinatehis will. (Marx, Cap I pp. 177–8)
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/ContributionBut What is Work?
Hold on!
Are animals really lacking in purpose?The spider may be so small, and her brain so tiny, that it seemsplausible that blind instinct, rather than the conscious prospect offlies, drives her to spin. But it is doubtful that the same applies tomammals.The horse at the plough may not envisage in advance the corn hehelps to produce, but then he is a slave, bent to the purpose of theploughman. Reduced to a source of mechanical power, overcomingthe dumb resistance the soil, he is readily replaced by a John Deere.Does the wolf stalking its prey not intend to eat it?
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/ContributionBut What is Work?
Information
Turn to Marx’s human example, an architect, and his argumentlooks even shakier. For do architects ever build things themselves?They may occasionally build their own homes, but in general whatgives them the status of architects is that they don’t get theirhands dirty with anything worse than India Ink.The key to our ability to build complicated structures is that wecan store the information required to build them in an externalpaper and ink plan and then follow the plan.Stored information is what distinguishes our work from animals.
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/ContributionBut What is Work?
Links
Watt develops the steam engine
Smith studies division of labour
De Prony applied Smith’s division of labour to producing logand trig tables
Babbage studied this, and extracted the general principle ofthe division of mental from manual labour. Babbage and hisfriend, the astronomer, John Herschel, checking manuallycalculated tables, Babbage, finding error after error, wasdriven to exclaim ’I wish to God these calculations had beenexecuted by steam’.
From this came the difference engine which mechanised DeProny’s manual division of labour in the production of tables
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/ContributionBut What is Work?
Engines
Figure: The difference engine of Babbage, note the sequence controller on the right
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/ContributionBut What is Work?
Links
1 Babbage needs precision engineering to make difference engine
2 He studies most existing production technologies and developsprecision die casting
3 He writes the On the Economy of Machinery andManufactures,
4 Marx, in London studying political economy, comes accrossBabbage’s work and uses it as the basis for his understandingof automated ( self acting ) factory production.
5 Babbage’s idea that the key to automation is the developmentof automatic sequential control mechanisms, taking controlfrom the worker, gives Marx the key to the alienation oflabour in the automated factory
Smith + Watt → Babbage → Marx → communism
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/Contribution
MaxwellShannon
Gas initially in equilibrium. Demon opens door only for fastmolecules to go from A to B, or slow ones from B to A.→ Slow molecules in A, fast in B.� B hotter than A, and can be used for power.Information or intelligence has produced energy.....
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/Contribution
MaxwellShannon
Formula for Entropy
Maxwell’s proposed counter-example to the second law wasexplicitly based on atomism. With Boltzmann, entropy is placed onan explicitly atomistic foundation, in terms of an integral overmolecular phase space.
S =−k
ˆf (v) log f (v)dv (1)
where v denotes volume in six-dimensional phase space, f (v) is thefunction that counts the number of molecules present in thatvolume, and k is Boltzmann’s constant.
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/Contribution
MaxwellShannon
Shanons equationThe communications engineer Shannon introduces the concept ofentropy as being relevant to sending messages by teletype.
H =−n
∑i=1
pi log2 pi (2)
The mean information content of an ensemble of messages isobtained by weighting the log of the probability of each messageby the probability of that message. He showed that no encoding ofmessages in 1s and 0s could be shorter than this
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/Contribution
MaxwellShannon
Chaitin
Chaitin’s algorithmic information theory defines the informationcontent of a number to be the length of the shortest computerprogram capable of generating it. This introduction of numbers isa slight shift of terrain. Shannon talked about the informationcontent of messages. Whereas numbers as such are not messages,all coded messages are numbers. An information theory defined interms of numbers no longer needs the support of a prioriprobabilities.Whereas Shannon’s theory depended upon the a priori probabilityof messages, Chaitin dispenses with this support.
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/Contribution
MaxwellShannon
Mandelbrot
Figure: The Mandelbrot set, a complex image generated from a tinyamount of information. In fact it uses the formula z = z2 + c where z is acomplex number.
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/Contribution
MaxwellShannon
Links
1 Babbage and Marx thought human work was the source ofvalue - an objectivist theory. Marx: value a crystalization ofhuman energy.
2 20th century economics rejected this in favour of saying thatvalue was something subjective.
3 The problem with the work time theory was that it had noplausible mechanism.
4 It was not till the 20th century that mathematician computerscientists came up with plausible mechanisms for the theory.
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/Contribution
MaxwellShannon
Links: Turing, Capek - universality
1 Prague 1921 Capek invents the idea of the Universal Robot.The Universal Robot was an android that could do anyproductive task
2 England 1936 Turing (re-)invents Babbage’s idea of theUniversal Computer – one which can compute anymathematical function. Cycles of the Unversal Turingmachine are the fundamental cost metric for algorithms.
3 London 1938 World’s first science fiction TV broadcastRossum’s Universal Robots by Capek, introduces the notion ofthe Robot to the English.
4 Here is the first key to understanding Babbage and Marx’sideas of value
1 we humans are the only universal robots so far known,2 thus human cycle times are the basic cost metric for any task
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/Contribution
MaxwellShannon
Links: Boltzmann, von Neumann, Machover
1 1872 Boltzmann founds entropy on statistical mechanics
2 1938 von Neumann’s Model of General Economic Equilibriumtheoretically applies matrix mechanics to the economy.
3 1945 von Neumann invents the current model of universalcomputer.
4 1984 at UCL recursion theorist Machover shows that Babbageand Marx’s theory of value follows as a necessary consequenceof Boltzmann’s entropy maximisation in chaotic marketeconomy.
5 1998 Shaikh The Empirical Strength of the Labor Theory ofValue applies von Neumann’s maths and a von Neumannmachine to empirically test Machover, and show that Babbageand Marx were right.
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/Contribution
Information can be applied to industrial processesCopying technologiesParallel transmission
There are three fundamental ways by which the flow through anyproduction process can be increased.
1 Accelerating the production cycle.
2 Parallelizing production.
3 Eliminating wasted effort.
These are all well studied. What is less understood is the role thatentropy and information play in this.
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/Contribution
Information can be applied to industrial processesCopying technologiesParallel transmission
Paper making
Consider the process of producing a book we see that theproduction process encompasses two opposite phases.First, we have the production of the paper. This is anentropy-reducing process. The blank sheets of paper obviouslyhave low information content with respect to human language, butthey also constitute a low entropy state with respect to the rawmaterial.In a sheet of paper the cellulose fibres are constrained in bothorientation and position. This implies a reduction in the volume ofstate space that the fibres occupy, and thus, from Boltzmann, acorresponding reduction in entropy.
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/Contribution
Information can be applied to industrial processesCopying technologiesParallel transmission
Printing
Next, there is the writing of the text—whether by hand, as in thedistant past, or using a printing press. This is an entropy-increasingprocess. Imagine that the text to be printed exists as binary datain a file on disk, encoded using ASCII or UNICODEClearly the book contains this information, since by sending thebook in the post to someone we enable them to recreate therelevant binary file. Thus, by the equivalence of information andentropy, we have increased the entropy of the book relative to theblank sheets of paper.
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/Contribution
Information can be applied to industrial processesCopying technologiesParallel transmission
2 phases
In the first phase a low-entropy material is created; in the secondphase the entropy of this material is increased in a controlled way.Initially natural information is removed; subsequently anthropic orhuman-created information is added. The natural informationremoved in the first stage is of no interest to us, while that addedin the second stage is dictated by our concerns.
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/Contribution
Information can be applied to industrial processesCopying technologiesParallel transmission
Is this general?
It is easy to see the relevance of information theory to the printingindustry. Its product, after all, contains information in the everydayas well as the technical sense of that word. Does this approachprovide insights into how other production processes function?
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/Contribution
Information can be applied to industrial processesCopying technologiesParallel transmission
Weaving and Spinning
For a rather different example, consider the process of producingcloth. The starting material is wool or cotton fibres in a randomtangled state. This is first carded to bring the fibres into roughalignment, and then simultaneously twisted and drawn to spin thefibres into yarn. In the yarn both the volume and orientation aresharply reduced. Energy is used to reduce the entropy of thecotton. The weaving of the cotton then increases the entropy byallowing two possible orientations of the fibres at right angles toone another (or more if we take into account the differences inpossible weave).
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/Contribution
Information can be applied to industrial processesCopying technologiesParallel transmission
Car production
Other industries that use thin, initially flat materials clearly have alot in common with printing. The manufacture of car body partsfrom sheet steel, or the garment industry, share the pattern ofproducing a low entropy raw material and adding information to it.In pressed steel construction, added information is encoded in theshape of the dies used to form the car doors, roof panels etc.We can quantify it using Chaitin’s algorithmic information theory,as proportional to the length of the numerically controlled machinetool tape that is used to direct the carving of the die.
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/Contribution
Information can be applied to industrial processesCopying technologiesParallel transmission
Replication
If steam powered the industrial revolution, the technologies ofreplication were the key to mass production. The classic exampleof the importance of accurate replication was in the production ofthe Colt revolver in the mid 19th century. Prior to Coltestablishing his factory the gun trade was dominated by handicraftmanufacturing techniques. The different parts of a gun’smechanism were individually made by a gunsmith so that theyfitted accurately together.For parts to be interchangeable they must be made to very precisetolerances. This improvement in accuracy of production involvesthe parts having a lower entropy, occupying a smaller volume ofphase space, than the old hand made parts. Again by theequivalence of information and entropy this means that thestandardized parts embody less information than the hand madeones.
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/Contribution
Information can be applied to industrial processesCopying technologiesParallel transmission
Accuracy and information
Prior to the introduction of numerically controlled machine tools,replicated parts had to be composed of circular and planarelements which could be produced on lathes or milling machines.The limited information content of these can be seen when youconsider that in turning a smooth bore gun barrel one only has tospecify the inner and outer radii and its length. If an axle and abearing are being produced separately to fit together, then onewants the uncertainty in the surface of the bearing, given thesurface of the axle, to be reduced below a certain limit.
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/Contribution
Information can be applied to industrial processesCopying technologiesParallel transmission
Accuracy continued
���������������������������������
���������������������������������
��������������������������������A
B
Figure: When inserting axle A into bearing B we want to minimize theconditional information H(B|A), between B and A.
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/Contribution
Information can be applied to industrial processesCopying technologiesParallel transmission
Binary cross section
000000111111111111111111111111111110000000000001111111111111111111111111111100000000000001111111111111111111111111110000000000000001111111111111111111111111000000000000000001111111111111111111111100000000000000000001111111111111111111110000000000000000000000111111111111111110000000000000000000000000011111111111110000000000000000000000000000000111111100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
Figure: A slice through the axle.
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/Contribution
Information can be applied to industrial processesCopying technologiesParallel transmission
Generator
program circ ;const
b: array [boolean ] of char =( ‘1’ , ‘0’ );c =20;r =18;
vara: array [-c ..c ,-c ..c ] of boolean ;
begin
a←√
ι20 + ι2
1 < r ;write(ba);
end .
We cannot guarantee to have found the shortest such program.1
1The program is in Vector Pascal which is fairly concise .Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/Contribution
Information can be applied to industrial processesCopying technologiesParallel transmission
Bearing Clearly if the bearing exactly fitted the axle the expandedencoding for a slice through the bearing would be an array similarbut with 1s and 0s swapped round. This can be produced by atrivial change to the program circ, the addition of a singlestatement.write(bnota);replaces the line:write(ba);This must come close to minimizing the conditional entropy of thetwo parts.
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/Contribution
Information can be applied to industrial processesCopying technologiesParallel transmission
Imperfect construction
0000001111111111111111111111111111100000000000011111111111111111111111111111000000000000011111111111111111111111111100000000000000011111111111111111111111110000000000000000011111111111111111111111000000000000000000001111111111111111111000000000000000000000001111111111111111100000000000000000000000000011110111111000000000000000000000000000000000001110000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000
Figure: Part of a pin with a fault on its circumference.
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/Contribution
Information can be applied to industrial processesCopying technologiesParallel transmission
means added code
we would need to add the following lines to the generator of A’ tomake the bitmap for B:ar,1← false;ar,2← false;ar+1,0← false;ar,0← false;ar−1,−1← true;
write(bnota);This contains extra information, to correct the bitmap of A’ togenerate that of B. In pre-industrial production, the extras steps inthe generator program would imply additional steps of filing andgrinding to make parts fit.
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/Contribution
Information can be applied to industrial processesCopying technologiesParallel transmission
Pottery
1 Hand formation
2 Turn on wheel – only have to supply diameter as information
3 Casting – parallel information transmission, the Romansamian ware industry based on such casting is the firstparallelised mass production of consumer goods
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/Contribution
Information can be applied to industrial processesCopying technologiesParallel transmission
Law of reproduction
The algorithmic information in repeated production grows by a lawof the form H(P)≤ H(c) + log n where P is the total productmade up of n repetitions of c . If we look at the process ofreproduction as a whole there are two terms the first given by thecomplexity of the original and a second logarithmic term given bythe number of repetitions.
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/Contribution
Information can be applied to industrial processesCopying technologiesParallel transmission
Law of reproduction - pottery
In the case of the Samian ware pottery there is the original work ofproducing the master or pattern piece which corresponds to H(c),but then the number of copies that could be made growsexponentially with the number of successive steps of copying: ifthe master is used directly to produce the pots then L pots can bemade, where L is the lifetime of the master. If the master producesmoulds which in turn produce the pots then L2 pots can be made,etc. Invert the relationship and we find that the number ofsuccessive steps of copying will be related to the number of potsproduced n as logL(n), a relationship suggested by the predictionsof information theory.
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/Contribution
Information can be applied to industrial processesCopying technologiesParallel transmission
A key advantage of moulding is in the information transmission
On a potters wheel the information relating to shape istransmitted serially by the potter moving his hands upwardsand controling the diameter
I samian ware the information about shape is transmitted inparallel accross the whole surface.
The information content that is transmitted can also be muchhigher. On a turned pot the information consists of only twonumbers for each height from base.
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/Contribution
Information can be applied to industrial processesCopying technologiesParallel transmission
Engines
a b
a. Spinning wheel,b.Compton’s Mule. A multiplicity ofspindles and the moving frame substituted for the stretching
movement of the hand spinner’s arms.
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/Contribution
Information can be applied to industrial processesCopying technologiesParallel transmission
Parallelism elsewhere
There are may other instances where increased productivity hasarisen from switching from serial to parallel transmission ofanthropic information in the 2nd stage of a production process:
1 The shift from manuscripts to printed books.
2 The invention of iron casting replacing the work ofblacksmithing. Look at the streets of Edinburgh New Town,the profusion of iron railings came from the cheap mouldingtechnologies of the Carron ironworks.
3 The shift from Edison phonograph drums to gramophonediscs.
4 The change from wire-wrap backplanes to printed circuitbackplanes in computing.
Productivity advances in chip techology all stem from the parallelimaging processes used in photo-lithography.
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/Contribution
Information can be applied to industrial processesCopying technologiesParallel transmission
Cast Iron
The ability to melt and cast iron as opposed to just forging ironproducts allowed the mass production of iron goods
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/Contribution
Information can be applied to industrial processesCopying technologiesParallel transmission
Sound Recording
The cylindrical phonograph could not be readily produced bystamping from a mould, but the gramophone could be.
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/Contribution
Information can be applied to industrial processesCopying technologiesParallel transmission
Computer Circuits
Mid 1960s computers (left) had to be sequentially wired usingdiscrete wiring. By the early 1980s the equivalent circuit wereprinted (right) opening the way for mass production and thedevelopment of computers as a consumer good.
Author, WPC Information and production
Motivation : Information Work and MoneyUnderstanding entropy
Our Results/Contribution
Information can be applied to industrial processesCopying technologiesParallel transmission
1
2 1832 Babbage, C.: 1832, The Economy of Machinery andManufactures, London.
3
4 1999 Chaitin, G.: 1999, Information and randomness: Asurvey of algorithmic information theory, The Unknowable,Springer, Signapore.
5
6 1875 Maxwell, J. C.: 1875, The Theory of Heat, London.
7
8 1906 Marx, K., :1867, Capital, 1906 edition, downloadabletext and pdf from Google Books.
Author, WPC Information and production