bits, qubits and neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · bits, qubits...
TRANSCRIPT
![Page 1: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/1.jpg)
Bits,QubitsandNeurons
Thenear(andnotsonear)futureofCompu9ngBillCamp
![Page 2: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/2.jpg)
Preamble
• (ThiscouldbeusedasacoursesyllabusJ)
Let’slookatunexpectedconnec1ons
![Page 3: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/3.jpg)
TheIsingModelprovides
Thesimplestnon-trivialmodelof: amagnet,aferroelectric,aliquid,abinaryalloy,aglass,aquantumfieldtheory,quantumcompu9ng,neuralnets,
…
![Page 4: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/4.jpg)
TheIsingModelinaparallel
fieldH=-Σi,j{Jijσziσzj-Σhiσzi}σzi=+/-1
![Page 5: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/5.jpg)
OmnisComputa9ointrespartesdivisusest#
• Turing/VonNeumannBased
• Neuro-Inspired
• Quantum-based
#Allofcompu9ngisdividedintothreeparts--withapologiestothosewhohavereadCaesar
![Page 6: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/6.jpg)
QuantumCompu9ng
GateandCircuit-based
Adiaba9cQuantumRelaxa9on-based
![Page 7: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/7.jpg)
Awarp-speedlookatquantummechanics
• Newtongotitrightif…• Thingsweren’ttoo9ny• Tooheavy• Ortoofast!• Newton’sphysicsisdeterminis9candreversibleinthesmallbutirreversibleinthelarge…tryunmixingfluidsbyreversingthepaddle.
![Page 8: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/8.jpg)
Demys9fyingQuantumPhysics
• “Things”aredescribedbytheir“State”,avectorinalinearspaceofvectors(AHilbertSpace).
• ReversibleandDeterminis9c– Un9lyoumeasuresomething
• Newton’sphysicalvariablesarenolongernumbers– Posi9on,energy,momentum,angularmomentum,dipolemoment,…
• Theyare(self-adjoint)operatorsinthatHilbertspace
![Page 9: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/9.jpg)
Demys9fyingQuantumPhysics
• Self-adjointoperatorsA=A* (Thatis,Aij=Aji*)
• Ordermacers:Operatorsdon’talwayscommutee.g.,[x,p]=(xp-px)=ih/2π Heisenberg!
• Thingsaremeasuredusinginnerproducts– Innerproduct:<ψ|φ> = Σjψj
*φj(j=1,N)– MeasuredEnergyinstateψ =<Η>=<ψ|{Η|ψ>
![Page 10: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/10.jpg)
Schrödinger’sEqua9on
• iδt|ψ(t)>=Η|ψ(t)>(weletħ=h/2π=1)
• Thismeansthat|ψ(t)>=e-iΗt |ψ(0)>
• SinceΗ=Η*,theevolu9onoperator,U(t)=e-iΗt , isunitary:
• U*=U-1U*U=I
![Page 11: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/11.jpg)
QUBITS
=|0>isastatewithitsqubit=0=|1>isastatewithitsqubit=1Fornormalbitsthisisthewholestory.Notsoforcubits!
1
0
0
1
![Page 12: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/12.jpg)
QUBITS
=|ψ>isastatewithitsqubitina superposi9onofanemptyandafull bit:
|ψ>=α|0>+β|1>Thisseemsweird;butitistrueandisthesourceof(nearly)allofquantumcompu9ng’spromise!
α
β
![Page 13: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/13.jpg)
Quantumparallelism
• Wecreateaquantumcircuitusingquantumgates.
• Q-gatesareunitaryoperatorsthatoperateonthebit-states.
• Wecancreateastar9ngstateforallthebits:
• |Ψ0>=|ψ0,1;ψ0,2;ψ0,3;…ψ0,j;…ψ0,N>• Where|ψ0,j>(=α0,j|0>+β0,j|1>)istheini9alstateofthejthcubit
![Page 14: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/14.jpg)
QuantumparallelismTodoacalcula9onwemovetheN-bitstateΨ0throughthesuccessivesetofLQ-gates:|Ψf>=ULUL-1
…Uk…U1|Ψ0>
|Ψf>=|ψf,1;ψf,2;ψf,3;…ψf,j;…ψf,N>And
|ψf,j>=αf,j|0>+βf,j|1>So,weevolve2NbitpaSernsatonce!Unfortunatelyallthatparallelismcollapsesatoutput!
![Page 15: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/15.jpg)
PauliOperatorsand1-bitQ-gates
11
1-1=√2�H=σx+σz(HadamardGate)
10
0i=S(phasegate)
10
01 =I
10
0-1=σz
01
10=σx
0-i
i0=σy
10
0√i=T(π/8gate)
![Page 16: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/16.jpg)
2-bitQ-gates:C-NOT
�
¢
|A> |A>
|B> |B+A>
![Page 17: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/17.jpg)
2-bitQ-gates:C-NOT
UC-NOT=1000010000010010
UC-NOTisunitary–checkitout!
![Page 18: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/18.jpg)
2-bitQ-gates:C-NOTexamples
UC-NOT
00
011011
� =
00
011110
!
![Page 19: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/19.jpg)
2-bitQ-gates:C-NOTgeneralcase
UC-NOT
αAαB
αAβBβAαB
βAβB
�
αAαB
αAβBβAβBβAαB
=!
![Page 20: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/20.jpg)
C-NOTisimportant
AmazinglyC-NOTandthesingle-bitQ-gates{I,σx,σy,σz}areallyouneedtotocreateanymul9-Q-bitgate.Theyformauniversalsetofbuildingblocksforquantumcircuitry
![Page 21: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/21.jpg)
QuantumNoise&Q-ECC• Areallimita9onofofquantumcompu9ngisnoise,whicheventuallybreaksthecoherenceofquantumsuperposi9onsofstates.• Fortunatelyquantumcodeshavebeeninventedwhich,ifnoiseisbelowathreshold,guaranteecoherence.• Nonetheless,thisisnotatrivialissueatall!
![Page 22: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/22.jpg)
Isquantumreallybecer?
Some9mes–inprinciple!Forexample,ThankstoBobGriffiths’productformforthequantumFouriertransform,QuantumalgorithmscandoFTsexponen9allyfasterthantheFFT.
![Page 23: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/23.jpg)
Isquantumreallybecer?
UnfortunatelyitisseeminglyimpossibletoinputanarbitrarystatetotheQFTItisalsoimpossibleinprincipletooutputtheQFTNonetheless,QFTenablesustodoquantumphasees9ma9on,whichallowsustodoquantumorderingandfactoring–exponen9allyfasterthanNumbertheore9csieves!
![Page 24: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/24.jpg)
Realquantumcomputers?
• Tomakequantumcompu9ngareality,weneedtobeableto– Buildlotsofrobustbits– Efficientlyperformalargesetofunitarytransforma9ons
– Prepareini9alstatesreliably– MeasureoutputsNoneofthesearecurrentlyingoodshapeforanycandidatetechnology
![Page 25: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/25.jpg)
Takeawaymessage
• Duetowavefunc9oncollapseuponmeasuringoutput,weonlygetN-wayinstead2N-wayparallelismintheoutput.
• Wecannotquerysimula9onsinmidstream• Onlyinafewcaseshavequantumalgorithmsbeenshowntohavehugegainsin9metosolu9on
• Quantumseemsdes9nedfornowtobebestforques9onsthatcanbewithminimaloutput
![Page 26: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/26.jpg)
Adiaba9cQuantumRelaxa9on
• Createacollec9onofcubits.PreparetheminthelowesteigenstateoftheHamiltonianofasimplequantumsystem.
• Slowlytransi9onthesystembyopera9ngonthequbitswithanincreasing“perturba9on”thateffec9velyremovestheoriginalsimpleHamiltonianandtransi9onsittoacomplexHamiltonianthatrepresentsa“Hard”problem.
• Outputthefinalstates.
![Page 27: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/27.jpg)
TopicalExample–IsingModelinatransversemagne9cfield
H(t)={1-τ(t)}H0+τ(t)H∞
H0=-Σhiσxi
H∞=-ΣJijσziσzj[σzj,σxi]=σyjδj,i
i
i,j
τ(0)=0,τ(∞)=1τissmoothlyIncreasing(σ×σ=iσ)!
![Page 28: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/28.jpg)
IsingQAR
• MaptheIsingmodelontoafeasiblegraph(network),G,withconnec9onsrepresen9ng{Jij}• {Jij}andGarechosentomaptheIsingproblemH∞ontotheNP-Hardproblemtobeacacked.• EvolvetheproblemontheQAR.
![Page 29: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/29.jpg)
IssuesAdiaba9cbehavior:Thislookslikeacon9nua9onmethodforthegroundstateofa9medependentHamiltonianThestar9ngandendingHamiltoniansdonotcommute;sotheydonotshareeigenvectorsEhrenfest’sTheoremguaranteessuccessoftheadiaba9capproxima9on(con9nua9on):ifevolu9onisslowenoughandifsymmetryissuesandlevelcrossingsdonotviolateassump9ons.
![Page 30: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/30.jpg)
Issues
Iamnotawareofresearchthatshowsdefini9velywhencon9nua9onsuccesscondi9onsaremetinQARs.Decoherenceduetoquantumnoiseis(inmyken)s9llpoorlyunderstoodinQARsQARsaremorelimitedinscopethancircuitquantumcomputers.Howlimitedisunknown.
![Page 31: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/31.jpg)
Neuro-mime9ccompu9ng
PartofaNeuralCortexformaratbrain
![Page 32: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/32.jpg)
BRAINCELL
Rathermorecomplexthanintegra9ngsigmoids,hyberbolictangents,orrec9fiedlinearfunc9ons
![Page 33: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/33.jpg)
Nonetheless,itworks!
![Page 34: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/34.jpg)
AliclehistoryofNNmilestones
![Page 35: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/35.jpg)
MACHINELEARNING/ARTIFICIALINTELLIGENCE
• In1943McCullochandPics(M-P)introducedthear9ficialneuronandpointedtopacernclassifica9onasfrontandcentertoatheoryofintelligence!
• In1949Hebbproposedthatlearningchangesthemorphologyofintelligentnetworks.“Therichgetricherandthepoorpoorer.”(training)
• In1949,Rosenblacintroducedtheperceptron:anM-Pnetworkwithtrainablesynapses.
• In1969,MinskyandPapertpointedouttheseverefailingsoftheoringinalperceptronandshowedwhatneededtobedonetomakeitaTuringmachine.
![Page 36: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/36.jpg)
MACHINELEARNING/ARTIFICIALINTELLIGENCE
• In1982,HopfieldintroducedtheHopfieldnet–thebasisformanyofourmodernadvances,thoughnotpowerfulbytoday’sstandards.
• In1982,Hintonetal.introducedRestrictedBoltzmannMachines
• In1984,Fukushima’sNeocognitronovercamemanyoftheproblemsofscale,transla9onandrota9on.
• In1985,AmitpointedoutthattheHopfieldNetwasinmanywaysiden9caltotheIsingSpinGlassinitsMeanFieldApproxima9on.
![Page 37: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/37.jpg)
MACHINELEARNING/ARTIFICIALINTELLIGENCE
• In1986,Hintonetal.introducedbackpropaga9onasalearningmethod.
• In1987,WolnesdiscoveredthatproteinfoldingisalsostronglyanalogoustothebehaviorofIsingSpinglasses–anddiscoveredminimalfrustra9onandspinfunnels.
• In1987,PedersonandAndersonshowedhowtospeedupSBMsandRBMsbyusingtheIsingMeanFieldapproxima9on
• In2004,LeCunandBocouemphasizedtheroleofstochas9cgradientdescentindeepnetworks.
![Page 38: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/38.jpg)
MACHINELEARNING/ARTIFICIALINTELLIGENCE
• In2006,Hintonetal.introduceddeepbeliefnetworks
• In2006,Bengioetal.analyzedDeepauto-encoders:Deepnetworkswithgreedylayer-by-layertraining
• In2012,Hintonetal.analyzedandemphasizedtheroleofhierarchicalabstrac9oninDeepNeuralNetworks.
![Page 39: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/39.jpg)
MACHINELEARNING/AI
• In2014,MehtaandSchwabshowedanexactmappingbetweenKadanoff’svaria9onalapproxima9onfortheRenormaliza9onGroupapproachtotheIsingModelanddeepneuralnets!
• In2015LeCunetaldemonstratedtherela9onshipofdeepNNstotheIsingspinglassinthesphericalapproxima9on–Insightsgalore!
![Page 40: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/40.jpg)
From:DeepLearningMethodsandApplica1onsLiDengandDongYu
![Page 41: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/41.jpg)
From:DeepLearningMethodsandApplica1onsLiDengandDongYu
StartofuseofDNNs
![Page 42: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/42.jpg)
Simplifiedneuron
InputAxon Synapse
dendrite
Σjwjxj+b fw0x0
w1x1
w2x2
OutputAxon
f(Σjwjxj+b)
Cellbody
Ac9va9onfunc9on
![Page 43: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/43.jpg)
Ac9va9onFunc9ons
Sigmoid:f(x)=1/(1+e-x)(mostlikebiology)HyperbolicTangent:f(x)=[ex-e-x]/[ex+e-x](mostusefulforIsingModelcomparisons)ReLU:f(x)=Θ(x)�x(Θ(x)istheunitstepfn)(currentlymostfavored)
![Page 44: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/44.jpg)
PrototypicalNeuralNetworks
���
HiddenLayersInputLayer
OutputLayer
![Page 45: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/45.jpg)
(RestrictedBoltzmannMachinelookslikethiswithoutthe(red)intra-layerconnec9ons)
TwolayersofaNetworktopologyforaStochas9cBoltzmannMachine
![Page 46: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/46.jpg)
Lossfunc9onforanRBM
H=-Σv,h{Jv,htzvtzh-Σi=v,hbitzi}vissummedoverthevisibleunitsvissummedoverthehiddenunits
tzi=0,1(=(2σzi-1)/2
ThisistheIsingmodelonabi-par1tegraph!
![Page 47: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/47.jpg)
Whatdowe(typically)dowithNNs?
• Aneuralnetworkispresentedwithtrainingdata.• Itsjobistolearnthatdatathenextrapolateformittocorrectlyrecognizeandclassifynewdata.
• Trivialexample:showtheNNmanypicturesofcatsandthenaskittoclassifyotherpicturesbywhethertheycontaincatsandperhapshowmanycatstheycontain,andperhapstheirbreeds,andperhapstheirmaturity,….
![Page 48: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/48.jpg)
HowNNsdoitisfascina9ng!• 1.Prepara9onofdata• 2.Ini9aliza9onofthenet• 3.Pre-trainingifused• 4.Forwardpropaga9on:learningbyac9va9ngneurons• 5.Back-propaga9on:changingtheweightsintheac9va9onfunc9onstominimizethe“Loss”func9on(theerrorinlearning)
• 6.repeat• Typicallytherearetwovisiblelayers(in,out)andoneormore“hidden”layers.
• 7.Outputop9malclassifica9ons
![Page 49: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/49.jpg)
DeepNeuralNetworks(DNNs)
• DNNsarenetworkswithmanyhiddenlayers• (typically10—20layers)
• Theyarehuge– (o�en100millionplusnetworkparameters)
• Theirinputlayeriso�enaDeepBeliefNetwork(DBN):astackofrestrictedBoltzmannmachinesusedonlearning
![Page 50: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/50.jpg)
DeepLearning
• InDeepLearning,learningandtrainingisdonelayer-by-layerfrominputthroughmanyhiddenlayerstooutput
• Aswegoupthestackoflayerslearning/trainingbecomesmoreabstractandpowerful.
• Thiswasarealsurprise!• Whyithappensisnotfullyunderstood
![Page 51: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/51.jpg)
DeepLearning
• Recently,evidenceismoun9ngthattheanswerliesintwopiecesofphysics:– theIsingModelanditsSpinGlasses– theresolu9onoftheirdeepphysicsviatheRenormaliza9onGroupTheory(RGT).
• Thisisalsohappeninginacemptstodevelopsuccessfulfoldingmethodsforproteins.
![Page 52: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/52.jpg)
Therestofthestory• Ihaveglossedovernearly75yearsofR&D!• Ihaven’ttoldyouhowRGTworks• Ihaven’ttoldyouhowseveralofus(myselfand,
independently,KenWilson,andacoupleofyearslaterMasuoSuzuki)discoveredthattheIsingModelind-dimensionsis1-1andontoa(scalar)quantumfieldtheoryind-1dimensions(50yearsago!)
• Ihaven’ttoldyouhowtheIsingmodelcanbeusedinshowingthatinaninfiniteuniversethereareaninfinite#ofBillCampstellingthisstorytoaninfinite#ofiden9calaudiences
• Ihaven’tdiscussedLandauer,orBlackholes,theholographictheoryofmemory,informa9onandMaldecenauniverses.
![Page 53: Bits, Qubits and Neuronssalishan.ahsc-nm.org/uploads/4/9/7/0/49704495/2017-camp.pdf · Bits, Qubits and Neurons The near (and not so near) future of Compung Bill Camp Preamble •](https://reader033.vdocuments.us/reader033/viewer/2022050420/5f8f69ca163d8d181b20434b/html5/thumbnails/53.jpg)
GodwillingandtheCreekdon’trise,there’salwaysnextyearJ