lec10-modeling-physics - hacettepe Üniversitesiaykut/classes/... · forwardmodel: supervised,...
TRANSCRIPT
![Page 1: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/1.jpg)
Lecture #10 – Modeling the Physical World
Aykut Erdem // Hacettepe University // Spring 2019
CMP722ADVANCED COMPUTER VISION
Video: The Jenga-playing robot (MIT)
![Page 2: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/2.jpg)
• graph structured data
• graph neural nets (GNNs)
• GNNs for ”classical network
problems”
Previously on CMP722Illustration: Kevin Hong // Quanta Magazine
![Page 3: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/3.jpg)
Lecture overview• physical scene understanding• intuitive physics• interaction networks• relation networks• visual interaction networks• learning physics engines via graph networks
• Disclaimer: Much of the material and slides for this lecture were borrowed from —Peter Battaglia’s slides on “Structure in physical intelligence”
3
![Page 4: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/4.jpg)
How do you understand a scene?
4
![Page 5: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/5.jpg)
How do you understand a scene?1. Parse it into physical objects and relations
2. Reason about the objects and their interactions
5
Attached? Fall?
Support
"Precarious"
![Page 6: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/6.jpg)
“Infinite use of finite means”- von Humboldt, on the productivity of language
6
"Precarious"
![Page 7: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/7.jpg)
Kenneth Craik, “The Nature of Explanation”, 1943: "If the organism carries a 'small-scale model’ of externalreality and of its own possible actions within its head, it is able to try out various alternatives, conclude which is thebest of them, react to future situations before they arise, utilize the knowledge of past events in dealing with thepresent and future, and in every way to react in a muchfuller, safer, and more competent manner to theemergencies which face it." (pg 61)
"This concept of 'thinghood' is of fundamental importancefor any theory of thought." (pg 77)
7
![Page 8: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/8.jpg)
Claim: Human intelligence is structured
• Objects and relations reflect decisions made by evolution, experience, and task demands about how to represent the world in an efficient anduseful way
• Structure in our core cognitive knowledge evident very early in infancy(Spelke)
• Model-building over recognizing patterns (Tenenbaum)
• Combinatorial generalization via compositionality ("infinite use of finitemeans”)
8
Founded on objects, relations, reasoning
![Page 9: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/9.jpg)
What is the mechanism of human intuitive physics?
9
Intuitive Physics Engine: the "physics engine in the head"
Battaglia, Hamrick, Tenenbaum, 2013, PNAS
![Page 10: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/10.jpg)
Experiments: What will happen? Why?
10
Will it fall? In which direction? Different masses
Comples scenes Infer the mass Predict fluids
Battaglia et al., 2013 Hamrick et al., 2016 Bates et al., 2015, 2018
![Page 11: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/11.jpg)
Message from cognitionHumans use richly structured representations of objects and relations
to reason about, and interact with, their everyday environment.
What insights does humans’ structured intelligence offer AI?
11
![Page 12: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/12.jpg)
We need better object- and relation-centricmodels in AIA graph is a natural way to represent entities and their relations: • “Nodes“ correspond to entities, objects, events, etc.• “Edges“ correspond to their relations, interactions, transitions, etc. • Inferences about entities and relations respect the graphical structure.
Graphs can capture data from many complex systems:
12
• Physical systems• Scene graphs• Social networks• Linguistic structure• Programs
• Search trees• Communication networks• Transportation networks• Chemical structure• Phylogenetic trees
![Page 13: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/13.jpg)
Intuitive physics as reasoning about graphs
13
![Page 14: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/14.jpg)
Intuitive physics as reasoning about graphs
14
![Page 15: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/15.jpg)
Interaction Network
Strong relational inductive bias: Deep learning architecture which operates on graphs
Related to the broad family of "Graph Neural Networks" (Scarselli et al, 2009; Li et al, 2015) and "Message-Passing Neural Networks" (Gilmer et al., 2017).Chang et al. (2016) also proposed a similar version in parallel.
15Battaglia et al., 2016, NeurIPS
![Page 16: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/16.jpg)
Interaction Network
16Battaglia et al., 2016, NeurIPS
![Page 17: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/17.jpg)
Interaction NetworkCan learn a general-purpose physics engine, simulating future states from initial ones
17Battaglia et al., 2016, NeurIPS
Gravitational forces Rigid collisions betweenwalls and balls
Springs and rigid collisions
![Page 18: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/18.jpg)
1000-step rollouts from 1-step supervised training
18Battaglia et al., 2016, NeurIPS
n-body
Gro
und
trut
hM
odel
Balls Strings
![Page 19: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/19.jpg)
Zero-shot generalization to larger systems
19Battaglia et al., 2016, NeurIPS
Ballsn-body
Gro
und
trut
hM
odel
Strings
![Page 20: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/20.jpg)
Interaction Network for system-level predictionsA "global model" can be added, which aggregates the per-object outputs to make predictions.
Can be trained to predict potential energy of a system, outperforming MLP baselines
20Battaglia et al., 2016, NeurIPS
![Page 21: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/21.jpg)
Relation NetworkRemove “object model” and predict global outputs only using “relation model”’s output
21
Raposo et al., 2017, ICLR workshop; Santoro et al., 2017, NeurIPS
![Page 22: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/22.jpg)
Relation Networks can infer relations in dot motion
22Santoro et al., 2017, NeurIPS
Trained on mass-spring systems
Generalizes to point-light walkers
Input Model Ground truth
Input Model Ground truth
![Page 23: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/23.jpg)
"Visual interaction network"
23
An interaction network augmented with a learnable perception system
![Page 24: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/24.jpg)
"Visual interaction network"
24
Multi-frame encoder (conv net-based) Interaction network
Watters et al., 2017, NeurIPS
![Page 25: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/25.jpg)
"Visual interaction network"
25Watters et al., 2017, NeurIPS
Can even predictinvisible objects, inferred fromhow they affectvisible ones
Spring Gravity Magnetic Billiards Billiards Drift
Exa
mple
Frame
Truth
Prediction
Exa
mple
Frame
Truth
Prediction
Table 1: Rollout Trajectories. For each of our datasets, we show an example frame, an example truefuture trajectory, and a corresponding predicted rollout trajectory (for 40-60 timesteps, depending onthe dataset). This top half shows the 3-object regime and the bottom half shows the 6-object regime.For visual clarity, all objects are rendered at a higher resolution here than in the training input.
8
Spring Gravity Magnetic Billiards Billiards Drift
![Page 26: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/26.jpg)
Learning to simulate more complex robotic systems
26
Alvaro Sanchez-Gonzalez, Nicolas Heess, Tobi Springenberg, Josh Merel, Martin Riedmiller, Raia Hadsell, Peter Battaglia
ICML, 2018
![Page 27: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/27.jpg)
Systems: "DeepMind Control Suite" (Mujoco) & real JACO
27
JACO Arm
DeepMind Control Suite (Tassa et al., 2018)
![Page 28: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/28.jpg)
28
Systems: "DeepMind Control Suite" (Mujoco) & real JACO
JACO Arm
![Page 29: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/29.jpg)
Kinematic tree of the actuated systemas a graphRepresenting physical system as a graph: • Bodies → Nodes
• Joints → Edges
• Global properties
Similar representation to: • Interaction Networks (Battaglia et al. 2016)
• NerveNet (Wang et al. 2018) (graph-structured policy, rather than model)
29
![Page 30: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/30.jpg)
Graph Network (GN)
30
Battaglia et al., 2018 Graph Network (GN)
Graph-to-graph, modular block design
Battaglia et al., 2018g g*
g
{ni}
{ej}
g*
{n*i}
{e*j}
Edge update
Node update
Global update
Graph-to-graph, modular block design
Edgeupdate
Nodeupdate
Globalupdate
![Page 31: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/31.jpg)
Forward model: supervised, 1-step training w/ random control inputs
31
Forward model: supervised, 1-step training w/ random control inputs
Chained 100-step predictions
Input graph (t) Next graph (t+1)
Sanchez-Gonzalez et al., 2018, ICML
Sanchez-Gonzalez et al., 2018, ICML
Chained
100-step predictions
![Page 32: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/32.jpg)
Results: Graph Net (GN) vs MLP forward models
32Sanchez-Gonzalez et al., 2018, ICML
More repeated structure:Better performance over MLP
Better test generalization, within and outside of the
training distribution
![Page 33: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/33.jpg)
GN forward model: Multiple systems & zero-shot generalization
Single model trained:
• Pendulum, Cartpole, Acrobot, Swimmer6 & Cheetah
Zero-shot generalization: Swimmer
• # training links: {3,4,5,6, -,8,9, -, -, ...}
• # testing links: {-, -, -, -, 7, -, -, 10-14}
33Sanchez-Gonzalez et al., 2018, ICML
![Page 34: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/34.jpg)
GN forward model: Real JACO data
34Sanchez-Gonzalez et al., 2018, ICML
GN forward model: Real JACO data
(Real JACO trajectories, rendered using Mujoco)
Recurrent graph network
Sanchez-Gonzalez et al., 2018, ICML
Recurrent graph network
(Real JACO trajectories, rendered using Mujoco)
![Page 35: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/35.jpg)
System identification: GN-based inference, under diagnostic control inputs
35
System identification: GN-based inference, under diagnostic control inputs
Unobserved system parameters (e.g. mass, length) are implicitly inferred
Sanchez-Gonzalez et al., 2018, ICML
Unobserved system parameters (e.g. mass, length) are implicitly inferred
Sanchez-Gonzalez et al., 2018, ICML
![Page 36: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/36.jpg)
Using learned models for control
36
![Page 37: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/37.jpg)
Control: Model-based planningTrajectory optimization: the GN-based forward model is differentiable, so we can backpropagate through it, and find a sequence of actions that maximize reward
37Sanchez-Gonzalez et al., 2018, ICML
![Page 38: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/38.jpg)
Control: Multiple systems via a single model
38Sanchez-Gonzalez et al., 2018, ICML
![Page 39: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/39.jpg)
Control: Zero-shot control
39Sanchez-Gonzalez et al., 2018, ICML
![Page 40: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/40.jpg)
Control: Multiple reward functions
40Sanchez-Gonzalez et al., 2018, ICML
![Page 41: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/41.jpg)
Learning to use mental simulation
41
![Page 42: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/42.jpg)
Learning to use mental simulation"Imagination-based metacontroller"
42
"Spaceship task":• Navigate to your home planet by choosing a force vector• Challenging because the planets exert gravity
The agent learns 3 components: 1. Action policy (via stochastic value gradients (Heess et al.
2015)) 2. GN-based forward model (via supervised 1-step training) 3. Internal strategy for using imagination to test potential
actions before selecting one to execute (via REINFORCE)
Hamrick et al., 2017, ICLR
![Page 43: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/43.jpg)
Learning to use mental simulation"Imagination-based planner"
43Pascanu et al., 2017, arXiv
Learning to use mental simulation "Imagination-based planner"
Pascanu et al., 2017, arXiv
• Red: real actions • Blue: 1 step of imagination • Green: 2+ steps of imagination
![Page 44: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/44.jpg)
Graph-structured model-free policies
44
![Page 45: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/45.jpg)
Graph-structured model-free policies forphysical construction in humans and AI
45Hamrick et al., 2018, Proc Cog Sci
Jess Hamrick, Kelsey Allen, Victor Bapst, Tina Zhu, Kevin McKee,
Josh Tenenbaum, Peter BattagliaProc Cog Sci, 2018
The "glue task" Goal: Glue blocks together to make the towerstable, using the minimum amount of glue.
![Page 46: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/46.jpg)
46
Structured model-free policies for physicalconstruction in humans and AI
Hamrick et al., 2018, Proc Cog Sci
![Page 47: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/47.jpg)
47
GM<latexit sha1_base64="vjTCpRgsPEJfhljVzwQb7AFhV5c=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0MIiYKGNENF8QHKEvc1csmRv79jdE8KRn2BjoYitv8jOf+MmuUITHww83pthZl6QCK6N6347hZXVtfWN4mZpa3tnd6+8f9DUcaoYNlgsYtUOqEbBJTYMNwLbiUIaBQJbweh66reeUGkey0czTtCP6EDykDNqrPRw07vrlStu1Z2BLBMvJxXIUe+Vv7r9mKURSsME1brjuYnxM6oMZwInpW6qMaFsRAfYsVTSCLWfzU6dkBOr9EkYK1vSkJn6eyKjkdbjKLCdETVDvehNxf+8TmrCSz/jMkkNSjZfFKaCmJhM/yZ9rpAZMbaEMsXtrYQNqaLM2HRKNgRv8eVl0jyrem7Vuz+v1K7yOIpwBMdwCh5cQA1uoQ4NYDCAZ3iFN0c4L8678zFvLTj5zCH8gfP5A+mdjYU=</latexit><latexit sha1_base64="vjTCpRgsPEJfhljVzwQb7AFhV5c=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0MIiYKGNENF8QHKEvc1csmRv79jdE8KRn2BjoYitv8jOf+MmuUITHww83pthZl6QCK6N6347hZXVtfWN4mZpa3tnd6+8f9DUcaoYNlgsYtUOqEbBJTYMNwLbiUIaBQJbweh66reeUGkey0czTtCP6EDykDNqrPRw07vrlStu1Z2BLBMvJxXIUe+Vv7r9mKURSsME1brjuYnxM6oMZwInpW6qMaFsRAfYsVTSCLWfzU6dkBOr9EkYK1vSkJn6eyKjkdbjKLCdETVDvehNxf+8TmrCSz/jMkkNSjZfFKaCmJhM/yZ9rpAZMbaEMsXtrYQNqaLM2HRKNgRv8eVl0jyrem7Vuz+v1K7yOIpwBMdwCh5cQA1uoQ4NYDCAZ3iFN0c4L8678zFvLTj5zCH8gfP5A+mdjYU=</latexit><latexit sha1_base64="vjTCpRgsPEJfhljVzwQb7AFhV5c=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0MIiYKGNENF8QHKEvc1csmRv79jdE8KRn2BjoYitv8jOf+MmuUITHww83pthZl6QCK6N6347hZXVtfWN4mZpa3tnd6+8f9DUcaoYNlgsYtUOqEbBJTYMNwLbiUIaBQJbweh66reeUGkey0czTtCP6EDykDNqrPRw07vrlStu1Z2BLBMvJxXIUe+Vv7r9mKURSsME1brjuYnxM6oMZwInpW6qMaFsRAfYsVTSCLWfzU6dkBOr9EkYK1vSkJn6eyKjkdbjKLCdETVDvehNxf+8TmrCSz/jMkkNSjZfFKaCmJhM/yZ9rpAZMbaEMsXtrYQNqaLM2HRKNgRv8eVl0jyrem7Vuz+v1K7yOIpwBMdwCh5cQA1uoQ4NYDCAZ3iFN0c4L8678zFvLTj5zCH8gfP5A+mdjYU=</latexit><latexit sha1_base64="vjTCpRgsPEJfhljVzwQb7AFhV5c=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0MIiYKGNENF8QHKEvc1csmRv79jdE8KRn2BjoYitv8jOf+MmuUITHww83pthZl6QCK6N6347hZXVtfWN4mZpa3tnd6+8f9DUcaoYNlgsYtUOqEbBJTYMNwLbiUIaBQJbweh66reeUGkey0czTtCP6EDykDNqrPRw07vrlStu1Z2BLBMvJxXIUe+Vv7r9mKURSsME1brjuYnxM6oMZwInpW6qMaFsRAfYsVTSCLWfzU6dkBOr9EkYK1vSkJn6eyKjkdbjKLCdETVDvehNxf+8TmrCSz/jMkkNSjZfFKaCmJhM/yZ9rpAZMbaEMsXtrYQNqaLM2HRKNgRv8eVl0jyrem7Vuz+v1K7yOIpwBMdwCh5cQA1uoQ4NYDCAZ3iFN0c4L8678zFvLTj5zCH8gfP5A+mdjYU=</latexit>
GN1<latexit sha1_base64="oAmr7/S238q10w2wEvXkfEGmAr8=">AAAB9HicbVDLSgMxFL1TX7W+qi7dBIvgqsyIUBcuCi50JRXsA9qhZNJMG5pkxiRTKEO/w40LRdz6Me78GzPtLLT1QOBwzr3ckxPEnGnjut9OYW19Y3OruF3a2d3bPygfHrV0lChCmyTikeoEWFPOJG0aZjjtxIpiEXDaDsY3md+eUKVZJB/NNKa+wEPJQkawsZLfE9iMlEhv72d9r1+uuFV3DrRKvJxUIEejX/7qDSKSCCoN4VjrrufGxk+xMoxwOiv1Ek1jTMZ4SLuWSiyo9tN56Bk6s8oAhZGyTxo0V39vpFhoPRWBncxC6mUvE//zuokJr/yUyTgxVJLFoTDhyEQoawANmKLE8KklmChmsyIywgoTY3sq2RK85S+vktZF1XOr3sNlpX6d11GEEziFc/CgBnW4gwY0gcATPMMrvDkT58V5dz4WowUn3zmGP3A+fwCg3ZH4</latexit><latexit sha1_base64="oAmr7/S238q10w2wEvXkfEGmAr8=">AAAB9HicbVDLSgMxFL1TX7W+qi7dBIvgqsyIUBcuCi50JRXsA9qhZNJMG5pkxiRTKEO/w40LRdz6Me78GzPtLLT1QOBwzr3ckxPEnGnjut9OYW19Y3OruF3a2d3bPygfHrV0lChCmyTikeoEWFPOJG0aZjjtxIpiEXDaDsY3md+eUKVZJB/NNKa+wEPJQkawsZLfE9iMlEhv72d9r1+uuFV3DrRKvJxUIEejX/7qDSKSCCoN4VjrrufGxk+xMoxwOiv1Ek1jTMZ4SLuWSiyo9tN56Bk6s8oAhZGyTxo0V39vpFhoPRWBncxC6mUvE//zuokJr/yUyTgxVJLFoTDhyEQoawANmKLE8KklmChmsyIywgoTY3sq2RK85S+vktZF1XOr3sNlpX6d11GEEziFc/CgBnW4gwY0gcATPMMrvDkT58V5dz4WowUn3zmGP3A+fwCg3ZH4</latexit><latexit sha1_base64="oAmr7/S238q10w2wEvXkfEGmAr8=">AAAB9HicbVDLSgMxFL1TX7W+qi7dBIvgqsyIUBcuCi50JRXsA9qhZNJMG5pkxiRTKEO/w40LRdz6Me78GzPtLLT1QOBwzr3ckxPEnGnjut9OYW19Y3OruF3a2d3bPygfHrV0lChCmyTikeoEWFPOJG0aZjjtxIpiEXDaDsY3md+eUKVZJB/NNKa+wEPJQkawsZLfE9iMlEhv72d9r1+uuFV3DrRKvJxUIEejX/7qDSKSCCoN4VjrrufGxk+xMoxwOiv1Ek1jTMZ4SLuWSiyo9tN56Bk6s8oAhZGyTxo0V39vpFhoPRWBncxC6mUvE//zuokJr/yUyTgxVJLFoTDhyEQoawANmKLE8KklmChmsyIywgoTY3sq2RK85S+vktZF1XOr3sNlpX6d11GEEziFc/CgBnW4gwY0gcATPMMrvDkT58V5dz4WowUn3zmGP3A+fwCg3ZH4</latexit><latexit sha1_base64="oAmr7/S238q10w2wEvXkfEGmAr8=">AAAB9HicbVDLSgMxFL1TX7W+qi7dBIvgqsyIUBcuCi50JRXsA9qhZNJMG5pkxiRTKEO/w40LRdz6Me78GzPtLLT1QOBwzr3ckxPEnGnjut9OYW19Y3OruF3a2d3bPygfHrV0lChCmyTikeoEWFPOJG0aZjjtxIpiEXDaDsY3md+eUKVZJB/NNKa+wEPJQkawsZLfE9iMlEhv72d9r1+uuFV3DrRKvJxUIEejX/7qDSKSCCoN4VjrrufGxk+xMoxwOiv1Ek1jTMZ4SLuWSiyo9tN56Bk6s8oAhZGyTxo0V39vpFhoPRWBncxC6mUvE//zuokJr/yUyTgxVJLFoTDhyEQoawANmKLE8KklmChmsyIywgoTY3sq2RK85S+vktZF1XOr3sNlpX6d11GEEziFc/CgBnW4gwY0gcATPMMrvDkT58V5dz4WowUn3zmGP3A+fwCg3ZH4</latexit>
GN2<latexit sha1_base64="pet508CCa1uIZM8cv8xqNGylB9w=">AAAB9HicbVDLSgMxFL1TX7W+qi7dBIvgqswUQRcuCi50JRXsA9qhZNJMG5pkxiRTKEO/w40LRdz6Me78GzPtLLT1QOBwzr3ckxPEnGnjut9OYW19Y3OruF3a2d3bPygfHrV0lChCmyTikeoEWFPOJG0aZjjtxIpiEXDaDsY3md+eUKVZJB/NNKa+wEPJQkawsZLfE9iMlEhv72f9Wr9ccavuHGiVeDmpQI5Gv/zVG0QkEVQawrHWXc+NjZ9iZRjhdFbqJZrGmIzxkHYtlVhQ7afz0DN0ZpUBCiNlnzRorv7eSLHQeioCO5mF1MteJv7ndRMTXvkpk3FiqCSLQ2HCkYlQ1gAaMEWJ4VNLMFHMZkVkhBUmxvZUsiV4y19eJa1a1XOr3sNFpX6d11GEEziFc/DgEupwBw1oAoEneIZXeHMmzovz7nwsRgtOvnMMf+B8/gCiYZH5</latexit><latexit sha1_base64="pet508CCa1uIZM8cv8xqNGylB9w=">AAAB9HicbVDLSgMxFL1TX7W+qi7dBIvgqswUQRcuCi50JRXsA9qhZNJMG5pkxiRTKEO/w40LRdz6Me78GzPtLLT1QOBwzr3ckxPEnGnjut9OYW19Y3OruF3a2d3bPygfHrV0lChCmyTikeoEWFPOJG0aZjjtxIpiEXDaDsY3md+eUKVZJB/NNKa+wEPJQkawsZLfE9iMlEhv72f9Wr9ccavuHGiVeDmpQI5Gv/zVG0QkEVQawrHWXc+NjZ9iZRjhdFbqJZrGmIzxkHYtlVhQ7afz0DN0ZpUBCiNlnzRorv7eSLHQeioCO5mF1MteJv7ndRMTXvkpk3FiqCSLQ2HCkYlQ1gAaMEWJ4VNLMFHMZkVkhBUmxvZUsiV4y19eJa1a1XOr3sNFpX6d11GEEziFc/DgEupwBw1oAoEneIZXeHMmzovz7nwsRgtOvnMMf+B8/gCiYZH5</latexit><latexit sha1_base64="pet508CCa1uIZM8cv8xqNGylB9w=">AAAB9HicbVDLSgMxFL1TX7W+qi7dBIvgqswUQRcuCi50JRXsA9qhZNJMG5pkxiRTKEO/w40LRdz6Me78GzPtLLT1QOBwzr3ckxPEnGnjut9OYW19Y3OruF3a2d3bPygfHrV0lChCmyTikeoEWFPOJG0aZjjtxIpiEXDaDsY3md+eUKVZJB/NNKa+wEPJQkawsZLfE9iMlEhv72f9Wr9ccavuHGiVeDmpQI5Gv/zVG0QkEVQawrHWXc+NjZ9iZRjhdFbqJZrGmIzxkHYtlVhQ7afz0DN0ZpUBCiNlnzRorv7eSLHQeioCO5mF1MteJv7ndRMTXvkpk3FiqCSLQ2HCkYlQ1gAaMEWJ4VNLMFHMZkVkhBUmxvZUsiV4y19eJa1a1XOr3sNFpX6d11GEEziFc/DgEupwBw1oAoEneIZXeHMmzovz7nwsRgtOvnMMf+B8/gCiYZH5</latexit><latexit sha1_base64="pet508CCa1uIZM8cv8xqNGylB9w=">AAAB9HicbVDLSgMxFL1TX7W+qi7dBIvgqswUQRcuCi50JRXsA9qhZNJMG5pkxiRTKEO/w40LRdz6Me78GzPtLLT1QOBwzr3ckxPEnGnjut9OYW19Y3OruF3a2d3bPygfHrV0lChCmyTikeoEWFPOJG0aZjjtxIpiEXDaDsY3md+eUKVZJB/NNKa+wEPJQkawsZLfE9iMlEhv72f9Wr9ccavuHGiVeDmpQI5Gv/zVG0QkEVQawrHWXc+NjZ9iZRjhdFbqJZrGmIzxkHYtlVhQ7afz0DN0ZpUBCiNlnzRorv7eSLHQeioCO5mF1MteJv7ndRMTXvkpk3FiqCSLQ2HCkYlQ1gAaMEWJ4VNLMFHMZkVkhBUmxvZUsiV4y19eJa1a1XOr3sNFpX6d11GEEziFc/DgEupwBw1oAoEneIZXeHMmzovz7nwsRgtOvnMMf+B8/gCiYZH5</latexit>
GNM<latexit sha1_base64="1uUQuLnXmq2FrQq5fvsHBbzm7v8=">AAAB9HicbVDLSgMxFL1TX7W+qi7dBIvgqsyIoAsXBRe6USrYB7RDyaSZNjTJjEmmUIZ+hxsXirj1Y9z5N2baWWjrgcDhnHu5JyeIOdPGdb+dwsrq2vpGcbO0tb2zu1feP2jqKFGENkjEI9UOsKacSdowzHDajhXFIuC0FYyuM781pkqzSD6aSUx9gQeShYxgYyW/K7AZKpHe3E97d71yxa26M6Bl4uWkAjnqvfJXtx+RRFBpCMdadzw3Nn6KlWGE02mpm2gaYzLCA9qxVGJBtZ/OQk/RiVX6KIyUfdKgmfp7I8VC64kI7GQWUi96mfif10lMeOmnTMaJoZLMD4UJRyZCWQOozxQlhk8swUQxmxWRIVaYGNtTyZbgLX55mTTPqp5b9R7OK7WrvI4iHMExnIIHF1CDW6hDAwg8wTO8wpszdl6cd+djPlpw8p1D+APn8wfLTZIU</latexit><latexit sha1_base64="1uUQuLnXmq2FrQq5fvsHBbzm7v8=">AAAB9HicbVDLSgMxFL1TX7W+qi7dBIvgqsyIoAsXBRe6USrYB7RDyaSZNjTJjEmmUIZ+hxsXirj1Y9z5N2baWWjrgcDhnHu5JyeIOdPGdb+dwsrq2vpGcbO0tb2zu1feP2jqKFGENkjEI9UOsKacSdowzHDajhXFIuC0FYyuM781pkqzSD6aSUx9gQeShYxgYyW/K7AZKpHe3E97d71yxa26M6Bl4uWkAjnqvfJXtx+RRFBpCMdadzw3Nn6KlWGE02mpm2gaYzLCA9qxVGJBtZ/OQk/RiVX6KIyUfdKgmfp7I8VC64kI7GQWUi96mfif10lMeOmnTMaJoZLMD4UJRyZCWQOozxQlhk8swUQxmxWRIVaYGNtTyZbgLX55mTTPqp5b9R7OK7WrvI4iHMExnIIHF1CDW6hDAwg8wTO8wpszdl6cd+djPlpw8p1D+APn8wfLTZIU</latexit><latexit sha1_base64="1uUQuLnXmq2FrQq5fvsHBbzm7v8=">AAAB9HicbVDLSgMxFL1TX7W+qi7dBIvgqsyIoAsXBRe6USrYB7RDyaSZNjTJjEmmUIZ+hxsXirj1Y9z5N2baWWjrgcDhnHu5JyeIOdPGdb+dwsrq2vpGcbO0tb2zu1feP2jqKFGENkjEI9UOsKacSdowzHDajhXFIuC0FYyuM781pkqzSD6aSUx9gQeShYxgYyW/K7AZKpHe3E97d71yxa26M6Bl4uWkAjnqvfJXtx+RRFBpCMdadzw3Nn6KlWGE02mpm2gaYzLCA9qxVGJBtZ/OQk/RiVX6KIyUfdKgmfp7I8VC64kI7GQWUi96mfif10lMeOmnTMaJoZLMD4UJRyZCWQOozxQlhk8swUQxmxWRIVaYGNtTyZbgLX55mTTPqp5b9R7OK7WrvI4iHMExnIIHF1CDW6hDAwg8wTO8wpszdl6cd+djPlpw8p1D+APn8wfLTZIU</latexit><latexit sha1_base64="1uUQuLnXmq2FrQq5fvsHBbzm7v8=">AAAB9HicbVDLSgMxFL1TX7W+qi7dBIvgqsyIoAsXBRe6USrYB7RDyaSZNjTJjEmmUIZ+hxsXirj1Y9z5N2baWWjrgcDhnHu5JyeIOdPGdb+dwsrq2vpGcbO0tb2zu1feP2jqKFGENkjEI9UOsKacSdowzHDajhXFIuC0FYyuM781pkqzSD6aSUx9gQeShYxgYyW/K7AZKpHe3E97d71yxa26M6Bl4uWkAjnqvfJXtx+RRFBpCMdadzw3Nn6KlWGE02mpm2gaYzLCA9qxVGJBtZ/OQk/RiVX6KIyUfdKgmfp7I8VC64kI7GQWUi96mfif10lMeOmnTMaJoZLMD4UJRyZCWQOozxQlhk8swUQxmxWRIVaYGNtTyZbgLX55mTTPqp5b9R7OK7WrvI4iHMExnIIHF1CDW6hDAwg8wTO8wpszdl6cd+djPlpw8p1D+APn8wfLTZIU</latexit>
. . .<latexit sha1_base64="Gj7yv98SlyD93Ghofp+NnyXvd2c=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUA8eCl48VjBtoQ1ls920SzebsPsilNDf4MWDIl79Qd78N27bHLR1YGGYecO+N2EqhUHX/XZKG5tb2zvl3cre/sHhUfX4pG2STDPus0QmuhtSw6VQ3EeBkndTzWkcSt4JJ3dzv/PEtRGJesRpyoOYjpSIBKNoJb8/TNAMqjW37i5A1olXkBoUaA2qXzbHspgrZJIa0/PcFIOcahRM8lmlnxmeUjahI96zVNGYmyBfLDsjF1YZkijR9ikkC/V3IqexMdM4tJMxxbFZ9ebif14vw+gmyIVKM+SKLT+KMkkwIfPLyVBozlBOLaFMC7srYWOqKUPbT8WW4K2evE7aV3XPrXsP17XmbVFHGc7gHC7BgwY04R5a4AMDAc/wCm+Ocl6cd+djOVpyiswp/IHz+QPvmo68</latexit><latexit sha1_base64="Gj7yv98SlyD93Ghofp+NnyXvd2c=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUA8eCl48VjBtoQ1ls920SzebsPsilNDf4MWDIl79Qd78N27bHLR1YGGYecO+N2EqhUHX/XZKG5tb2zvl3cre/sHhUfX4pG2STDPus0QmuhtSw6VQ3EeBkndTzWkcSt4JJ3dzv/PEtRGJesRpyoOYjpSIBKNoJb8/TNAMqjW37i5A1olXkBoUaA2qXzbHspgrZJIa0/PcFIOcahRM8lmlnxmeUjahI96zVNGYmyBfLDsjF1YZkijR9ikkC/V3IqexMdM4tJMxxbFZ9ebif14vw+gmyIVKM+SKLT+KMkkwIfPLyVBozlBOLaFMC7srYWOqKUPbT8WW4K2evE7aV3XPrXsP17XmbVFHGc7gHC7BgwY04R5a4AMDAc/wCm+Ocl6cd+djOVpyiswp/IHz+QPvmo68</latexit><latexit sha1_base64="Gj7yv98SlyD93Ghofp+NnyXvd2c=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUA8eCl48VjBtoQ1ls920SzebsPsilNDf4MWDIl79Qd78N27bHLR1YGGYecO+N2EqhUHX/XZKG5tb2zvl3cre/sHhUfX4pG2STDPus0QmuhtSw6VQ3EeBkndTzWkcSt4JJ3dzv/PEtRGJesRpyoOYjpSIBKNoJb8/TNAMqjW37i5A1olXkBoUaA2qXzbHspgrZJIa0/PcFIOcahRM8lmlnxmeUjahI96zVNGYmyBfLDsjF1YZkijR9ikkC/V3IqexMdM4tJMxxbFZ9ebif14vw+gmyIVKM+SKLT+KMkkwIfPLyVBozlBOLaFMC7srYWOqKUPbT8WW4K2evE7aV3XPrXsP17XmbVFHGc7gHC7BgwY04R5a4AMDAc/wCm+Ocl6cd+djOVpyiswp/IHz+QPvmo68</latexit><latexit sha1_base64="Gj7yv98SlyD93Ghofp+NnyXvd2c=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUA8eCl48VjBtoQ1ls920SzebsPsilNDf4MWDIl79Qd78N27bHLR1YGGYecO+N2EqhUHX/XZKG5tb2zvl3cre/sHhUfX4pG2STDPus0QmuhtSw6VQ3EeBkndTzWkcSt4JJ3dzv/PEtRGJesRpyoOYjpSIBKNoJb8/TNAMqjW37i5A1olXkBoUaA2qXzbHspgrZJIa0/PcFIOcahRM8lmlnxmeUjahI96zVNGYmyBfLDsjF1YZkijR9ikkC/V3IqexMdM4tJMxxbFZ9ebif14vw+gmyIVKM+SKLT+KMkkwIfPLyVBozlBOLaFMC7srYWOqKUPbT8WW4K2evE7aV3XPrXsP17XmbVFHGc7gHC7BgwY04R5a4AMDAc/wCm+Ocl6cd+djOVpyiswp/IHz+QPvmo68</latexit>G1
<latexit sha1_base64="YNShseMoKm2HdChKvcjMRmoBu5o=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EiIVFwELLiOYDkiPsbfaSJXt7x+6cEI78BBsLRWz9RXb+GzfJFZr4YODx3gwz84JECoOu++0U1tY3NreK26Wd3b39g/LhUcvEqWa8yWIZ605ADZdC8SYKlLyTaE6jQPJ2ML6Z+e0nro2I1SNOEu5HdKhEKBhFKz3c9r1+ueJW3TnIKvFyUoEcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7lioaceNn81On5MwqAxLG2pZCMld/T2Q0MmYSBbYzojgyy95M/M/rphhe+ZlQSYpcscWiMJUEYzL7mwyE5gzlxBLKtLC3EjaimjK06ZRsCN7yy6ukdVH13Kp3f1mpX+dxFOEETuEcPKhBHe6gAU1gMIRneIU3RzovzrvzsWgtOPnMMfyB8/kDvy2NaQ==</latexit><latexit sha1_base64="YNShseMoKm2HdChKvcjMRmoBu5o=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EiIVFwELLiOYDkiPsbfaSJXt7x+6cEI78BBsLRWz9RXb+GzfJFZr4YODx3gwz84JECoOu++0U1tY3NreK26Wd3b39g/LhUcvEqWa8yWIZ605ADZdC8SYKlLyTaE6jQPJ2ML6Z+e0nro2I1SNOEu5HdKhEKBhFKz3c9r1+ueJW3TnIKvFyUoEcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7lioaceNn81On5MwqAxLG2pZCMld/T2Q0MmYSBbYzojgyy95M/M/rphhe+ZlQSYpcscWiMJUEYzL7mwyE5gzlxBLKtLC3EjaimjK06ZRsCN7yy6ukdVH13Kp3f1mpX+dxFOEETuEcPKhBHe6gAU1gMIRneIU3RzovzrvzsWgtOPnMMfyB8/kDvy2NaQ==</latexit><latexit sha1_base64="YNShseMoKm2HdChKvcjMRmoBu5o=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EiIVFwELLiOYDkiPsbfaSJXt7x+6cEI78BBsLRWz9RXb+GzfJFZr4YODx3gwz84JECoOu++0U1tY3NreK26Wd3b39g/LhUcvEqWa8yWIZ605ADZdC8SYKlLyTaE6jQPJ2ML6Z+e0nro2I1SNOEu5HdKhEKBhFKz3c9r1+ueJW3TnIKvFyUoEcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7lioaceNn81On5MwqAxLG2pZCMld/T2Q0MmYSBbYzojgyy95M/M/rphhe+ZlQSYpcscWiMJUEYzL7mwyE5gzlxBLKtLC3EjaimjK06ZRsCN7yy6ukdVH13Kp3f1mpX+dxFOEETuEcPKhBHe6gAU1gMIRneIU3RzovzrvzsWgtOPnMMfyB8/kDvy2NaQ==</latexit><latexit sha1_base64="YNShseMoKm2HdChKvcjMRmoBu5o=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EiIVFwELLiOYDkiPsbfaSJXt7x+6cEI78BBsLRWz9RXb+GzfJFZr4YODx3gwz84JECoOu++0U1tY3NreK26Wd3b39g/LhUcvEqWa8yWIZ605ADZdC8SYKlLyTaE6jQPJ2ML6Z+e0nro2I1SNOEu5HdKhEKBhFKz3c9r1+ueJW3TnIKvFyUoEcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7lioaceNn81On5MwqAxLG2pZCMld/T2Q0MmYSBbYzojgyy95M/M/rphhe+ZlQSYpcscWiMJUEYzL7mwyE5gzlxBLKtLC3EjaimjK06ZRsCN7yy6ukdVH13Kp3f1mpX+dxFOEETuEcPKhBHe6gAU1gMIRneIU3RzovzrvzsWgtOPnMMfyB8/kDvy2NaQ==</latexit>
G0<latexit sha1_base64="vj48jMMQe2f55rU6zb6RZp+K9y4=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EiIVFwELLiOYDkiPsbfaSJXt7x+6cEI78BBsLRWz9RXb+GzfJFZr4YODx3gwz84JECoOu++0U1tY3NreK26Wd3b39g/LhUcvEqWa8yWIZ605ADZdC8SYKlLyTaE6jQPJ2ML6Z+e0nro2I1SNOEu5HdKhEKBhFKz3c9t1+ueJW3TnIKvFyUoEcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7lioaceNn81On5MwqAxLG2pZCMld/T2Q0MmYSBbYzojgyy95M/M/rphhe+ZlQSYpcscWiMJUEYzL7mwyE5gzlxBLKtLC3EjaimjK06ZRsCN7yy6ukdVH13Kp3f1mpX+dxFOEETuEcPKhBHe6gAU1gMIRneIU3RzovzrvzsWgtOPnMMfyB8/kDvamNaA==</latexit><latexit sha1_base64="vj48jMMQe2f55rU6zb6RZp+K9y4=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EiIVFwELLiOYDkiPsbfaSJXt7x+6cEI78BBsLRWz9RXb+GzfJFZr4YODx3gwz84JECoOu++0U1tY3NreK26Wd3b39g/LhUcvEqWa8yWIZ605ADZdC8SYKlLyTaE6jQPJ2ML6Z+e0nro2I1SNOEu5HdKhEKBhFKz3c9t1+ueJW3TnIKvFyUoEcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7lioaceNn81On5MwqAxLG2pZCMld/T2Q0MmYSBbYzojgyy95M/M/rphhe+ZlQSYpcscWiMJUEYzL7mwyE5gzlxBLKtLC3EjaimjK06ZRsCN7yy6ukdVH13Kp3f1mpX+dxFOEETuEcPKhBHe6gAU1gMIRneIU3RzovzrvzsWgtOPnMMfyB8/kDvamNaA==</latexit><latexit sha1_base64="vj48jMMQe2f55rU6zb6RZp+K9y4=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EiIVFwELLiOYDkiPsbfaSJXt7x+6cEI78BBsLRWz9RXb+GzfJFZr4YODx3gwz84JECoOu++0U1tY3NreK26Wd3b39g/LhUcvEqWa8yWIZ605ADZdC8SYKlLyTaE6jQPJ2ML6Z+e0nro2I1SNOEu5HdKhEKBhFKz3c9t1+ueJW3TnIKvFyUoEcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7lioaceNn81On5MwqAxLG2pZCMld/T2Q0MmYSBbYzojgyy95M/M/rphhe+ZlQSYpcscWiMJUEYzL7mwyE5gzlxBLKtLC3EjaimjK06ZRsCN7yy6ukdVH13Kp3f1mpX+dxFOEETuEcPKhBHe6gAU1gMIRneIU3RzovzrvzsWgtOPnMMfyB8/kDvamNaA==</latexit><latexit sha1_base64="vj48jMMQe2f55rU6zb6RZp+K9y4=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EiIVFwELLiOYDkiPsbfaSJXt7x+6cEI78BBsLRWz9RXb+GzfJFZr4YODx3gwz84JECoOu++0U1tY3NreK26Wd3b39g/LhUcvEqWa8yWIZ605ADZdC8SYKlLyTaE6jQPJ2ML6Z+e0nro2I1SNOEu5HdKhEKBhFKz3c9t1+ueJW3TnIKvFyUoEcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7lioaceNn81On5MwqAxLG2pZCMld/T2Q0MmYSBbYzojgyy95M/M/rphhe+ZlQSYpcscWiMJUEYzL7mwyE5gzlxBLKtLC3EjaimjK06ZRsCN7yy6ukdVH13Kp3f1mpX+dxFOEETuEcPKhBHe6gAU1gMIRneIU3RzovzrvzsWgtOPnMMfyB8/kDvamNaA==</latexit>
GM<latexit sha1_base64="vjTCpRgsPEJfhljVzwQb7AFhV5c=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0MIiYKGNENF8QHKEvc1csmRv79jdE8KRn2BjoYitv8jOf+MmuUITHww83pthZl6QCK6N6347hZXVtfWN4mZpa3tnd6+8f9DUcaoYNlgsYtUOqEbBJTYMNwLbiUIaBQJbweh66reeUGkey0czTtCP6EDykDNqrPRw07vrlStu1Z2BLBMvJxXIUe+Vv7r9mKURSsME1brjuYnxM6oMZwInpW6qMaFsRAfYsVTSCLWfzU6dkBOr9EkYK1vSkJn6eyKjkdbjKLCdETVDvehNxf+8TmrCSz/jMkkNSjZfFKaCmJhM/yZ9rpAZMbaEMsXtrYQNqaLM2HRKNgRv8eVl0jyrem7Vuz+v1K7yOIpwBMdwCh5cQA1uoQ4NYDCAZ3iFN0c4L8678zFvLTj5zCH8gfP5A+mdjYU=</latexit><latexit sha1_base64="vjTCpRgsPEJfhljVzwQb7AFhV5c=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0MIiYKGNENF8QHKEvc1csmRv79jdE8KRn2BjoYitv8jOf+MmuUITHww83pthZl6QCK6N6347hZXVtfWN4mZpa3tnd6+8f9DUcaoYNlgsYtUOqEbBJTYMNwLbiUIaBQJbweh66reeUGkey0czTtCP6EDykDNqrPRw07vrlStu1Z2BLBMvJxXIUe+Vv7r9mKURSsME1brjuYnxM6oMZwInpW6qMaFsRAfYsVTSCLWfzU6dkBOr9EkYK1vSkJn6eyKjkdbjKLCdETVDvehNxf+8TmrCSz/jMkkNSjZfFKaCmJhM/yZ9rpAZMbaEMsXtrYQNqaLM2HRKNgRv8eVl0jyrem7Vuz+v1K7yOIpwBMdwCh5cQA1uoQ4NYDCAZ3iFN0c4L8678zFvLTj5zCH8gfP5A+mdjYU=</latexit><latexit sha1_base64="vjTCpRgsPEJfhljVzwQb7AFhV5c=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0MIiYKGNENF8QHKEvc1csmRv79jdE8KRn2BjoYitv8jOf+MmuUITHww83pthZl6QCK6N6347hZXVtfWN4mZpa3tnd6+8f9DUcaoYNlgsYtUOqEbBJTYMNwLbiUIaBQJbweh66reeUGkey0czTtCP6EDykDNqrPRw07vrlStu1Z2BLBMvJxXIUe+Vv7r9mKURSsME1brjuYnxM6oMZwInpW6qMaFsRAfYsVTSCLWfzU6dkBOr9EkYK1vSkJn6eyKjkdbjKLCdETVDvehNxf+8TmrCSz/jMkkNSjZfFKaCmJhM/yZ9rpAZMbaEMsXtrYQNqaLM2HRKNgRv8eVl0jyrem7Vuz+v1K7yOIpwBMdwCh5cQA1uoQ4NYDCAZ3iFN0c4L8678zFvLTj5zCH8gfP5A+mdjYU=</latexit><latexit sha1_base64="vjTCpRgsPEJfhljVzwQb7AFhV5c=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0MIiYKGNENF8QHKEvc1csmRv79jdE8KRn2BjoYitv8jOf+MmuUITHww83pthZl6QCK6N6347hZXVtfWN4mZpa3tnd6+8f9DUcaoYNlgsYtUOqEbBJTYMNwLbiUIaBQJbweh66reeUGkey0czTtCP6EDykDNqrPRw07vrlStu1Z2BLBMvJxXIUe+Vv7r9mKURSsME1brjuYnxM6oMZwInpW6qMaFsRAfYsVTSCLWfzU6dkBOr9EkYK1vSkJn6eyKjkdbjKLCdETVDvehNxf+8TmrCSz/jMkkNSjZfFKaCmJhM/yZ9rpAZMbaEMsXtrYQNqaLM2HRKNgRv8eVl0jyrem7Vuz+v1K7yOIpwBMdwCh5cQA1uoQ4NYDCAZ3iFN0c4L8678zFvLTj5zCH8gfP5A+mdjYU=</latexit>
G0<latexit sha1_base64="vj48jMMQe2f55rU6zb6RZp+K9y4=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EiIVFwELLiOYDkiPsbfaSJXt7x+6cEI78BBsLRWz9RXb+GzfJFZr4YODx3gwz84JECoOu++0U1tY3NreK26Wd3b39g/LhUcvEqWa8yWIZ605ADZdC8SYKlLyTaE6jQPJ2ML6Z+e0nro2I1SNOEu5HdKhEKBhFKz3c9t1+ueJW3TnIKvFyUoEcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7lioaceNn81On5MwqAxLG2pZCMld/T2Q0MmYSBbYzojgyy95M/M/rphhe+ZlQSYpcscWiMJUEYzL7mwyE5gzlxBLKtLC3EjaimjK06ZRsCN7yy6ukdVH13Kp3f1mpX+dxFOEETuEcPKhBHe6gAU1gMIRneIU3RzovzrvzsWgtOPnMMfyB8/kDvamNaA==</latexit><latexit sha1_base64="vj48jMMQe2f55rU6zb6RZp+K9y4=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EiIVFwELLiOYDkiPsbfaSJXt7x+6cEI78BBsLRWz9RXb+GzfJFZr4YODx3gwz84JECoOu++0U1tY3NreK26Wd3b39g/LhUcvEqWa8yWIZ605ADZdC8SYKlLyTaE6jQPJ2ML6Z+e0nro2I1SNOEu5HdKhEKBhFKz3c9t1+ueJW3TnIKvFyUoEcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7lioaceNn81On5MwqAxLG2pZCMld/T2Q0MmYSBbYzojgyy95M/M/rphhe+ZlQSYpcscWiMJUEYzL7mwyE5gzlxBLKtLC3EjaimjK06ZRsCN7yy6ukdVH13Kp3f1mpX+dxFOEETuEcPKhBHe6gAU1gMIRneIU3RzovzrvzsWgtOPnMMfyB8/kDvamNaA==</latexit><latexit sha1_base64="vj48jMMQe2f55rU6zb6RZp+K9y4=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EiIVFwELLiOYDkiPsbfaSJXt7x+6cEI78BBsLRWz9RXb+GzfJFZr4YODx3gwz84JECoOu++0U1tY3NreK26Wd3b39g/LhUcvEqWa8yWIZ605ADZdC8SYKlLyTaE6jQPJ2ML6Z+e0nro2I1SNOEu5HdKhEKBhFKz3c9t1+ueJW3TnIKvFyUoEcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7lioaceNn81On5MwqAxLG2pZCMld/T2Q0MmYSBbYzojgyy95M/M/rphhe+ZlQSYpcscWiMJUEYzL7mwyE5gzlxBLKtLC3EjaimjK06ZRsCN7yy6ukdVH13Kp3f1mpX+dxFOEETuEcPKhBHe6gAU1gMIRneIU3RzovzrvzsWgtOPnMMfyB8/kDvamNaA==</latexit><latexit sha1_base64="vj48jMMQe2f55rU6zb6RZp+K9y4=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EiIVFwELLiOYDkiPsbfaSJXt7x+6cEI78BBsLRWz9RXb+GzfJFZr4YODx3gwz84JECoOu++0U1tY3NreK26Wd3b39g/LhUcvEqWa8yWIZ605ADZdC8SYKlLyTaE6jQPJ2ML6Z+e0nro2I1SNOEu5HdKhEKBhFKz3c9t1+ueJW3TnIKvFyUoEcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7lioaceNn81On5MwqAxLG2pZCMld/T2Q0MmYSBbYzojgyy95M/M/rphhe+ZlQSYpcscWiMJUEYzL7mwyE5gzlxBLKtLC3EjaimjK06ZRsCN7yy6ukdVH13Kp3f1mpX+dxFOEETuEcPKhBHe6gAU1gMIRneIU3RzovzrvzsWgtOPnMMfyB8/kDvamNaA==</latexit>
GNcore<latexit sha1_base64="sfcetjjriA53KVhP8LRkSGs9KNA=">AAAB+3icbVDLSsNAFJ34rPUV69LNYBFclUQEXbgouNCVVLAPaEOYTCft0HmEmYlYQn7FjQtF3Poj7vwbJ20W2npg4HDOvdwzJ0oY1cbzvp2V1bX1jc3KVnV7Z3dv3z2odbRMFSZtLJlUvQhpwqggbUMNI71EEcQjRrrR5Lrwu49EaSrFg5kmJOBoJGhMMTJWCt3agCMzVjy7ucvDDEtF8tCtew1vBrhM/JLUQYlW6H4NhhKnnAiDGdK673uJCTKkDMWM5NVBqkmC8ASNSN9SgTjRQTbLnsMTqwxhLJV9wsCZ+nsjQ1zrKY/sZJFUL3qF+J/XT018GWRUJKkhAs8PxSmDRsKiCDikimDDppYgrKjNCvEYKYSNratqS/AXv7xMOmcN32v49+f15lVZRwUcgWNwCnxwAZrgFrRAG2DwBJ7BK3hzcufFeXc+5qMrTrlzCP7A+fwBopiUyw==</latexit><latexit sha1_base64="sfcetjjriA53KVhP8LRkSGs9KNA=">AAAB+3icbVDLSsNAFJ34rPUV69LNYBFclUQEXbgouNCVVLAPaEOYTCft0HmEmYlYQn7FjQtF3Poj7vwbJ20W2npg4HDOvdwzJ0oY1cbzvp2V1bX1jc3KVnV7Z3dv3z2odbRMFSZtLJlUvQhpwqggbUMNI71EEcQjRrrR5Lrwu49EaSrFg5kmJOBoJGhMMTJWCt3agCMzVjy7ucvDDEtF8tCtew1vBrhM/JLUQYlW6H4NhhKnnAiDGdK673uJCTKkDMWM5NVBqkmC8ASNSN9SgTjRQTbLnsMTqwxhLJV9wsCZ+nsjQ1zrKY/sZJFUL3qF+J/XT018GWRUJKkhAs8PxSmDRsKiCDikimDDppYgrKjNCvEYKYSNratqS/AXv7xMOmcN32v49+f15lVZRwUcgWNwCnxwAZrgFrRAG2DwBJ7BK3hzcufFeXc+5qMrTrlzCP7A+fwBopiUyw==</latexit><latexit sha1_base64="sfcetjjriA53KVhP8LRkSGs9KNA=">AAAB+3icbVDLSsNAFJ34rPUV69LNYBFclUQEXbgouNCVVLAPaEOYTCft0HmEmYlYQn7FjQtF3Poj7vwbJ20W2npg4HDOvdwzJ0oY1cbzvp2V1bX1jc3KVnV7Z3dv3z2odbRMFSZtLJlUvQhpwqggbUMNI71EEcQjRrrR5Lrwu49EaSrFg5kmJOBoJGhMMTJWCt3agCMzVjy7ucvDDEtF8tCtew1vBrhM/JLUQYlW6H4NhhKnnAiDGdK673uJCTKkDMWM5NVBqkmC8ASNSN9SgTjRQTbLnsMTqwxhLJV9wsCZ+nsjQ1zrKY/sZJFUL3qF+J/XT018GWRUJKkhAs8PxSmDRsKiCDikimDDppYgrKjNCvEYKYSNratqS/AXv7xMOmcN32v49+f15lVZRwUcgWNwCnxwAZrgFrRAG2DwBJ7BK3hzcufFeXc+5qMrTrlzCP7A+fwBopiUyw==</latexit><latexit sha1_base64="sfcetjjriA53KVhP8LRkSGs9KNA=">AAAB+3icbVDLSsNAFJ34rPUV69LNYBFclUQEXbgouNCVVLAPaEOYTCft0HmEmYlYQn7FjQtF3Poj7vwbJ20W2npg4HDOvdwzJ0oY1cbzvp2V1bX1jc3KVnV7Z3dv3z2odbRMFSZtLJlUvQhpwqggbUMNI71EEcQjRrrR5Lrwu49EaSrFg5kmJOBoJGhMMTJWCt3agCMzVjy7ucvDDEtF8tCtew1vBrhM/JLUQYlW6H4NhhKnnAiDGdK673uJCTKkDMWM5NVBqkmC8ASNSN9SgTjRQTbLnsMTqwxhLJV9wsCZ+nsjQ1zrKY/sZJFUL3qF+J/XT018GWRUJKkhAs8PxSmDRsKiCDikimDDppYgrKjNCvEYKYSNratqS/AXv7xMOmcN32v49+f15lVZRwUcgWNwCnxwAZrgFrRAG2DwBJ7BK3hzcufFeXc+5qMrTrlzCP7A+fwBopiUyw==</latexit>
⇥M<latexit sha1_base64="xCEPSgjeJaAOppNxwTZXrwRukIg=">AAAB73icbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0MIiYGMjRDAfkBxhb7OXLNnbu+zOCeHIn7CxUMTWv2Pnv3GTXKGJDwYe780wMy9IpDDout9OYW19Y3OruF3a2d3bPygfHjVNnGrGGyyWsW4H1HApFG+gQMnbieY0CiRvBaPbmd964tqIWD3iJOF+RAdKhIJRtFK7iyLihtz3yhW36s5BVomXkwrkqPfKX91+zNKIK2SSGtPx3AT9jGoUTPJpqZsanlA2ogPesVRRu8bP5vdOyZlV+iSMtS2FZK7+nshoZMwkCmxnRHFolr2Z+J/XSTG89jOhkhS5YotFYSoJxmT2POkLzRnKiSWUaWFvJWxINWVoIyrZELzll1dJ86LquVXv4bJSu8njKMIJnMI5eHAFNbiDOjSAgYRneIU3Z+y8OO/Ox6K14OQzx/AHzucPqJqPrw==</latexit><latexit sha1_base64="xCEPSgjeJaAOppNxwTZXrwRukIg=">AAAB73icbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0MIiYGMjRDAfkBxhb7OXLNnbu+zOCeHIn7CxUMTWv2Pnv3GTXKGJDwYe780wMy9IpDDout9OYW19Y3OruF3a2d3bPygfHjVNnGrGGyyWsW4H1HApFG+gQMnbieY0CiRvBaPbmd964tqIWD3iJOF+RAdKhIJRtFK7iyLihtz3yhW36s5BVomXkwrkqPfKX91+zNKIK2SSGtPx3AT9jGoUTPJpqZsanlA2ogPesVRRu8bP5vdOyZlV+iSMtS2FZK7+nshoZMwkCmxnRHFolr2Z+J/XSTG89jOhkhS5YotFYSoJxmT2POkLzRnKiSWUaWFvJWxINWVoIyrZELzll1dJ86LquVXv4bJSu8njKMIJnMI5eHAFNbiDOjSAgYRneIU3Z+y8OO/Ox6K14OQzx/AHzucPqJqPrw==</latexit><latexit sha1_base64="xCEPSgjeJaAOppNxwTZXrwRukIg=">AAAB73icbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0MIiYGMjRDAfkBxhb7OXLNnbu+zOCeHIn7CxUMTWv2Pnv3GTXKGJDwYe780wMy9IpDDout9OYW19Y3OruF3a2d3bPygfHjVNnGrGGyyWsW4H1HApFG+gQMnbieY0CiRvBaPbmd964tqIWD3iJOF+RAdKhIJRtFK7iyLihtz3yhW36s5BVomXkwrkqPfKX91+zNKIK2SSGtPx3AT9jGoUTPJpqZsanlA2ogPesVRRu8bP5vdOyZlV+iSMtS2FZK7+nshoZMwkCmxnRHFolr2Z+J/XSTG89jOhkhS5YotFYSoJxmT2POkLzRnKiSWUaWFvJWxINWVoIyrZELzll1dJ86LquVXv4bJSu8njKMIJnMI5eHAFNbiDOjSAgYRneIU3Z+y8OO/Ox6K14OQzx/AHzucPqJqPrw==</latexit><latexit sha1_base64="xCEPSgjeJaAOppNxwTZXrwRukIg=">AAAB73icbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0MIiYGMjRDAfkBxhb7OXLNnbu+zOCeHIn7CxUMTWv2Pnv3GTXKGJDwYe780wMy9IpDDout9OYW19Y3OruF3a2d3bPygfHjVNnGrGGyyWsW4H1HApFG+gQMnbieY0CiRvBaPbmd964tqIWD3iJOF+RAdKhIJRtFK7iyLihtz3yhW36s5BVomXkwrkqPfKX91+zNKIK2SSGtPx3AT9jGoUTPJpqZsanlA2ogPesVRRu8bP5vdOyZlV+iSMtS2FZK7+nshoZMwkCmxnRHFolr2Z+J/XSTG89jOhkhS5YotFYSoJxmT2POkLzRnKiSWUaWFvJWxINWVoIyrZELzll1dJ86LquVXv4bJSu8njKMIJnMI5eHAFNbiDOjSAgYRneIU3Z+y8OO/Ox6K14OQzx/AHzucPqJqPrw==</latexit>
⇡(E)<latexit sha1_base64="jh7gUoALVOlBmg3iPkbszI9veEE=">AAAB7XicbVBNSwMxEJ3Ur1q/qh69BItQL2VXBD0WROixgq2FdinZNNvGZpMlyQpl6X/w4kERr/4fb/4b03YP2vpg4PHeDDPzwkRwYz3vGxXW1jc2t4rbpZ3dvf2D8uFR26hUU9aiSijdCYlhgkvWstwK1kk0I3Eo2EM4vpn5D09MG67kvZ0kLIjJUPKIU2Kd1O4lvHp73i9XvJo3B14lfk4qkKPZL3/1BoqmMZOWCmJM1/cSG2REW04Fm5Z6qWEJoWMyZF1HJYmZCbL5tVN85pQBjpR2JS2eq78nMhIbM4lD1xkTOzLL3kz8z+umNroOMi6T1DJJF4uiVGCr8Ox1POCaUSsmjhCqubsV0xHRhFoXUMmF4C+/vEraFzXfq/l3l5V6I4+jCCdwClXw4Qrq0IAmtIDCIzzDK7whhV7QO/pYtBZQPnMMf4A+fwCn5I6H</latexit><latexit sha1_base64="jh7gUoALVOlBmg3iPkbszI9veEE=">AAAB7XicbVBNSwMxEJ3Ur1q/qh69BItQL2VXBD0WROixgq2FdinZNNvGZpMlyQpl6X/w4kERr/4fb/4b03YP2vpg4PHeDDPzwkRwYz3vGxXW1jc2t4rbpZ3dvf2D8uFR26hUU9aiSijdCYlhgkvWstwK1kk0I3Eo2EM4vpn5D09MG67kvZ0kLIjJUPKIU2Kd1O4lvHp73i9XvJo3B14lfk4qkKPZL3/1BoqmMZOWCmJM1/cSG2REW04Fm5Z6qWEJoWMyZF1HJYmZCbL5tVN85pQBjpR2JS2eq78nMhIbM4lD1xkTOzLL3kz8z+umNroOMi6T1DJJF4uiVGCr8Ox1POCaUSsmjhCqubsV0xHRhFoXUMmF4C+/vEraFzXfq/l3l5V6I4+jCCdwClXw4Qrq0IAmtIDCIzzDK7whhV7QO/pYtBZQPnMMf4A+fwCn5I6H</latexit><latexit sha1_base64="jh7gUoALVOlBmg3iPkbszI9veEE=">AAAB7XicbVBNSwMxEJ3Ur1q/qh69BItQL2VXBD0WROixgq2FdinZNNvGZpMlyQpl6X/w4kERr/4fb/4b03YP2vpg4PHeDDPzwkRwYz3vGxXW1jc2t4rbpZ3dvf2D8uFR26hUU9aiSijdCYlhgkvWstwK1kk0I3Eo2EM4vpn5D09MG67kvZ0kLIjJUPKIU2Kd1O4lvHp73i9XvJo3B14lfk4qkKPZL3/1BoqmMZOWCmJM1/cSG2REW04Fm5Z6qWEJoWMyZF1HJYmZCbL5tVN85pQBjpR2JS2eq78nMhIbM4lD1xkTOzLL3kz8z+umNroOMi6T1DJJF4uiVGCr8Ox1POCaUSsmjhCqubsV0xHRhFoXUMmF4C+/vEraFzXfq/l3l5V6I4+jCCdwClXw4Qrq0IAmtIDCIzzDK7whhV7QO/pYtBZQPnMMf4A+fwCn5I6H</latexit><latexit sha1_base64="jh7gUoALVOlBmg3iPkbszI9veEE=">AAAB7XicbVBNSwMxEJ3Ur1q/qh69BItQL2VXBD0WROixgq2FdinZNNvGZpMlyQpl6X/w4kERr/4fb/4b03YP2vpg4PHeDDPzwkRwYz3vGxXW1jc2t4rbpZ3dvf2D8uFR26hUU9aiSijdCYlhgkvWstwK1kk0I3Eo2EM4vpn5D09MG67kvZ0kLIjJUPKIU2Kd1O4lvHp73i9XvJo3B14lfk4qkKPZL3/1BoqmMZOWCmJM1/cSG2REW04Fm5Z6qWEJoWMyZF1HJYmZCbL5tVN85pQBjpR2JS2eq78nMhIbM4lD1xkTOzLL3kz8z+umNroOMi6T1DJJF4uiVGCr8Ox1POCaUSsmjhCqubsV0xHRhFoXUMmF4C+/vEraFzXfq/l3l5V6I4+jCCdwClXw4Qrq0IAmtIDCIzzDK7whhV7QO/pYtBZQPnMMf4A+fwCn5I6H</latexit>
Structured model-free policies for physical construction in humans and AI
Hamrick et al., 2018, Proc Cog Sci
Structured model-free policies for physicalconstruction in humans and AI
![Page 48: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/48.jpg)
Graph-structured representations for model-free RL
48
![Page 49: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/49.jpg)
Zambaldi et al., 2018, arXiv/under review
Relational deep reinforcement learningUnderlying graphObservation
Bra
nch
leng
th =
1B
ranc
h le
ngth
= 3
0 1 2 3 4 5 6 7 8Environment steps 1e8
1e8
0.0
0.2
0.4
0.6
0.8
1.0
Frac
tion
solv
ed
Environment steps
Frac
tion
solv
ed
0 2 4 6 8 10 12 140.0
0.2
0.4
0.6
0.8
1.0
Relational (1 block)Relational (2 blocks)Baseline (3 blocks)Baseline (6 blocks)
Relational (2 block)Relational (4 blocks)
Box-World:• Acquire gem (white) by
opening a sequence of locked boxes
• Model-free (A2C) with self-attention/GN state representation, and message-passing
Relational deep reinforcement learning
Box-World: • Acquire gem (white) by
opening a sequence of locked boxes
• Model-free (A2C) withself-attention / GN state representation, and message-passing
49Zambaldi et al., 2018, arXiv/under reviewZambaldi et al., 2018, arXiv/under review
Relational deep reinforcement learning
Box-World:• Acquire gem (white) by
opening a sequence of locked boxes
• Model-free (A2C) with self-attention/GN state representation, and message-passing
![Page 50: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/50.jpg)
ConclusionsHuman use richly structured generative knowledge• Combinatorial generalization: “Infinite use of finite means”
• Object- and relation-centric representations
• Structured mental simulation
Graph Networks: strong relational inductive bias• Naturally support combinatorial generalization via compositional sharing
• Graph-structured representations and policies
• Open-source library: github.com/deepmind/graph_nets (with demos, includingphysics!)
50
![Page 51: lec10-modeling-physics - Hacettepe Üniversitesiaykut/classes/... · Forwardmodel: supervised, 1-step trainingw/ randomcontrolinputs 31 Forward model: supervised, 1-step training](https://reader033.vdocuments.us/reader033/viewer/2022050203/5f5674093cf9595f7c1ee27d/html5/thumbnails/51.jpg)
Reject false choices
• The “bias versus variance trade-off” is real—however the emphasis shouldn’t be on “versus", but rather on “trade-off”.
• Biology doesn’t choose between nature versus nurture. It uses nature andnurture jointly, to build wholes which are greater than the sums of their parts.
• There’s great promise in synthesizing new techniques by drawing on the full AI toolkit and marrying the best approaches from today with those which wereessential during times when data and computation were at a premium.
51
• Nature and Nurture
• Structure and Flexibility
• Symbolic and Connectionist
• Hand-engineered and End-to-end