openai five model architecture · openai five model architecture (06/06/2018) title:...

Player 5 Player 4 Player 3 Player 2 Player 1 nearby terrain 8x8 grid of height, traversability, creep occupancy for each hero in team Ability N Ability N Ability N Pickup 1 Embedding Pickup Type distance from all team heroes, present/missing concat FC-relu FC-relu max-pool Unit N … Unit 2 Unit 1 Heroes only LSTM 1024 units Unit Type distance from allied heroes, enemy heroes orientation absolute position animation health over last 12 frames unit stats (health, regen, attack,…) Embedding Embedding cos sin Ability N Ability N Ability N Ability 1 Embedding Ability type Stats (cooldown, etc.) concat FC-relu FC-relu max-pool Ability N Ability N Ability N Item 1 Embedding Item type Stats (charges, etc.) concat FC-relu FC-relu max-pool Ability N Ability N Ability N Modifier 1 Embedding Modifier type Stats (duration, etc.) concat FC-relu FC-relu max-pool concat attacking/ attacked by hero for all allied and enemy heroes FC-relu FC-relu FC-relu FC Enemy Heroes Allied Heroes Enemy Non-Heroes Allied Non-Heroes Neutrals concat max-pool slice 0:64 FC-relu FC max-pool slice 0:64 FC-relu FC max-pool slice 0:64 FC-relu FC max-pool slice 0:64 FC-relu FC max-pool slice 0:64 concat •Allied & enemy glyph cooldown •is Night •time until creepwave •time since enemy courier last seen •time until night •courier number of flask, clarity, enchanted mangoes, town portals, magic sticks •Total value courier items FC-relu FC Available Actions Embedding dot Softmax Selected Action Sample/Argmax FC Softmax Offset X Sample/Argmax FC Softmax Offset Y Sample/Argmax FC Softmax Move X Sample/Argmax FC Softmax Move Y Sample/Argmax FC Softmax Teleport Destination Sample/Argmax FC Softmax Delay Sample/Argmax dot FC Unit Attention Keys Softmax Target Unit Sample/Argmax •Buyback Cost, Cooldown •Number of deaths •Ability is active, used, or phased •Teleport destination, time, ongoing •Team •Level, Max Mana, Magic resist, Agility, Intelligence, etc. OpenAI Five Model Architecture (06/06/2018)

Upload: others

Post on 19-Jun-2020

4 views

Category:

Documents

0 download

Report

Download

Embed Size (px):

TRANSCRIPT

Page 1: OpenAI Five Model Architecture · OpenAI Five Model Architecture (06/06/2018) Title: dota_network_diagram Created Date: 6/24/2018 4:00:19 PM

Player 5Player 4

Player 3Player 2

Player 1

nearby terrain 8x8 grid of height, traversability,

creep occupancy for each hero in team

Ability NAbility NAbility NPickup 1

Embedding

Pickup Type distance from all team heroes, present/missing

concat

FC-relu

max-pool

Unit N…

Unit 2Unit 1

Heroes only

LSTM1024 units

Unit Type distance from allied

heroes, enemy heroes

orientation absolute position animationhealth over

last 12 framesunit stats (health, regen, attack,…)

Embedding Embeddingcos sin

Ability NAbility NAbility NAbility 1

Embedding

Ability type Stats (cooldown, etc.)

concat

FC-relu

max-pool

Ability NAbility NAbility NItem 1

Embedding

Item type Stats (charges, etc.)

concat

FC-relu

max-pool

Ability NAbility NAbility NModifier 1

Embedding

Modifier type Stats (duration, etc.)

concat

FC-relu

max-pool

concat

attacking/attacked by hero for all allied and

enemy heroes

FC-relu

Enemy HeroesAllied Heroes

Enemy Non-Heroes

Allied Non-Heroes Neutrals

concat

max-pool slice0:64

FC-relu

max-pool slice0:64

FC-relu

max-pool slice0:64

FC-relu

max-pool slice0:64

FC-relu

max-pool slice0:64

concat

•Allied & enemy glyph cooldown•is Night•time until creepwave•time since enemy courier last seen

•time until night•courier number of flask, clarity, enchanted mangoes, town portals, magic sticks

•Total value courier items

FC-relu

Available Actions

Embedding

dot Softmax Selected ActionSample/Argmax

FC Softmax Offset XSample/Argmax

FC Softmax Offset YSample/Argmax

FC Softmax Move XSample/Argmax

FC Softmax Move YSample/Argmax

FC Softmax Teleport DestinationSample/Argmax

FC Softmax DelaySample/Argmax

dotFC

UnitAttentionKeys

Softmax Target UnitSample/Argmax

•Buyback Cost, Cooldown•Number of deaths•Ability is active, used, or phased•Teleport destination, time, ongoing•Team•Level, Max Mana, Magic resist, Agility, Intelligence, etc.

OpenAI Five Model Architecture (06/06/2018)

Five android architecture

Five Reasons for Architecture Failure ... and How to Avoid Them

The Five Key Elements of Enterprise Services Architecture

Temple Architecture Devalaya Vastu Part Five

Extending the OpenAI Gym for robotics: a toolkit for ... · OpenAI Gym [1] is a is a toolkit for reinforcement learning research that has recently gained popularity in the machine

Landscape Architecture Planning Urban Designswacdn.s3.amazonaws.com/1/8addafd3_newsswa_awards_2009.pdfSWA Landscape Architecture Planning Urban Design /// Five 2009 National Honor

SAP’s five BPM Deliverables - The Open Grouparchive.opengroup.org/public/member/proceedings/q109/q109a/... · SAP’s five BPM Deliverables ... SAP Enterprise Service Architecture

Extending the OpenAI Gym for robotics: a toolkit for ...erlerobotics.com/whitepaper/robot_gym.pdfExtending the OpenAI Gym for robotics: a toolkit for reinforcement learning using ROS

Measuring the Algorithmic Efficiency of Neural NetworksMeasuring the Algorithmic Efﬁciency of Neural Networks Danny Hernandez OpenAI [email protected] Tom B. Brown OpenAI [email protected]

Five&Components&of&aComputer& Components… · GreatIdeas&in&Computer&Architecture&& Computer)Components) Instructors:& ... • Five&Components&of&aComputer& – Processor/Control&+Datapath&

NIPS 2016 · (2015) Google gave its introduction/tutorial on TensorFlow, released its best model on ImageNet (2015) OpenAI announced its existence OpenAI released their Universe platform