Upload File

deep multi-task learning with shared memory

14
Deep Multi-Task Learning with Shared Memory Pengfei Liu, Xipeng Qiu, Xuanjing Huang EMNLP2016 reading group presenter: ryosuke miyazaki

Upload: marujirou

Post on 12-Apr-2017

156 views

Category:

Technology


2 download

Report

TRANSCRIPT

Page 1: Deep Multi-Task Learning with Shared Memory

DeepMulti-TaskLearningwithSharedMemoryPengfei Liu,Xipeng Qiu,Xuanjing HuangEMNLP2016readinggrouppresenter:ryosukemiyazaki

Page 2: Deep Multi-Task Learning with Shared Memory

AbstractDuetothelargenumberofparametersneuralmodelsneedalarge-scalecorpus. →unsupervisedpre-trainingiseffectiveMulti-tasklearningalsoimprovethefinalperformance.ThispaperproposeLSTMwithexternalmemoryformulti-tasklearning.

Page 3: Deep Multi-Task Learning with Shared Memory

Model:ME-LSTM

Keyvector,Erasevector,Addvector

Page 4: Deep Multi-Task Learning with Shared Memory

Model:ME-LSTM

Readingoperation

Ksegment,Mdimensionsperonesegment

,

Page 5: Deep Multi-Task Learning with Shared Memory

Model:ME-LSTM

DeepFusionstrategy

Page 6: Deep Multi-Task Learning with Shared Memory

Model:ME-LSTM

Writingoperation

Page 7: Deep Multi-Task Learning with Shared Memory

Twoarchitectures

ARC-1 ARC-2

Page 8: Deep Multi-Task Learning with Shared Memory

TrainingTask-specificoutputlayer

Linearcombinationofcostfunction

λm istheweightsforeachtaskm

Page 9: Deep Multi-Task Learning with Shared Memory

Experiment:textclassification

Page 10: Deep Multi-Task Learning with Shared Memory

Result:Movie

Page 11: Deep Multi-Task Learning with Shared Memory

Result:Product

Page 12: Deep Multi-Task Learning with Shared Memory

Analysis:Visualizedeepfusiongate

Sentimentscore

Dimensionsofdeepfusiongategt

Activate→ black

Page 13: Deep Multi-Task Learning with Shared Memory

Analysis:Visualizedeepfusiongate

Page 14: Deep Multi-Task Learning with Shared Memory

Conclusion・ Thispaperproposetwodeeparchitecturesformulti-tasklearning.・ Theydesignanexternalmemorytostoretheknowledgebyrelatedtasks.・ Deepfusionstrategyenablingthemodeltogivesharedinformation.


Simulink shared memory

Memory Coherence in Shared Virtual Memory Systemsdahlin/Classes/GradOS/papers/p321-li.pdf · A prototype shared virtual memory on an Apollo ring based ... Memory Coherence in Shared

Shared Memory and OpenMP - NYU Courantstadler/hpc17/material/ompLec.pdfShared Memory and OpenMP Background Shared Memory Hardware Shared Memory Languages OpenMP. Parallel Hardware

Shared Memory – Consistency of Shared Variables

CIS 720 Distributed Shared Memory. Shared Memory Shared memory programs are easier to write Multiprocessor systems Message passing systems: - no physically

16 Distributed Shared Memory - Osos.itec.kit.edu/downloads/DS16_DSM1.pdfDistributed Shared Memory Distributed Shared Memory (DSM) allows applications running on separate computers

Shared Memory Considerations

Shared Memory Introduction - Kohala Software: Home · PDF file · 2008-02-07SIGSEGV. Shared Memory Introduction Chapter 12. Shared Memory Introduction Chapter 12 12.1

CSE 502: Computer ArchitectureCSE502: Computer Architecture Shared-Memory Multiprocessors •Multiple threads use shared memory (address space) –“SysV Shared Memory” or “Threads”

Shared Memory Architecture

Shared Memory Multiprocessing

CS519: Lecture 10 z (Software) Distributed Shared Memory (aka Shared Virtual Memory)

Shared Memory

Distributed Shared Memory (part 1). Distributed Shared Memory (DSM) mem0 proc0 mem1 proc1 mem2 proc2 memN procN network... shared memory

Shared-memory Architectures

Shared Memory Consistency Models. Quiz (1) Let’s define shared memory

Shared Memory Multiprocessors

Dsm Shared Memory

Shared Memory Multiprocessors

Shared Memory – Consistency of Shared Variables The ideal picture of shared memory: CPU0CPU1CPU2CPU3 Shared Memory Read/ Write The actual architecture

L7 Shared Memory Multiprocessors · L7 Shared Memory Multiprocessors Logical design and software interactions 2 Shared Memory Multiprocessors Symmetric Multiprocessors (SMPs) •

Unix Shared Memory - Intranet DEIBhome.deib.polimi.it/fornacia/lib/exe/fetch.php?media=teaching:... · Unix Shared Memory. 2 What is Shared Memory?

Shared Memory Coordination

1 Shared Memory. 2 Introduction Creating a Shared Memory Segment Shared Memory Control Shared Memory Operations Using a File as Shared Memory

Shared memory architectures

HPC Architectures · Outline • Shared memory architectures • Distributed memory architectures • Distributed memory with shared-memory nodes • Accelerators • What is the

Winter 2013Parallel Processing, Shared-Memory ParallelismSlide 1 Part II Shared-Memory Parallelism Part II Shared-Memory Parallelism 5. PRAM and Basic

Distributed Shared Memory Providing a shared-memory abstraction in a distributed memory system

Lecture 5: Shared-memory Computing with Open MP. Shared Memory Computing

Shared Memory & Synchronization

Shared Memory Systems

CS 551-Memory Management1 Learning Objectives Centralized Memory Management -review Simple Memory Model Shared Memory Model Distributed Shared Memory Memory

Memory Coherence in Shared Virtual Memory Systemsdcslab.snu.ac.kr/courses/dip2016f/Papers/01_Memory... · 2019-03-16 · Memory Coherence in Shared Virtual Memory Systems l 323 Shared

Distributed Shared Memory (DSM). Distributed Shared Memory Message Passing Paradigm Send (recipient, data) Receive (data) Shared Memory Paradigm Data=

Distributed Shared Memory Distributed Memory[1]