Upload File

visualizing a convolutional neural networktmm/courses/547-17/projects/hooman-mahdi/... · this...

  • Home
  • Documents
  • Visualizing a Convolutional Neural Networktmm/courses/547-17/projects/hooman-mahdi/... · This project as requested by Mahdi’s supervisor, focuses on delivering a tool that resembles
5
Visualizing a Convolutional Neural Network Mahdi Ghodsi, [email protected] Hooman Shariati, [email protected] Recently, Deep Neural Network (DNN) has gained much attention due to its success in improving tasks such as image classification and speech recognition. Amongst different DNN approaches Convolution Neural Networks (CNNs) is extremely popular in particular due to their outstanding capacity to utilize spatial information. We have taken a course (EECE592) that covers DNNs. Currently, there has been a shift on applying bioinformatics data to CNNs[1]. While much success have been achieved in the biomedical imaging domain [2] using CNNs, there is currently ongoing research in other bioinformatics domains such as have been successful in many application and have been applied to many fields such as genomic sequence [3] motifs and EEG[4]. Despite the encouraging success of CNNs, many still see CNNs as a promising black box with little insight into the behavior of internal components of CNNs. This fact leaves many researchers to relying on try-and error to achieve better performance and fine-tune the hyper parameters involved. Large modern neural networks are even harder to study because of their size; for example, understanding the widely used AlexNet DNN involves making sense of the values taken by the 60 million trained network parameters [5,6]. Recently, visualizing DNNs in particular CNNs has resulted in many publications. The most cited and basis of recent approaches are based on a paper by Zeiler and Ferfus et al. in which they propose a multi-layered deconvolutional Network (deconvnet) to project the feature activations back to the input pixel space for a trained network. This technique reveals the input stimuli that excite individual feature maps at any layer in the model [5]. The second most cited paper by Yosinski et al. improves this method by proposing several new regularization methods that combine to produce more human interpretable visualizations [6]. Recently, Google’s Tensorflow (an open source software library for machine learning) is complemented by a visualization tool called Tensorboard [7]. In addition to the above tools and many more scholarly published papers, there are many publicly available online tools that are worth investigating [8].

Upload: vuonghanh

Post on 14-Sep-2018

217 views

Category:

Documents


0 download

Report

TRANSCRIPT

Page 1: Visualizing a Convolutional Neural Networktmm/courses/547-17/projects/hooman-mahdi/... · This project as requested by Mahdi’s supervisor, focuses on delivering a tool that resembles

VisualizingaConvolutionalNeuralNetworkMahdiGhodsi,[email protected]

HoomanShariati,[email protected]

Recently,DeepNeuralNetwork(DNN)hasgainedmuchattentionduetoitssuccessinimproving

taskssuchasimageclassificationandspeechrecognition.AmongstdifferentDNNapproaches

ConvolutionNeuralNetworks(CNNs)isextremelypopularinparticularduetotheiroutstanding

capacitytoutilizespatialinformation.Wehavetakenacourse(EECE592)thatcoversDNNs.

Currently,therehasbeenashiftonapplyingbioinformaticsdatatoCNNs[1].Whilemuchsuccess

havebeenachievedinthebiomedicalimagingdomain[2]usingCNNs,thereiscurrentlyongoing

researchinotherbioinformaticsdomainssuchashavebeensuccessfulinmanyapplicationand

havebeenappliedtomanyfieldssuchasgenomicsequence[3]motifsandEEG[4].

DespitetheencouragingsuccessofCNNs,manystillseeCNNsasapromisingblackboxwithlittle

insightintothebehaviorofinternalcomponentsofCNNs.Thisfactleavesmanyresearchersto

relyingontry-anderrortoachievebetterperformanceandfine-tunethehyperparameters

involved.Largemodernneuralnetworksareevenhardertostudybecauseoftheirsize;for

example,understandingthewidelyusedAlexNetDNNinvolvesmakingsenseofthevaluestaken

bythe60milliontrainednetworkparameters[5,6].

Recently,visualizingDNNsinparticularCNNshasresultedinmanypublications.Themostcited

andbasisofrecentapproachesarebasedonapaperbyZeilerandFerfusetal.inwhichthey

proposeamulti-layereddeconvolutionalNetwork(deconvnet)toprojectthefeatureactivations

backtotheinputpixelspaceforatrainednetwork.Thistechniquerevealstheinputstimulithat

exciteindividualfeaturemapsatanylayerinthemodel[5].Thesecondmostcitedpaperby

Yosinskietal.improvesthismethodbyproposingseveralnewregularizationmethodsthat

combinetoproducemorehumaninterpretablevisualizations[6].Recently,Google’sTensorflow

(anopensourcesoftwarelibraryformachinelearning)iscomplementedbyavisualizationtool

calledTensorboard[7].Inadditiontotheabovetoolsandmanymorescholarlypublished

papers,therearemanypubliclyavailableonlinetoolsthatareworthinvestigating[8].

Page 2: Visualizing a Convolutional Neural Networktmm/courses/547-17/projects/hooman-mahdi/... · This project as requested by Mahdi’s supervisor, focuses on delivering a tool that resembles

Thisproject’smaingoalistocombinethepreviousworksinacompleteinteractivevisualization

toolthatfocusesonvisualizingtheoperationofinternalcomponentsofagivenneuralnetwork.

ThisprojectasrequestedbyMahdi’ssupervisor,focusesondeliveringatoolthatresemblesthe

familiararchitecturethatisshowninfigure1.Therequirementsforthistoolarederivedfroma

studythatsurveyedresearchersandidentifiedthemaintasksforavisualizationtool[9].Afew

ofthesetaskscombinedwithwhatMahdi’ssupervisor’srequestsareselectedandweaimto

deliverthemwithinourfinalsolution.Thesetasksare:

1. Overallviewofthearchitectureandnetworkdepth

2. Overviewoflearnedfeaturesofneurons

3. Interactivelyexploringmultiplefacetsofneurons(learnedfeatures,activation,numerical

values)

4. Theinputtothenetworkandtheeffectofapplyingconvolutionthroughoutthesystem

Inordertodeliveratoolthatfocusesontheabovetasks,wearehopingtouseaCNN,possibly

fedbyMNISThandwrittendigitaldatabase,thatispublicallyavailableandproducetherequired

databymodifying/addingtotheavailablecode.Thedataproducedwillbeasetofweightsfor

eachneuronwithineachlayerofthenetwork.Thenetworkisconsistedofdifferentlayersand

dependingonthelayerandtheneuronthenumberoflearnableweightsdiffer.Forinstancea

fullyconnectedlayerforatypicalMNISTfedCNNhasatotalapproximately128neurons.Eachof

theseneuronstakestheoutputofpreviouslayerthatinthiscaseare1764values.Thesevalues

arethenprocessedandasingleoutputisproducedbyeachneuron.Ourtaskwillbeinvestigating

previouspublicationstorealizethestateofartmethodsinproducingameaningfulvisualization.

Thisisslightlydifferentforvisualizingconvolutionallayers,aseachlayerwillconsistofasetof

convolutionalfilters(forinstance5x5filters)thatallowsustovisualizeasetofweightsasone

filter.Forinstanceaconvolutionallayermayconsistsof16filtersof5x5sized,allowingusto

visualize16boxesratherthan16x5x5=375values.

Belowisascenarioforuser“X”,whohasjusttrainedaCNNandwantstovisualizethenetwork:

Page 3: Visualizing a Convolutional Neural Networktmm/courses/547-17/projects/hooman-mahdi/... · This project as requested by Mahdi’s supervisor, focuses on delivering a tool that resembles

Xhastorunaforwardpropagationofoneinputsampleandincludetheinstructionsgivenbyus

forhowtodosoandthenthetoolwillbereadytouse.Thetoolaggregatesallthedetailed

informationandillustratesanoverviewofthearchitecturesuchastheonegiveninfigure1.

ThenXcanchoosetoexplorethefiltersinfirstconvolutionallayer.Xneedstoclickonthe

appropriatelayertoseemoredetails.Thedetailscouldbefiltersshown.IftheXwantstoseethe

effectofthatfilterontheinputimage,therewillbeabutton/slidertochangethedetailshownon

demand.Theconnectionbetweenthatfilterandalltheothernodes/filtersareshown.Forany

otherinformationabouteachlayerXneedstofollowthesamelogicofclicktoviewmoredetails.

Thistoolwillhelpresearcherstogetsomeinsightabouthoweachindividualneuronisactivated

orinformationinregardstotheoutputofasetofneuronsthatconstructonefulllayerofthe

NeuralNetworkarchitecture.ThescopeofthisprojectislimitedtoatrainedCNNtoavoid

complexityofrunningdynamicanalysisofthenetworkwhichrequiresheavycomputationand

ontheotherhanditisnotaseasytointerpretforahumanusersinceeachtrainingsetcontains

thousandsofdatapoints.VisualizingatrainedCNNmaygiveadomainexpertenoughintuition

toanalyzeandimprovetheperformanceoforiginalnetwork.

ThefinalproductwillbedeliveredbytheendofAprilasatoolthatcouldvisualizeatrainedCNN

ofourchoice.However,itwillbecomplementedwithenoughdocumentationusableforother

researcherswhowhishtovisualizeagivenCNN.

Page 4: Visualizing a Convolutional Neural Networktmm/courses/547-17/projects/hooman-mahdi/... · This project as requested by Mahdi’s supervisor, focuses on delivering a tool that resembles

Figure1-OverviewofCNNtovisualize

Figure2-ProjectMilestone

Page 5: Visualizing a Convolutional Neural Networktmm/courses/547-17/projects/hooman-mahdi/... · This project as requested by Mahdi’s supervisor, focuses on delivering a tool that resembles

References:

[1]S.Min,B.Lee,andS.Yoon,“Deeplearninginbioinformatics,”BriefingsinBioinformatics,

2016.

[2]Roth,HolgerR.,LeLu,AmalFarag,Hoo-ChangShin,JiaminLiu,EvrimB.Turkbey,andRonald

M.Summers."DeepOrgan:Multi-levelDeepConvolutionalNetworksforAutomatedPancreas

Segmentation."LectureNotesinComputerScienceMedicalImageComputingandComputer-

AssistedIntervention--MICCAI2015(2015):556-64.Web.

[3]J.Lanchantin,R.Singh,B.Wang,andY.Qi,“DeepMotifDashboard:VisualizingAnd

UnderstandingGenomicSequencesUsingDeepNeuralNetworks,”Biocomputing2017,2016.

[4]P.Bashivan,I.Rish,M.Yeasin,N.Codella,“LearningRepresentationFromEEGWithDeep

Recurrent-ConvolutionNeuralNetwokrs,”Preceding5thInternationalConferenceonLearning

Representations(ICLR),2016.

[5]Zeiler,MatthewD.,andRobFergus."Visualizingandunderstandingconvolutionalnetworks."

Europeanconferenceoncomputervision.SpringerInternationalPublishing,2014.

[6]Yosinski,Jason,etal."Understandingneuralnetworksthroughdeepvisualization."arXiv

preprintarXiv:1506.06579(2015).

[7]https://www.tensorflow.org/get_started/summaries_and_tensorboard

[8]https://handong1587.github.io/deep_learning/2015/10/09/visulize-cnn.html

[9]Liu, Mengchen, et al. "Towards better analysis of deep convolutional neural

networks." IEEE Transactions on Visualization and Computer Graphics 23.1 (2017):

91-100.


Darwin's Theory ·Our anatomy resembles the anatomy of other animals

Equity Market Volatility Resembles 2008’s Final Quarter · Equity Market Volatility Resembles 2008’s Final Quarter From the perspective of the U.S. equity market, COVID-19 is

Using IDs to Improve Web Navigation with a Keyboard - Hooman

Portfolio,hooman khadem

Island Based GA for Optimization University of Guelph School of Engineering Hooman Homayounfar March 2003

Dec 2005, Changing Times Understanding the Logic of Atheism Creating a Bridge of Understanding Hooman Katirai

Chapter 1 - CorwinChapter 1 A Life Course Perspective Elizabeth D. Hutchison Key Ideas Case Study 1.1: David Sanchez’s Search for Connections Case Study 1.2: Mahdi Mahdi’s Shared

Hooman invaders tower defense the story a free android, i os and webgl game

Imam Mahdi (ATFS)islamicmobility.com/pdf/8_Imam_Mahdi_sakina_askari.pdf · Imam Mahdi’s (AS) father was our eleventh Imam Hasan As-kari (AS),who was the son of the 10th Imam Ali

A city charter closely resembles a..? Constitution

China's toxic air pollution resembles nuclear winter, say scientists _ Environment _ theguardian.pdf

The Drina - Tara region resembles a patchwork of breathtaking naturalseerural.org/wp-content/uploads/2014/11/4.-Attractions... · 2014-11-17 · The Drina - Tara region resembles

QUARTERLY NEWSLETTER · QUARTERLY NEWSLETTER. Welcome. August 2020. Hooman Hakami Appointed to Quanta Board. Quanta announced the appointment of. Hooman Hakami to the Company’s

Cloud Connect Santa Clara 2013: Web Acceleration and Front-End Optimization (Hooman Beheshti)

RWCS 2014 Hooman Dehghan, MD Rheumatology Fellow Stanford University

Fruits and Vegetables that Resembles to Body Organs and … · Fruits and Vegetables that Resembles to Body Organs and have Significant Role on them Bhavani Ramesh. T, Prem Kumar

The Forty Hadith About the Mahdi By Hafiz Abu Na eem Al ...3 يمركلا نآرقلا و ءادوسلا بوقثلا في ةلاسر [email protected] | The 1st Hadith Al Mahdi’s

Copia conforme all’originale Copy resembles original i.V ... · Copia conforme all’originale Copy resembles original i.V. Frank Greizer ZDK594011_17-11:BI1007

Something that resembles someone you Something red that ...images.quickhunts.com/scavenger/dfe884cec8112702d055b8a260c… · Something red that you can twist. Something that resembles

Hooman M. Melamed, MD...Page 3 of 11 Hooman M. Melamed, M.D. Orthopaedic Spinal Surgeon ph (310) 574-0400 fax (310) 574-0422 13160 Mindanao Way, Suite 300 8750 Wilshire Blvd., …

Nephrotic vs Nephritic syndrome - iums.ac.iriums.ac.ir/uploads/problem_based_NS_vs_APSGN_12264.pdf · Nephrotic vs Nephritic syndrome Hooman N .MD ... (Lasix, thiazides) •Albumin

Case Study MICR 410 - Hematology Spring, 2011 Case # 5 Hee Jin Kim, Hooman Nikizad and Arthur Omuro

Electronic Submission and Reviewing Methodology Hooman Momen Editor Bulletin of the World Health Organization

 · Taste delicious of a spicy lemon • Suitable for Halal accreditation Full of natural ingredients with no artificial flavours Marketing resembles the taste of burning lemons Resembles

SPA2015: Hooman Beheshti – The Future of CDNs

2014 Toyota Tundra Brochure - Hooman Toyota, a Long Beach Dealer Serving All Of Los Angeles and Orange County

Language barriers and translation, implications for open access journals Hooman Momen World Health Organization

How to manage projectsUnravel ‘04 HOW TO MANAGE PROJECTS Hooman Katirai · Unravel 2004 · Toronto

Hooman Toyota Coming Soon To Willow - Long Beach Business Journal April 29 - May 12 2014

Another galaxy: NGC 4414. The Milky Way roughly resembles it

Name Hometown Degree Evelyn’R.’Perrin Skien,’Norway ... · Name Hometown Degree Evelyn’R.’Perrin Skien,’Norway Interdisciplinary’Studies(BIS) Mahdi’S.’Al’Abalatihy

Facility Location Problems Hooman Mohajeri Moghaddam Winter 2011

Peace be upon you, O first of the martyrs from among the scions of the noblest descendants of the Prophet Ibrahim, Friend of Allah. Imam Mahdi’s (ATFS)

Basal Cell Glycoprotein in Pig Epidermis Closely Resembles ... · Basal Cell Glycoprotein in Pig Epidermis Closely Resembles the ßl Subunit of the Integrin Family of Cell Adhesion

Religious Normative Pluralism in Imam Mahdi’s Mode of ...coth.ui.ac.ir/article_22885_dcf63e34039a8b9e3d7431c870497e27.pdf · Comparative Theology, Vol: 9, No: 19, Spring & Summer