ttic 31190: natural language processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting...
TRANSCRIPT
![Page 1: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/1.jpg)
TTIC31190:NaturalLanguageProcessing
KevinGimpelWinter2016
Lecture9:SequenceModels
1
![Page 2: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/2.jpg)
Announcements• onThursday,classwillbeinRoom530(theroomdirectlybehindyou)
2
![Page 3: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/3.jpg)
Announcements• wewillgooverpartofAssignment1today(gradescomingsoon)
• Assignment2wasdueWed.Feb.3,nowdueFri.,Feb.5
• projectproposaldueTuesday,Feb.16• midtermonThursday,Feb.18
3
![Page 4: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/4.jpg)
• qualityofscientificjournalism:
4
OtherNaturally-OccurringData
![Page 5: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/5.jpg)
OtherNaturally-OccurringData• memorabilityofquotations:
5
![Page 6: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/6.jpg)
OtherNaturally-OccurringData• sarcasm(remove#sarcasmhashtagfromtweets):
6
![Page 7: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/7.jpg)
OtherNaturally-OccurringData• openingweekendmovierevenuepredictionfromcriticreviews:
7
![Page 8: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/8.jpg)
OtherNaturally-OccurringData• predictingnovelsuccessfromtextofnovels:
8
![Page 9: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/9.jpg)
ProjectProposal• dueFeb.16(intwoweeks)• 1-2pages• onepergroup• includethefollowing:– membersofyourgroup– describethetaskyouaregoingtoworkon(couldbeanewtaskyoucreateoranexistingtask)
– describethemethodsyouwilluse/developforthetask– giveabriefreviewofrelatedwork;i.e.,situateyourprojectwithrespecttotheliterature(www.aclweb.organdGoogleScholarareusefulforthis!)
– aproposedtimeline
9
![Page 10: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/10.jpg)
ProjectProposal(cont’d)
• yourresultsdonothavetobeatthestate-of-the-art!
• butyourprojectdoeshavetobecarefullydone,sothatyoucandrawconclusions
• youarewelcometostartbyreplicatinganNLPpaper(Icangivesuggestionsifyouneedsome)
• duringtheweekofFeb.22,pleasescheduleameetingwithmetodiscussyourproject– detailstofollow
10
![Page 11: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/11.jpg)
ClassPresentations• finaltwoclassmeetings(March3rd andMarch8th)willbemostlyusedforin-classpresentations
• onepresentationpergroup• 10-15minutesperpresentation(willbedeterminedonceIknowhowmanygroupsthereare)
• youwilleachtakenotesandemailmequestions/feedbackforthepresenter,whichIwillanonymize andsend
11
![Page 12: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/12.jpg)
Project• finalreportdueThursday,March17(originaldateofthefinalexam)
• sothepresentationwillbemorelikean“interimprogressreport”
12
![Page 13: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/13.jpg)
Roadmap• classification• words• lexicalsemantics• languagemodeling• sequencelabeling• neuralnetworkmethodsinNLP• syntaxandsyntacticparsing• semanticcompositionality• semanticparsing• unsupervisedlearning• machinetranslationandotherapplications
13
![Page 14: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/14.jpg)
determinerverb(past)prep.properproperposs.adj.noun
modalverbdet.adjectivenounprep.properpunc.
14
Part-of-SpeechTagging
determinerverb(past)prep.nounnounposs.adj.nounSomequestionedifTimCook’sfirstproduct
modalverbdet.adjectivenounprep.nounpunc.wouldbeabreakawayhitforApple.
Simplestkindofstructuredprediction:SequenceLabeling
![Page 15: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/15.jpg)
15
OOOB-PERSONI-PERSONOOOSomequestionedifTimCook’sfirstproduct
OOOOOOB-ORGANIZATIONOwouldbeabreakawayhitforApple.
NamedEntityRecognition
B=“begin”I=“inside”O=“outside”
FormulatingsegmentationtasksassequencelabelingviaB-I-Olabeling:
![Page 16: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/16.jpg)
• therearemanydownloadablepart-of-speechtaggersandnamedentityrecognizers:– StanfordPOStagger,NERlabeler– TurboTagger,TurboEntityRecognizer– IllinoisEntityTagger– CMUTwitterPOStagger– AlanRitter’sTwitterPOS/NERlabeler
16
![Page 17: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/17.jpg)
17
![Page 18: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/18.jpg)
HiddenMarkovModels
18
y1 y2 y3 y4
x1 x2 x3 x4
transitionparameters:
emissionparameters:
![Page 19: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/19.jpg)
HMMsforWordClustering(Brownetal.,1992)
19
eachisaclusterIDso,labelspaceis
justin bieber forpresident
y1 y2 y3 y4
![Page 20: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/20.jpg)
HMMsforPart-of-SpeechTagging
20
eachisapart-of-speechtagso,labelspaceis
whatparametersneedtobelearned?
transitionparameters:
emissionparameters:
justin bieber forpresident
propernoun
propernoun
prepo-sition
noun
![Page 21: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/21.jpg)
HowshouldwelearntheHMMparameters?
21
transitionparameters:
emissionparameters:
![Page 22: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/22.jpg)
SupervisedHMMs• givenadatasetofinputsequencesandannotatedoutputs:
• toestimatetransition/emissiondistributions,usemaximumlikelihoodestimation(countandnormalize):
22
justin bieber forpresident
propernoun
propernoun
prepo-sition
noun
![Page 23: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/23.jpg)
EstimatesofTagTransitionProbabilities
23
proper modalinfinitive adjectivenoun adverbdeterminernoun verbverb
![Page 24: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/24.jpg)
EstimatesofEmissionProbabilities
24
![Page 25: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/25.jpg)
InferenceinHMMs
25
• sincetheoutputisasequence,thisargmaxrequiresiteratingoveranexponentially-largeset
• lastweekwetalkedaboutusingdynamicprogramming(DP)tosolvetheseproblems
• forHMMs(andothersequencemodels),theforsolvingthisiscalledtheViterbialgorithm
![Page 26: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/26.jpg)
ViterbiAlgorithm• recursiveequations+memoization:
26
basecase:returnsprobabilityofsequencestartingwithlabely forfirstword
recursivecase:computesprobabilityofmax-probabilitylabelsequencethatendswithlabely atpositionm
finalvalueisin:
![Page 27: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/27.jpg)
Example:
Janetwillbackthebill
27
proper modalinfinitive determiner nounnoun verbverb
![Page 28: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/28.jpg)
Janetwillbackthebill
28
proper modalinfinitive determiner nounnoun verbverb
![Page 29: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/29.jpg)
ViterbiAlgorithm(onboard)
29
![Page 30: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/30.jpg)
ViterbiAlgorithm• spaceandtimecomplexity?• canbereadofffromtherecursiveequations:
30
spacecomplexity:sizeofmemoization table,whichis#ofuniqueindicesofrecursiveequations
so,spacecomplexityisO(|x||L|)
lengthofsentence
numberoflabels*
![Page 31: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/31.jpg)
ViterbiAlgorithm• spaceandtimecomplexity?• canbereadofffromtherecursiveequations:
31
timecomplexity:sizeofmemoization table*complexityofcomputingeachentry
so,timecomplexityisO(|x||L||L|)=O(|x||L|2)
lengthofsentence
numberoflabels*
eachentryrequiresiteratingthroughthelabels*
![Page 32: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/32.jpg)
LinearSequenceModels
• let’sgeneralizeHMMsandtalkaboutlinearmodelsforscoringlabelsequencesinourclassifierframework:
• butfirst,howdoweknowthatthisscoringfunctiongeneralizesHMMs?
32
![Page 33: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/33.jpg)
HMMasaLinearModel
• whatarethefeaturetemplatesandweights?
33
HMM:
linearmodel:
![Page 34: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/34.jpg)
HMMasaLinearModel
featuretemplatesandweights:
34
HMM:
linearmodel:
![Page 35: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/35.jpg)
LinearSequenceModels
• so,anHMMis:– alinearsequencemodel– withparticularfeaturesonlabeltransitionsandlabel-observationemissions
– andusesmaximumlikelihoodestimation(count&normalize)forlearning
• butwecoulduseanyfeaturefunctionswelike,anduseanyofourlossfunctionsforlearning!
35
![Page 36: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/36.jpg)
(Chain)ConditionalRandomFields
36
![Page 37: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/37.jpg)
(Chain)ConditionalRandomFields
• linearsequencemodel• arbitraryfeaturesofinputarepermitted• test-timeinferenceusesViterbiAlgorithm• learningdonebyminimizinglogloss(DPalgorithmsusedtocomputegradients)
37
![Page 38: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/38.jpg)
Max-MarginMarkovNetworks
38
![Page 39: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/39.jpg)
Maximum-MarginMarkovNetworks
• linearsequencemodel• arbitraryfeaturesofinputarepermitted• test-timeinferenceusesViterbiAlgorithm• learningdonebyminimizinghingeloss(DPalgorithmusedtocomputesubgradients)
39
![Page 40: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/40.jpg)
FeatureLocality
• featurelocality:roughly,how“big”areyourfeatures?
• whendesigningefficientinferencealgorithms(whetherw/DPorothermethods),weneedtobemindfulofthis
• featurescanbearbitrarilybigintermsoftheinputsequence
• butfeaturescannot bearbitrarilybigintermsoftheoutput sequence!
• thefeaturesinHMMsaresmallinboththeinputandoutputsequences(onlytwopiecesatatime)
40
![Page 41: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/41.jpg)
Arethesefeaturesbigorsmall?
41
feature bigorsmall?
featurethatcountsinstancesof“the”intheinputsentence small
featurethatreturns squarerootofsumofcountsofam/is/was/were small
featurethatcounts“verb verb”sequences small
featurethatcounts“determiner noun verb verb”sequences prettybig!
featurethatcountsthenumberofnounsinasentence
big,butwecandesignspecialized
algorithmstohandlethemifthey’retheonlybigfeatures
feature thatreturnstheratioofnounstoverbs
![Page 42: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/42.jpg)
Learningwithlinearsequencemodels• givenalinearsequencemodelwith“small”features,howshouldwedolearning?
42
![Page 43: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/43.jpg)
Lossfunctionsforlearninglinearsequencemodels
43
loss entryj of(sub)gradientofloss forlinearmodel
perceptron
hinge
log
samegradients/subgradients asbefore,thoughcomputingtheseterms(inference)requiresDP
algorithms
![Page 44: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/44.jpg)
ImplementingDPalgorithms
• startwithcountingmode,butkeepinmindhowthemodel’sscorefunctiondecomposesacrosspartsoftheoutputs– i.e.,how“large”arethefeatures?howmanyitemsintheoutputsequenceareneededtocomputeeachfeature?
– defineafunctioncalledpartScore thatcomputesallthefeatures(forcountingmode,thisfunctionwillreturn1)
44
![Page 45: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/45.jpg)
NeuralNetworksinNLP• neuralnetworks• deepneuralnetworks• neurallanguagemodels• recurrentneuralnetworksandLSTMs• convolutionalneuralnetworks
45
![Page 46: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/46.jpg)
Whatisaneuralnetwork?• justthinkofaneuralnetworkasafunction• ithasinputsandoutputs• theterm“neural”typicallymeansaparticulartypeoffunctionalbuildingblock(“neurallayers”),butthetermhasexpandedtomeanmanythings
46
![Page 47: TTIC 31190: Natural Language Processingkgimpel/teaching/31190/... · 2016. 2. 2. · • predicting novel success from text of novels: 8. Project Proposal • due Feb. 16 (in two](https://reader033.vdocuments.us/reader033/viewer/2022060816/6094bc9a25c09210f1468b34/html5/thumbnails/47.jpg)