nlpainter “text analysis for picture/movie generation”
DESCRIPTION
NLPainter “Text Analysis for picture/movie generation”. David Leoni Eduardo C á rdenas 11 /01/2012. Motivation for choosing the project:. The purpose of our project is to transform text in images trying that both express the same mining. More than 50% of human brain is devoted to vision - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: NLPainter “Text Analysis for picture/movie generation”](https://reader035.vdocuments.us/reader035/viewer/2022081604/568165d2550346895dd8e003/html5/thumbnails/1.jpg)
NLPAINTER
“TEXT ANALYSIS FOR PICTURE/MOVIE GENERATION”
David LeoniEduardo Cárdenas
11/01/2012
![Page 2: NLPainter “Text Analysis for picture/movie generation”](https://reader035.vdocuments.us/reader035/viewer/2022081604/568165d2550346895dd8e003/html5/thumbnails/2.jpg)
2
MOTIVATION FOR CHOOSING THE PROJECT:
The purpose of our project is to transform text in images trying that both express the same mining.
More than 50% of human brain is devoted to vision
• A fact mere text can't exploit, no matter how inspired and well written it is. • Adding illustrations to text can be of great help to memorize its contents• But searching images that represent the text is a time consuming task• Drawing entirely new images from scratch takes even longer.
![Page 3: NLPainter “Text Analysis for picture/movie generation”](https://reader035.vdocuments.us/reader035/viewer/2022081604/568165d2550346895dd8e003/html5/thumbnails/3.jpg)
3
HOW THE PROBLEM CAN BE SOLVE?
In order to solve this problem we are going to use different techniques like text mining, natural language processing and semantic web:
We obtained a big Image database.
We have image with tags with the things that are inside of them.
We selected the most representative picture in our database that describes a specific object.
![Page 4: NLPainter “Text Analysis for picture/movie generation”](https://reader035.vdocuments.us/reader035/viewer/2022081604/568165d2550346895dd8e003/html5/thumbnails/4.jpg)
4
HOW THE PROBLEM CAN BE SOLVE?
We used some text mining techniques in order to obtain entities, attributes, etc.
We used the PoS of the phrase that we want to convert to image.
We associated the text with the images.
![Page 5: NLPainter “Text Analysis for picture/movie generation”](https://reader035.vdocuments.us/reader035/viewer/2022081604/568165d2550346895dd8e003/html5/thumbnails/5.jpg)
5
WHY AN ONTOLOGY?
We need an ontology to hold implicit knowledge which might not be present in the text provided by the user
Examples:
• an animal is in the tundra
• We don’t have pictures tags that tell us animal, so we use an ontology in order to know that a lion is an animal.
• Which are the usual animals living in the tundra?
![Page 6: NLPainter “Text Analysis for picture/movie generation”](https://reader035.vdocuments.us/reader035/viewer/2022081604/568165d2550346895dd8e003/html5/thumbnails/6.jpg)
6
DATABASES
The following databases of images was used for our project:
LabelMe images are annotated with the shapes of the objects contained in the scene. labeling was done by unpaid users More than 70,000 shapes where obtained!
Animal Diversity Web •we fetched nearly 10000 pages.•1545 were information about animals.•3500 picture pages of animals (and for each picture page we extracted ~5 pics links) and 5000 were simply the pages about the hierarchy, needed to arrive to the information at the leaves•we fetched mammals,reptiles ,birds, bony fishes, insects, echinoderms, arthropods
![Page 7: NLPainter “Text Analysis for picture/movie generation”](https://reader035.vdocuments.us/reader035/viewer/2022081604/568165d2550346895dd8e003/html5/thumbnails/7.jpg)
7
LabelMe
DATABASES
![Page 8: NLPainter “Text Analysis for picture/movie generation”](https://reader035.vdocuments.us/reader035/viewer/2022081604/568165d2550346895dd8e003/html5/thumbnails/8.jpg)
8
Animal Diversity
DATABASES
![Page 9: NLPainter “Text Analysis for picture/movie generation”](https://reader035.vdocuments.us/reader035/viewer/2022081604/568165d2550346895dd8e003/html5/thumbnails/9.jpg)
9
The Story Picturing EngineA Text-to-Picture Synthesis System for Augmenting CommunicationWordsEye
State of the Art
![Page 10: NLPainter “Text Analysis for picture/movie generation”](https://reader035.vdocuments.us/reader035/viewer/2022081604/568165d2550346895dd8e003/html5/thumbnails/10.jpg)
10
PART OF SPEECH
STANFORD
This software is a Java implementation of the log-linear part-of-speech taggers. The English taggers use the Penn Treebank tag set. Have been improved its speed, performance, usability, and support for other languages.
![Page 11: NLPainter “Text Analysis for picture/movie generation”](https://reader035.vdocuments.us/reader035/viewer/2022081604/568165d2550346895dd8e003/html5/thumbnails/11.jpg)
11
STANFORD PROJECT
![Page 12: NLPainter “Text Analysis for picture/movie generation”](https://reader035.vdocuments.us/reader035/viewer/2022081604/568165d2550346895dd8e003/html5/thumbnails/12.jpg)
12
Ontology:
![Page 13: NLPainter “Text Analysis for picture/movie generation”](https://reader035.vdocuments.us/reader035/viewer/2022081604/568165d2550346895dd8e003/html5/thumbnails/13.jpg)
13
Ontology:
![Page 14: NLPainter “Text Analysis for picture/movie generation”](https://reader035.vdocuments.us/reader035/viewer/2022081604/568165d2550346895dd8e003/html5/thumbnails/14.jpg)
14
Ontology:
![Page 15: NLPainter “Text Analysis for picture/movie generation”](https://reader035.vdocuments.us/reader035/viewer/2022081604/568165d2550346895dd8e003/html5/thumbnails/15.jpg)
15
Ontology:
![Page 16: NLPainter “Text Analysis for picture/movie generation”](https://reader035.vdocuments.us/reader035/viewer/2022081604/568165d2550346895dd8e003/html5/thumbnails/16.jpg)
16
General Diagram:
Domain(Animals)
Obtain andprocess
Text
Obtain and processpictures
Create Ontology and data
Query according to
the text
Image representing
the textSemantic
WebImage
processing
![Page 17: NLPainter “Text Analysis for picture/movie generation”](https://reader035.vdocuments.us/reader035/viewer/2022081604/568165d2550346895dd8e003/html5/thumbnails/17.jpg)
17
Obtain the text Parse the text (NLP)
EntitiesAttributes
PathActions
Text ready for be converted to image
Specific Diagram (Text):
17
![Page 18: NLPainter “Text Analysis for picture/movie generation”](https://reader035.vdocuments.us/reader035/viewer/2022081604/568165d2550346895dd8e003/html5/thumbnails/18.jpg)
18
Pictures in
Internet
Pictures in Databases (LabelMe Matlab)
XML to RDF Retrieve Image
Image processing
Specific Diagram (Images):
18
![Page 19: NLPainter “Text Analysis for picture/movie generation”](https://reader035.vdocuments.us/reader035/viewer/2022081604/568165d2550346895dd8e003/html5/thumbnails/19.jpg)
19
Domain(Animals)
Create the ontology
Extract linked data into a
semantic repository
Semantic WebApplication
SPECIFIC DIAGRAM (IMAGES):
19
![Page 20: NLPainter “Text Analysis for picture/movie generation”](https://reader035.vdocuments.us/reader035/viewer/2022081604/568165d2550346895dd8e003/html5/thumbnails/20.jpg)
TECHNOLOGIES AND ALGORITHMS(TEXT)
20
Programmation Language: Java
Programming Environment:Netbeans
Packages:Stanford Parser
Additional Packages: Image Generator
20
![Page 21: NLPainter “Text Analysis for picture/movie generation”](https://reader035.vdocuments.us/reader035/viewer/2022081604/568165d2550346895dd8e003/html5/thumbnails/21.jpg)
TECHNOLOGIES AND ALGORITHMS(IMAGE)
21
Programmation Language:MATLAB, Java
Programming Environment:Netbeans
Packages:LabelMeXOM Image Processing (Crop, class, merge class, etc. )
21
![Page 22: NLPainter “Text Analysis for picture/movie generation”](https://reader035.vdocuments.us/reader035/viewer/2022081604/568165d2550346895dd8e003/html5/thumbnails/22.jpg)
TECHNOLOGIES AND ALGORITHMS(ONTOLOGY)
22
Editor:Protégé 4.1
RDF engine: OWLim Lite
Upper ontology: Wordnet
22
![Page 23: NLPainter “Text Analysis for picture/movie generation”](https://reader035.vdocuments.us/reader035/viewer/2022081604/568165d2550346895dd8e003/html5/thumbnails/23.jpg)
TECHNOLOGIES AND ALGORITHMS(GENERAL PROJECT)
23
Programmation Language: Java
Programming Environment:NetBeans
RDF engine:OWLIM lite
Packages:XOM Image Processing (awt)
Web server:Apache Tomcat 7.0 JSP
23
![Page 24: NLPainter “Text Analysis for picture/movie generation”](https://reader035.vdocuments.us/reader035/viewer/2022081604/568165d2550346895dd8e003/html5/thumbnails/24.jpg)
TECHNOLOGIES AND ALGORITHMS(GENERAL PROJECT)
24
Documentation: Google Wiki
Versioning:SVN
Project Web Page:http://code.google.com/p/nlpainter/
24
![Page 27: NLPainter “Text Analysis for picture/movie generation”](https://reader035.vdocuments.us/reader035/viewer/2022081604/568165d2550346895dd8e003/html5/thumbnails/27.jpg)
27
LET SE IT WORKS!!!
![Page 28: NLPainter “Text Analysis for picture/movie generation”](https://reader035.vdocuments.us/reader035/viewer/2022081604/568165d2550346895dd8e003/html5/thumbnails/28.jpg)
28
QUESTIONS?
![Page 29: NLPainter “Text Analysis for picture/movie generation”](https://reader035.vdocuments.us/reader035/viewer/2022081604/568165d2550346895dd8e003/html5/thumbnails/29.jpg)
29
REFERENCES: [LM] Bryan C. Russell and Antonio Torralba and Kevin P. Murphy
and William T. Freeman}, Labelme: A database and web-based tool for image annotation, MIT AI Lab Memo, 2005
[DBP] Christian Bizer, Jens Lehmann, Georgi Kobilarov, Sören Auer, Christian Becker, Richard Cyganiak, Sebastian Hellmann: DBpedia – A Crystallization Point for the We of Data. Journal of Web Semantics: Science, Services and Agents on the World Wide Web, Issue 7, Pages 154–165, 2009.
[TRA] Mihalcea, R., and Tarau, P. 2004. TextRank: Bringing order into texts. In Proc. Conf. Empirical Methods in Natural Language Processing, 404–411
[CAPS] Ken Xu and James Stewart and Eugene Fiume , Constraint-Based Automatic Placement for Scene Composition, Proc. Graphics Interface, 2002,May, Calgary, Alberta, pp 25--34[ADW] Myers, P., R. Espinosa, C. S. Parr, T. Jones, G. S. Hammond, and T. A. Dewey. 2006. The Animal Diversity Web (online). Accessed November 01, 2011 at http://animaldiversity.org