getting started with condor - mit opencourseware started collecting web content onedegreecollector...

Getting Started With Condor

1

Contents Getting Started Collecting Web Content OneDegreeCollector Building your own Startlists Collecting your E-Mail Collecting Facebook Data Collecting Wikipedia Data Collecting CoolPeople Coolhunting Blueprin

2t

3

Getting Data into Condor IMAP E-Mail (Mailcollector) Communication View (social net)

Eudora mailboxes Term view (semantic net)

Communication View (link net) Web/Blog/News/Scholar Term view (semantic net) (WebCollector) Communication View (semantic net)

Wikipedia (WikiFactFetcher)

C

ondo

r Term view (semantic net) nippets (OneDegreeCollector)

Communication View (social net) Twitter (TwitterCollector) Term view (semantic net)

Communication View (social net) FlatFiles (FileLoader) Term view (semantic net)

PeopleNetworks (CoolPeople) Communication View (social net)

Facebook Communication View (social net) 4

S

Con

dor

Temporal Visualization by a Sliding Time Frame

1 n 2 n+1

3 n+2 4 n+3

5 n+4 time

5

6

With and without history

Preparation Install MySQL Install Java (only Windows) Install Java 3D (only Windows) Start Java (if it does not run yet)

7

Contents Getting Started Collecting Web Content OneDegreeCollector Building your own Startlists Collecting your E-Mail Collecting Facebook Data Collecting Wikipedia Data Collecting CoolPeople Coolhunting Blueprin

8t

9

Collect Web Content

10

Communication View

11

Term view index

12

Term view index - 2

Contents Getting Started Collecting Web Content OneDegreeCollector Building your own Startlists Collecting your E-Mail Collecting Facebook Data Collecting Wikipedia Data Collecting CoolPeople Coolhunting Blueprin

13t

One-Degree-Collector

Complementary to the Blog Collector Fetches only one degree Retrieved websites are not aggregate

14

One-Degree-Collector - UI

15

GUIresembles Blog Collector

One-Degree-Collector - result

typical result of one-degree search

16

Contents Getting Started Collecting Web Content OneDegreeCollector Building your own Startlists Collecting your E-Mail Collecting Facebook Data Collecting Wikipedia Data Collecting CoolPeople Coolhunting Blueprin

17t

Creating Term View Without OneDegreeCollector Start List: Create Stoplist First

18

19

… then use this stop list for the term view

20

Creating Term View With Start AND Stop List

Contents Getting Started Collecting Web Content OneDegreeCollector Building your own Startlists Collecting your E-Mail Collecting Facebook Data Collecting Wikipedia Data Collecting CoolPeople Coolhunting Blueprin

21t

Collect E-Mail java -Xmx2048M -jar condor-2.1.jar

Condor Key

MySQL password (default: no password)

22

Tools to collect data

23

For username, host, port, and ssl check with your email provider (for gmail, see next slide) provider (for gmail, see

Anonymize will replace email addresses with random identifiers

Anonymize will replace

Left side: enter here the specification of the mailbox Right side: database related data, eg

MailCollector no pass. username: root,

Content: yes will download the whole emails, w/o content only the sender, Delete the present data

Here you can choose recipients and the subject line are in the database? specific folders to downloaded dow24 nload

Delete the present data

word MailCollector

Settings for gmail

yourname@gmail.com

Your gmail password

imap.gmail.com.

Don’t forget the access information for your mysql database on the right, then press start.It might take a while (esp. with huge mailboxes) before you see a progress bar.

1 2

3

4

26

Visualize Mail-Data

7

8

9

27

Visualize E-Mail Data (3)

28

Dynamic View of Communication

1

2 3

29

Visualize E-Mail Contents

4 5

30

Visualize E-Mail Contents (2)

Dynamic View of Terms

31

Contents Getting Started Collecting Web Content OneDegreeCollector Building your own Startlists Collecting your E-Mail Collecting Facebook Data Collecting Wikipedia Data Collecting CoolPeople Coolhunting Blueprin

32t

MIT OpenCourseWarehttp://ocw.mit.edu

15.599 Workshop in IT: Collaborative Innovation NetworksFall 2011 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.