Artificial IntelligenceArtificial IntelligenceProject 1Project 1

Neural NetworksNeural Networks

Biointelligence Lab

School of Computer Sci. & Eng.

Seoul National University

(C) 2000-2002 SNU CSE BioIntelligence Lab

2

OutlineOutline

Classification Problems Task 1

Estimate several statistics on Diabetes data set

Task 2 Given unknown data set, find the performance as good as you

can get The test data is hidden.

(C) 2000-2002 SNU CSE BioIntelligence Lab

3

Network Structure (1)Network Structure (1)

…

positive

negative

fpos(x) > fneg(x),→ x is postive

(C) 2000-2002 SNU CSE BioIntelligence Lab

4

Network Structure (2)Network Structure (2)

…

f (x) > thres,→ x is postive

Medical Diagnosis: DiabetesMedical Diagnosis: Diabetes

(C) 2000-2002 SNU CSE BioIntelligence Lab

6

Pima Indian DiabetesPima Indian Diabetes

Data (768) 8 Attributes

Number of times pregnant Plasma glucose concentration in an oral glucose tolerance test Diastolic blood pressure (mm/Hg) Triceps skin fold thickness (mm) 2-hour serum insulin (mu U/ml) Body mass index (kg/m2) Diabetes pedigree function Age (year)

Positive: 500, negative: 268

(C) 2000-2002 SNU CSE BioIntelligence Lab

7

Report (1/4)Report (1/4)

Number of Epochs

(C) 2000-2002 SNU CSE BioIntelligence Lab

8

Report (2/4)Report (2/4)

Number of Hidden Units At least, 10 runs for each setting

# Hidden

Units

Train Test

Average SD

Best Worst Average SD

Best Worst

Setting 1

Setting 2

Setting 3

(C) 2000-2002 SNU CSE BioIntelligence Lab

9

Report (3/4)Report (3/4)

(C) 2000-2002 SNU CSE BioIntelligence Lab

10

Report (4/4)Report (4/4)

Normalization method you applied. Other parameters setting

Learning rates Threshold value with which you predict an example as

positive. If f(x) > thres, you can say it is postive, otherwise negative.

(C) 2000-2002 SNU CSE BioIntelligence Lab

11

Challenge (1)Challenge (1)

Unknown Data Data for you: 2000 examples Pos: 1000, Neg: 1000

Test data 600 examples Pos: 300, Neg: 300 Labels are HIDDEN!

(C) 2000-2002 SNU CSE BioIntelligence Lab

12

Challenge (2)Challenge (2)

Data Train.data : 2000 x 500 (2000 examples with 500dim) Train.labels: positive 1, negative 0 Test.data: 600 x 500 (600 examples with 500 dim) Test.labels: not given to you.

Verify your NN at http://knight.snu.ac.kr/aiproj1/ai_nn_do.asp

(C) 2000-2002 SNU CSE BioIntelligence Lab

13

Challenge (3)Challenge (3) K-fold Cross Validation

The data set is randomly divided into k subsets. One of the k subsets is used as the test set and the other

k-1 subsets are put together to form a training set.

200 200200 200 200D1 D2 D3 D8 D9

200D10

200 200200 200 200D1 D2 D3 D8 D9

200D10

200 200200 200 200D2 D3 D4 D8 D9

200D10

…

…

…

(C) 2000-2002 SNU CSE BioIntelligence Lab

14

Challenge (4)Challenge (4)

Include followings at your report The best performance you achieved. The spec of your NN when achieving the performance.

Structure of NN Learning epochs Your techniques

Other remarks…

True

PredictPositive Negative

Positive

NegativeConfusion matrix

(C) 2000-2002 SNU CSE BioIntelligence Lab

15

ReferencesReferences

Source Codes Free softwares NN libraries (C, C++, JAVA, …) MATLAB Tool box Weka

Web sites http://www.cs.waikato.ac.nz/~ml/weka/

(C) 2000-2002 SNU CSE BioIntelligence Lab

16

Pay Attention!Pay Attention!

Due (October 14, 2004): until pm 11:59 Submission

Results obtained from your experiments Compress the data Via e-mail

Report: Hardcopy!! Used software and running environments Results for many experiments with various parameter settings Analysis and explanation about the results in your own way

(C) 2000-2002 SNU CSE BioIntelligence Lab

17

Optional ExperimentsOptional Experiments

Various learning rate Number of hidden layers Different k values Output encoding