n. s. reddy -...

N. S. Reddy

School of Materials Science and EngineeringDepartment of Metallurgical & Materials Engineering

Gyeongsang National University, Jinju, Gyeongnam 660-701, Korea

[email protected]

Artificial Neural Networks in Materials Science

Summer School for Integrated Computational Materials Education, Ann Arbor, June 5-16, 2017

NSR

Charpy

Fatigue

Corrosion

Tensile

Critical stress intensity Why

Axioms

• All properties can be measured.• Measurements can be used in safe design.• Measurements can be used in control.

Conclusions• There are useful ways of expressing properties• Limited models relating properties to microstructure

• No method for predicting properties in general

WhySummer School for Integrated Computational Materials Education, Ann Arbor, June 5-16, 2017

Source: http://www.msm.cam.ac.uk/phase-trans/abstracts/neural.review.html

Solidification conditions

Mechanical Treatment

Alloy contents: C, Si, Mn, S, P, Ni, Cr, Mo….

Heat treatment

Physical and Mechanical Properties

Materials

Complex Materials System

Microstructure of

Why

Pickering linear equations (1978)

σY = 53.9 + 32.3WMn + 83.2Wsi + 354.2(W Nf) 0.5 + 17.4(dα)-0.5

σU = 294.1 + 27.7WMn + 83.2Wsi + 3.85(%pearlite) + 7.7(dα)-0.5

σY is predicted yield strength in MPa and σU is predicted ultimate tensile strength in MPa, WMn, WSi and WNf are the contents of manganese, silicon and free nitrogen in weight percent respectively, and dα is the ferrite grain size in millimeters.

Source: http://www.msm.cam.ac.uk/phase-trans/2006/NNGA.html

Contents Artificial Neural Networks (ANN)• What• Why• When

Materials Science Problems• Grain refinement in Al-7Si alloy (3 1)• Phase volume fraction in Ti – 6Al – 4V alloy (6 2)• Mechanical Properties in Steels (10 5)

• How• Where

Summer School for Integrated Computational Materials Education, Ann Arbor, June 5-16, 2017

A Brief history• Early stages

– 1943 McCulloch-Pitts: Neuron as computing element – 1949 Hebb: Learning rule– 1958 Rosenblatt: Perceptron– 1960 Widrow-Hoff: Least mean square algorithm

• Recession– 1969 Minsky-Papert: Limitations perceptron model

• Revival– 1982 Hopfield: Recurrent network model– 1982 Kohonen: Self-organizing maps– 1986 Rumelhart et. al.: Backpropagation

When

Artificial Neural Networks• 65 journals

• 413 Conferences and workshops are presently dedicated

exclusively to artificial neural networks

• About 67138 articles are being published on the current trends

in artificial neural networks.

• USA, India, Japan, Brazil and Canada are some of the leading

countries where maximum studies are being carried out.

• Artificial Neural Networks (ANN) are computational models

inspired by an animal's central nervous systems

http://www.omicsonline.org/artificial‐neural‐network‐journals‐conferences‐list.php

A few Neural Networks Journals1. Neural Networks

2. Advances in Artificial Neural Systems

3. Applied Intelligence

4. IEEE Transactions on Neural Networks

5. Journal of Artificial Intelligence and Soft Computing

6. Neural Computing and Applications

7. International Journal of Neural Networks

8. Neural processing Letters

9. Applied Soft computing

IMPORTANCE OF ANN in Materials Modeling

0 0 ' 0 1 ' 0 2 ' 0 3 ' 0 4 ' 0 5 ' 0 6 ' 0 7 ' 0 8 ' 0 9 '0

1 0 0 0

2 0 0 0

3 0 0 0

4 0 0 0

5 0 0 0

6 0 0 0

7 0 0 06 3 6 1

5 0 2 14 4 6 3

3 8 9 1

3 2 7 02 8 8 6

2 6 5 0

1 9 3 11 5 9 1AN

N P

ublic

atio

ns

Y e a r

1 4 5 0

Number of ANN papers published in Materials Science Journals from Jan 2001 – Dec 2009. (Elsevier publishers only)

Keyword Search criteria: Neural networks and materials science in allfields. www.sciencedirect.com Why

Where are ANN used?• Recognizing and matching complicated,

vague, or incomplete patterns• Data is unreliable• Problems with noisy data

– Prediction– Classification– Data association– Data conceptualization– Filtering– Planning– **** …

Where

What’s a neural network?

What

Ariana Grande, US Singer, Actress, Manchester


What


Ravindra Jadeja and BoltWhat

Sherif Ismail, Prime Minister of Egypt


What

Yao Ming, Basket Ball Player, China


What

Saudi King Salman bin Abdulaziz Al Saud


What

Son ga-in


Shirin Ebadi, First Iranian woman , Nobel peace prize in 2003


What

Aziz Sancar, Nobel Laureate in Chemistry 2015


What


What


This is a dog This is a dog

This is a cat

This is a dog

This is a cat

This is a cat

This is a dog

What is this?

What

Artificial Neural networks (ANN) are modeling system, which mimics the human brain.

• Knowledge is acquired by the process of learning

• Storing the knowledge (like brain).• Generalizations capability of the situation

based on the acquired Knowledge

Neural Networks: Lessons from human brain

What

Simple Artificial Neuron

Inputvalues

weights

Summingfunction

Biasb

Activationfunction

u Outputy

x1

x2

xm

w2

wm

w1

)(u

)e 1/(1 y

xwu

u-

m

1jj

bj

Linear, Hyperbolic, Sigmoid

weighted sum

Inputvector x

weightvector w

The n-dimensional input vector x is mapped into variable y by means of the scalar product and a nonlinear function mapping What

Feed Forward Neural Networks (FFNN)• Neurons are arranged in layers.• Each unit is linked only in the unit in next layer, no

units are linked between the same layer, back to theprevious layer or skipping a layer.

• Computations can proceed uniformly from input tooutput units.

Inputlayer

Outputlayer

Hidden Layer

ParametersLearning RateMomentum TermHidden LayersHidden NeuronsIterations

Information

Error Propagation What

Updating Weights

ij

k

Wji

Wkj

WkjWji

)1()()()1( tjiWtjiWiOjtjiWtjiW

)1()()()1( tjiWtjiWiOjtjiWtjiW

)()( kkkk netfOT )()( kkkk netfOT

k

kjkjj Wnetf )( k

kjkjj Wnetf )(

Output Layer

Hidden Layer

Learning Rate

Momentum Rate

Xi

Ok Function SignalsError Signals

ANN Model Flowchart

Stop

Select Input and Outputs best represent the model

Y

Define Range & Distribution of Data

Data preprocessing

Neural Network Structure SelectionTraining Algorithm

Perform Training

Is Validation good Accuracy

Test the quality of neural ModelYes

No

How

yd is the desired response, yo is the output response from the ANN, and p is the of patterns presented

2

1

1 [ ]p

d oj j

jMSE y y

p

Criteria for model selection

where Etr(y) = average error in prediction of training andtesting data set for output parameter y, N = number ofdata sets, Ti(y) = targeted output, Oi(y) = outputcalculated.

N.S. Reddy et al., “Computational Materials Science”, Vol. 107, 2015

The sensitivity curve & the instantaneous sensitivity

N.S. Reddy et al., “Materials Science and Engineering A”, Vol. 434, 2006

Objectives

• To investigate the suitability of neural networks tocomplex Materials Systems.

• To predict properties/microstructure at newinstances.

• To examine the Effect of Individual Elements onoutput parameters keeping other elements unaltered.

• To design the Optimum Alloy for desired properties.

• To validate the model predictions with experiments

Why

1. Grain refinement in Al-7Si alloy(3 1)

2. Microstrucure in Titanium alloys(6 2)

3. Mechanical Properties in Steels(10 5)

Example.1: Grain Refinement of Al-7Si Alloy Importance of Al-Si alloys

Need of grain refinement

How to achieve grain refinement

Master alloys for Grain refinement of Al –7Si alloyBinary Master Alloys

• Al – 3B • Al – 3Ti

Ternary Master Alloys•Al – 1Ti – 3B•Al – 3Ti – 1B•Al – 3Ti – 3B•Al – 3Ti – 3B•Al – 3Ti – 3B

Where

“Al-Si alloys are frequently used in castingsdue to their combination of good physicalproperties and excellent castability”

• Light weight• Energy and Environmental

considerations• Castability, weldability• Corrosion resistive• Good at room temperatures & fairly

upto 200oC• High strength to weight ratio

Al-Si alloys are frequently used in castings due to their combination of good physical properties and excellent castability.

Problem?Coarse columnar grains

Al-5Ti-1B Master alloy

Example1. Grain refinement of Aluminum and its alloys

Further Processing

Rolling Forging Extrusion

Al with grain refiner

Al without grain refiner

Applications of Aluminium

Al

Slabs

Grain refiner

Mould Box

Aerospace Packaging Electrical

Molten

Aluminum

Molten Aluminium

Aluminium Castings

Tundish

Al-5Ti-1B Master alloy Grain refiner

(Columnar Grains)

(Fine Equiaxed grain)

Example 1. Statistics of Grain refinement dataSystem(Train + Test)Sets

Variables Minimum Maximum Mean StandardDeviation

Binary Masteralloy addition

(48 +12)

Ti (%) 0 0.10 0.021 0.03B (%) 0 0.10 0.021 0.03Time (min)

0 120.00 40.69 43.86

GS (m) 98 610.00 276.08 158.16

Ternary Masteralloy addition

(120 + 30)

Ti (%) 0 0.10 0.029 0.028B (%) 0 0.10 0.029 0.028Time(min)

0 120 42.27 43.74

GS (m) 68 610.00 186.74 116.24

Total data(binary +Ternary alloy)

(168 +42)

Ti (%) 0 0.10 0.023 0.029B (%) 0 0.10 0.022 0.028Time(min)

0 120.00 36.18 43.06

GS (m) 68 610.00 266.54 185.55

N.S. Reddy et al., “Materials Science and Engineering A”, Vol. 391, 2005 Where

Validation of ANN model predictions with Al-1Ti-3B alloy

0 2 30

60 120 150

5

(0.1% )


0 30 60 90 120 150400

450

500

550

600

650

700

(0.1% M13)

Exp.Model 2Model 3%E

Holding Time (min)

Gra

in S

ize

(m

)

0

1

2

3

4

5

6

%E

(Mod

el 3

)

Where

(B=0.003 & T=0.001)

2 5 30

15012060

0

Validation of ANN model predictions with Al-1Ti-3B alloy1.67%


0 30 60 90 120 1506065707580859095

100(1.67% M13) Exp.

Model 2Model 3%E

Holding Time (min)

Gra

in S

ize

(m

)

0

2

4

6

8

%E

(Mod

el 3

)

Where

(B=0.05 & T=0.0167)

2 5 30

15012060

0

Predicted Grain refinement map of Al-7Si alloy

0.00 0.05 0.10 0.15 0.20 0.25 0.300.00

0.05

0.10

0.15

0.20

0.25

0.30

Al3Ti + TiB2

Ideal

grain

refin

er

(Ti:B

=1:1.

67)

Al3Ti

Al3Ti + TiB2AlB2

M51

M31

M13

M15

% Titanium

% B

oron 0

80.00160.0240.0320.0400.0480.0560.0640.0700.0

N.S. Reddy et al., Journal of Materials Performance, Vol 22, 2013

Color band numbers indicates grain size in m

Master AlloysAl – 1Ti – 3B (M13)Al – 3Ti – 1B (M31)Al – 3Ti – 3B (M33)Al – 1Ti – 5B (M15)Al – 5Ti – 1B (M51)

WhereIdeal Grain Refiner

Validation of Predictions with Experiments

0.000 0.005 0.010 0.0150.00

0.01

0.02

0.03

0.04

0.05(74m)1.67% M13

(654m) 0.1% M13

0.3% M03

1.67% M03(96m)

(241m)

M51M31

080.00160.0240.0320.0400.0480.0560.0

% Titanium

% B

oron

M13M15

WhereN.S. Reddy et al. Journal of Materials Engineering and Performance, 696,Vol 22(3), March 2013, 696-699

M2A3 BradleyM2A3 Bradley TomahawkTomahawk

Few Applications of Titanium alloyFew Applications of Titanium alloy

Golf-headGolf-head

F-22 RaptorF-22 Raptor Guggenheim MuseumGuggenheim MuseumBiomaterialBiomaterial

Ti-6Al-4V alloy as a Biomaterial

Where

Alloy contents: Al, V, Fe,

O & N

Heat treatment

Volume fraction of α-β phases

Ti ALLOY

(a)

Beta Volume Fraction is 29%

(b)


(c)


(d)


Ti-6.19Al-4.05V-0.19Fe-0.12O-0.01N

[ 750 oC ] [ 900 oC ]

Ti-6.3Al-4.1V-0.21Fe-0.168O-0.005N

Example 2. : Phase volume fraction in Ti-6Al-4V alloy

Where

Statistics of data used for modeling (Ti Alloys)

Test Samples Variables Minimum Maximum Mean

Al (%) 5.72 7 6.245V (%) 1.5 5 3.939Fe (%) 0.01 3.04 0.462O (%) 0.08 0.3 0.149N (%) 0.003 0.02 0.007Temperature ( oC) 600 1000 865.75

104 Training + 15 validation + 9 test data sets

Vol. α and β 0 100 47.79


Performance of ANN Model

0 20 40 60 80 1000

20

40

60

80

100

0 20 40 60 80 1000

20

40

60

80

100

0 20 40 60 80 1000

20

40

60

80

100

0 20 40 60 80 1000

20

40

60

80

100

Predicted vs Experimental volume fraction of phase

(a)All data

R = 0.9982

Data Points Best Linear Fit

Expe

rimen

tal

Predicted

(b)Training

R = 0.9997

Data Points Best Linear Fit

Expe

rimen

tal

Predicted

(c)Testing

R = 0.9965

Data Points Best linear Fit

Expe

rimen

tal

Predicted

(d) Data Points Best Linear Fit

Validation

R = 0.9802Expe

rimen

tal

Predicted

WhereN.S. Reddy et al., “Materials Science and Engineering A”, Vol. 434, 2006

Comparison of experiment & calculation in Ti–6.3Al–4.1V–0.21Fe–0.17–0.005N alloy quenched at (a) 700 C, (b) 815 C, (c) 900 C, and (d) in Ti–6.85Al–1.6V– 0.13Fe–0.17–0.001N quenched at 900 C.


81/81.8

46/50.5

70/67.5

91/90.8

Unexpected Trend: Uncertainty

600 700 800 900 1000 11000

102030405060708090

100 (h)( tr = 933oC)

5.88Al-4.35V-3.04Fe-0.084O-0.004N Volu

me

fract

ion

of

pha

se

Tem perature oC


Effect of alloying elements on phase volume fraction

Volume Fraction of Beta map of Ti-Al- V at different temperatures

Calculating Index of Relative Importance


Ti–6.34Al–4V–0.19Fe–0.17O–0.01N at 980 C.

Experimental value for the alpha phase of the alloy 10.8% and predicted value is 11.8%

Validation of ANN model predictions

9 0 0

9 2 5

9 5 0

9 7 5

1 0 0 0

1 0 2 5

1 0 5 0

1 0 7 5

T i- 7 A l- 1 .5 V T i- 6 .8 5 A l- 1 .6 V T i- 6 .1 9 A l- 4 .0 5 V T i- 6 .3 3 A l- 4 V

Tr

ansu

s Te

mpe

ratu

re E x p e r im e n ta l P re d ic te d

Comparison of predicted and measured beta-transus temperatures for single-phase-alpha, near-alpha, and alpha/beta titanium alloys.


Thanks to Dr. Jeoung Han Kim for conducting experiments in POSCO

Where

Example 3.Low alloy steels data examples

Where

The range of low alloy steels (EN100 steels)

YS: Yield strength, UTS: Ultimate tensile strength, %El: % Elongation, %RA: %Reduction in Area. Total data sets are 140 and 112 were used for training and 28used for testing.

Composition (in wt.%)

Min. Max.

C 0.32 0.44

Si 0.19 0.37

Mn 0.33 1.51 S 0.01 0.042 P 0.02 0.038 Ni 0.56 1.08 Cr 0.21 0.57 Mo 0.11 0.25

Heat treatment variables

Cooling rate (oC/S) 2.8 118 Tempering Temperature (oC)

400 700

Mechanical properties

Y.S (MPa) 542 1194 UTS (MPa) 707 1295 % El 13 29 % RA 31 67 Impact Strength (J) 15 94


Model Predictions with Test data (EN100 Steels)

1 2 3 4 5 6 7 8 9 10 11 120

200400600800

100012001400

YS

Samples

Predicted Actual

1 2 3 4 5 6 7 8 9 10 11 120.00.51.01.52.02.53.03.54.0

YS

Samples

%Error

1 2 3 4 5 6 7 8 9 10 11 12012345678

%EL

Samples

%Error

1 2 3 4 5 6 7 8 9 10 11 120

5

10

15

20

25

30

%EL

Samples

Predicted Actual

Where

Hypothetical Alloy YS UTS % EL %RA ISS5 (Actual) 760 931 19.50 49.50 20.00Carbon (0.4) 753 921 18.93 47.73 17.80Silicon (0.3) 737 906 19.78 48.90 20.80Manganese (1.51) 738 905 19.67 48.8 20.00Phosphorous (0.034) 736 901 19.89 48.95 21.94Nickel (0.89) 738 906 19.78 48.90 20.80Chromium (0.21) 737 904 19.00 48.00 11.36Molybdenum (0.16) 734 904 19.49 48.45 16.00Mn/S ratio (41.94) 739 907 19.45 48.38 18.75Cooling rate (3.1) 724 916 19.19 49.82 22.37Tempering Temp (550oC) 737 904 19.64 48.64 20.06

Comparison of properties for hypothetical alloy of a sample.


Simulated effect of heat treatment parameters

0 20 40 60 80 100 120400

450

500

550

600

650

700(YS)

Cooling Rate (oC/s)

Tem

perin

g Te

mpe

ratu

re (o C

) 500.0660.0820.0980.011401300

0 20 40 60 80 100 120400

450

500

550

600

650

700(UTS)

Cooling Rate (oC/s)

700.0840.0980.0112012601400

0 20 40 60 80 100 120400

450

500

550

600

650

700(EL)

Cooling Rate (oC/s)

Tem

perin

g Te

mpe

ratu

re (o C

) 10.0014.0018.0022.0026.0030.00

5

12

34

6

12

34

6

12

34

65 5


Disadvantages of ANN modelingO

utpu

t

Input0

Over-fitted model “Real” model

cycles

Error

Overfitted model

holdout

training

Alternative Technology Lab, Graduate Institute for Ferrous Technology (GIFT), POSTECH, Korea

Difficult to design: There are no clear design rulesHard or impossible to train: When to stop and Over trainingDifficult to evaluate internal operation: It is difficult to find out whether, and if so what tasks are performed by different parts of the net

Variation of weights with varying iterations

Hidden layers

Iterations

Momentum Term and Learning rate

Artificial Neural Networks LiteratureMain text books:

– “Neural Networks: A Comprehensive Foundation”, S. Haykin(very good -theoretical)

– “Pattern Recognition with Neural Networks”, C. Bishop (very good-more accessible)

– “Practical Neural Network Recipe's in C++”’ T. Masters(emphasizing the practical aspects)

Review Articles: – R. P. Lippman, “An introduction to Computing with Neural Nets”’ IEEE

ASP Magazine, 4-22, April 1987.– A. K. Jain, J. Mao, K. Mohuiddin, “Artificial Neural Networks: A

Tutorial”’ IEEE Computer, March 1996’ p. 31-44.– H. K. D. H. Bhadeshia, "Neural Networks in Materials Science" ISIJ

International, Vol. 39, 1999, 966-979 – H.K.D.H. Bhadeshia: Performance of Neural Networks in Materials

Science, MST

Some Internet Resources for ANN

• http://www.faqs.org/faqs/ai-faq/neural-nets/part1/preamble.html(This FAQ provides Very Useful basic information for individuals who are new to the field of neural networks. The FAQ is divided into seven parts i.e., 1.Introduction, 2. Learning, 3. Generalization, 4. Books & data etc, 5. Free software, 6. commercial software and 7. hardware)

• www.inference.phy.cam.ac.uk/mackay/Bayes_FAQ.html (Bayesian methods for neural networks – FAQ)

• http://www.brain.riken.jp/labs/mns/geczy/Links-E.html (ANN around the world)• http://www.cs.nthu.edu.tw/~jang/nfsc.htm• http://www.msm.cam.ac.uk/phase-trans/abstracts/neural.review.html (Materials Science

& NN related literature) • http://calculations.ewi.org/vjp/secure/FNCoolingRate.asp (online Ferrite Number

calculations for stainless steel based on chemical composition)

Summary (broad)Ø Neural Networks are capable to predicting complex systems.

Ø Sensitivity Analysis examine the effects of input variables onthe output parameters, which is incredibly difficult to doexperimentally.

Ø Developed model is able to map the relation between

Ø inputs and outputs and

Ø between the output parameters though this information is notfed to the model.

Ø The model helps in reducing the experiments required and thereby saving a lot of money, material and manpower for designingthe new alloys for desired properties

I would like to acknowledge• Prof. Chong Soo Lee

Department of Materials Science and Engineering, POSTECH, Korea

• Dr. Jeoung Han Kim, Dr. You Hwan Lee, and Dr. Chan Hee Park, & Dr. YeomFormer students of Prof. C S Lee, Department of Materials Science and Engineering, POSTECH, Korea

• Dr. J. Krishnaiah, BHEL, Hyderabad, India• Yandra Kiran Kumar, Software Engineer,

Minneapolis

Swami Vivekananda Quote on Newton

The mind of the man is an infinite reservoir of knowledge, and all knowledge, present, past or future, is within man, manifested or non manifested.The object of every system of education

should be to help the mind to manifest it. For instance, the law of gravitation was within

man, and the fall of the apple helped Newtonto think upon it, and bring it out from within his mind

Thank you very much for your kind attention…

n. s. reddy -...

Documents