1 stock trend prediction using neural networks seminar presentation marco witzmann (11v101009) ravi...

1

Stock Trend Prediction using Neural Networks

Seminar PresentationMarco Witzmann (11V101009)Ravi Vishvakarma (09005075)Rishabh Singhal (09005005)

SupervisorProf. Pushpak Bhattacharyya

2

Outline• Introduction

• Price Fluctuation• Classical Methods

• Fundamental Analysis• Technical Analysis

• Multiple linear regression• Limits to classical Methods

• Neural networks• Implementation• Feed Forward with Back Propagation of errors

• Initialization• Random initialization• Multiple linear regression initialization

• Algorithm• Steepest descent• Conjugate gradient

• Experimental comparison• Recurrent networks• Network comparison

• Limits of Neural Networks• Conclusion

3

Introduction• What is Share?• Smallest unit of ownership of a company• Is Traded in Stock Markets

• What is Stock?• Sum of all shares of a company

• Price of a Share• Depends on supply and demand in the market

4

Price Fluctuation• Supply ≈ constant• Demand depends on• Company’s performance

• CEO change• Profits & Losses• Market developments in company’s sector

• Stock Market Speculations• Financial bubbles• Investor Psychology

• Unforeseeable External factors• Climate• Govt. Policies

5










6

Fundamental Analysis• In-depth analysis of company’s performance• Market participation• Competition• Expansion Strategy for futureDetermine expected returns/Profits

• Systematic approach• Good for long term investments

• Assumption : Stock price depends on actual company value• Short term predictions impossible

7

Technical Analysis• Analysis of stock charts• Used by 90% of major stock traders• Not taking into account company fundamentals• Subjective pattern recognition and trend analysisExtrapolating data to predict

• Can be automated• Simple trading rules can be formulated

• Assumption : History repeats itself• Exaggerates price indicators

8

Linear regression• Simple linear regression:

x

y

9

Multiple linear regression• Simple linear regression: • 1 scalar variable (y)• 1 explanatory variable (x)

• Multiple linear regression:• 1 scalar variable (y)• n explanatory variables (x1, x2,…,xn)• • Modelling of non-linear inputs possible by chosing e.g. x1=t2

10

Problems in Multiple Linear Regression• Many assumptions:• linearity of inputs• homoskedasticity (constant variance of errors) not true for :

• company value: 1,000,000$ ±200.000$ is possible variance • company value: 100,000$ ±200.000$ is implausable variance

• Independence of errors

• Anscombe Quartett:

11

Limits to Classical Methods• Rely on simplistic models• Fundamental Analysis: ignores market psychology As traders base decision on psychology it won’t work• Technical Analysis: ignores information outside stock market Man-made rules must be simplistic Multiple linear regression depends on assumptions as power of

inputs to be accurate

Need for system with a more holistic approach because• Mathematical models can’t be formulated• Should be able to learn from past and be optimized

12










13

Neural Networks• Mathematical models can’t be formulated• Stock Market

• Highly non-linear and dynamic system• Too many influencing parameters with different weightage

NN don’t need on Mathematical models

• Should be able to learn from past and be optimized NN can be trained and tested for accurate prediction with past data

14

Implementation

1. Subjectively deciding on NN inputs• Fundamental: Volume, Price, etc.• Technical: Moving Averages, Volume trends, etc.• Indices – NSE FMCG, gold metals, etc.• Exchange rates and Interest rates• Economic statistics: GDP, Export and Imports, etc.• Psychological inputs

2. Designing a model • Feed Forward(FF) v/s Recurrent Neural Network (RNN)• No. of hidden layers• Initial weightages• Connections between different nodes

15

Implementation (II)

3. Training the model• Algorithm

• FF Network with back propagation of errors• Prevent over-training

4. Testing • Validating prediction on existing data• Optimizing model

• Adding more inputs to get better accuracy• Removing non-correlative inputs

16










17

Feed Forward with Back Propagation• Information always moves towards forward direction• Error is measured in forward pass through network• Error is propagated backwards and weights are adjusted to

reduce overall error• Risk of being trapped in local minimum• Initialization of weights also becomes an important issue• Two methods to initialize weights will be discussed :-• Random Weight Initialization• Multiple Linear Regression

18

Random Initialization• Weights in NN is initialized to random values• Starting point for all initialization methods

• Minimum amount of time is spent• Leads to different result in each session of training of network• Most popular method due to its simplicity

• Doesn’t guarantee a good starting point• Initializes weight near saddle point of error surface and local

minima• Can be the cause of slow learning rate

19

Multiple linear Regression Initialization• Weights between input layer and hidden layer initialized

randomly• Weights between hidden layer and output layer is obtained by

Multiple linear Regression Initialization

: output unknown weights : input : input weigh

20

Multiple linear Regression Initialization• Equation derived above is a typical linear regression model• ‘s are regressors and ’s can be estimated by standard

regression method

• Weights are initialized near the global optimum• May results in faster convergence rate

• Improvement is not worth time spent in coding and initialization of start state for most cases

21

Steepest Descent• First order steepest descent technique• Weights are modified in a direction of negative gradient of the

error surface• ZigZag search motion• Sensitive to parameters like learning rate, etc.

• Simplicity • For most cases, it gives acceptable results

• ZigZag search motion may spoil good point • Slow Convergence

22

Conjugate Gradient• Algorithm

1. Set k = 1. Initialize .2. Compute .3. Set .

4. Compute by line search, where = arg min .5. Update weight vector by .

6. If network error is less than a pre-set minimum value or the maximum number of iterations has been reached, stop; else go to step 7.

7. If k+1 > n, then = , k = 1 and go to step 2; Else

a) set k= k+1b) compute . c) compute .d) compute new direction: .e) go to step 4

23

Conjugate Gradient• For calculating gradient in step 2 and 7b, objective function f is

defined as

: actual output : Expected outputN: number of training set

• Gradient is

• For output nodes,

Where is the derivative of the activation function

24

Conjugate Gradient• For the hidden node

• Converges faster• Othogonal search motion doesn‘t spoil good points• Less number of iterations are required

• Requires more time per iteration• Implementation is not simple

25

Experimental Comparison• Daily trading data of 11 companies in 1994-1996 was collected

from Shanghai Stock Exchange for technical Analysis• First 500 entries were used as training data• Rest 150 were testing data• Raw data was processed into various technical indicators• Ten technical indicators were selected as input to the neural

Network. Some of them are :-• Relative strength index on day t-1 (RSI(t-1))• Moving average convergence-divergence on day t-1 (MACD(t-1))• MACD signal line on day t-1 (MACD Signal Line (t-1))• Lagging input of past 5 days‘ change in exponential moving

average( , etc.• Stochastic %D and %K on day t-1 (%K(t-1) and %D(t-1))

26

Experimental Comparison• EMA gives an average value of data with greater weight to the

latest data• 12-day EMA and 26-day EMA are used

• RSI is an oscillator which measures strength of up versus down over a certain time interval• Higher value of RSI means stronger market and vice versa

• MACD is the difference between two moving average of price• MACD signal line smoothens MACD• Stochastic is an oscillator that tracks the relationship of each

closing price to the recent high-low range. It has 2 lines :• %K is the raw Stochastic• %D smoothens %K

27

Experimental Comparison• Indicator were normalized as NN can’t handle wide range of

values• Prediction of price change allow greater error tolerance than

prediction of exact price• A three layered network architecture was used. The required

number of hidden nodes is estimated byNo of hidden nodes = (M + N) / 2

where M and N is the number of input and output nodes respectively

28

Experimental Comparison• Following scenario were examined :-• Conjugate gradient with random intialization(CG/RI)• Conjugate gradient with multiple linear regression initialization

(CG/MLRI)• Steepest descent with random initialization(SD/RI)• Steepest descent with multiple linear regression initialization

• In steepest descent algorithm• Learning rate was set 0.1

• Training is terminated when mean square error (MSE) < 0.5%

29

Experimental Comparison

30

Experimental Comparison

31

Experimental Comparison• In figures above and actual have relative greater deviation in

some regions• After transformation to exact price value, deviation between

actual price value and predicted price is small• Selection of input was appropriate

Average no of Iteration

Average MSE % of correct direction prediction

CG / RI 56.636 0.001753 73.055

CG / MLRI 30.273 0.001768 73.545

SD / RI 497.818 0.001797 72.564

SD / MLRI 729.367 0.02580 69.303

32

Experimental Comparison• Conclusions• All scenarios except SD / MLRI achieve similar average MSE and

percentage of correct predictions• All scenario perform satisfactory• Conjugate learning requires significant less amount of iterations

than steepest descent algorithm• MLR initialization requires less number of iteration in CG

algorithm. This shows regression provides better starting point.• In case of SD, MLRI requires more iteration because good starting

point is spoiled by zigzag motion of Steepest descent algorithm• Efficiency of classical back propagation can be improved by

conjugate learning with multiple linear regression weight initialization

33

Recurrent Neural Networks• Connection between perceptron form a directed cycle• They have feed-back mechanism which allows it to exhibit dynamic temporal behaviour• Takes into account the sequence in which the input is presented. It can predict price according to recent history more closely• Most RNNs have had scaling issues. In particular, RNNs cannot

be easily trained for large numbers of neuron units nor for large numbers of inputs units. Successful training has been mostly in time series problems with few inputs.

34

Network Comparison• Main difference between FFN and RNN is existance of feed-

back mechanism in RNN • RNN can‘t be trained for larger no of inputs/neurons whereas

FFN can be trained for both the cases• RNN learn spatiotemporal information from training data

whereas FFN learns spatial relationship only

35

Limits of Neural Networks• Highly Subjective choice of inputs and model• Computational power• Less accessible• Costly• Black box• Cannot handle wide range of values• If implemented on large scale it create self-fulfilling prediction

36

Conclusion• Share Prices are predictable under reasonable constraints• Accuracy of Prediction increases from Fundamental Analysis

(FA) to Technical Analysis(TA) to Neural Network based prediction

FA < TA < NN• NN takes into account much more information than classical

methods• Use of Feed Forward Network is better than Recurrent Neural

Network due to higher difficulty in training of RNN• Feed Forward network with Back propagation trained using

Conjugate Gradient method and weights initialized using Multiple linear regression gives better accuracy than classical back propagation model

37

References• Ramon Lawrence. Using Neural Networks to Forecast Stock Market Prices. University of

Manitoba, 1997.• Buisnessweek.com. A “Neural” Approach to the Market. Bloomberg BusinessWeek• Jason E. Kutsurelis. Forecasting Financial Market Using Neural Networks: An Analysis of

Methods and Accuracy. Naval Postgraduate School, Monterey, California, 1998.• Charles Duhigg. Artificial Intelligence applied heavily to picking stocks – business –

International Herald Tribune. Nytimes.com, 2006.• CHAN Man-Chung, WONG Chi-Cheong, LAM Chi-Chung. Financial Time Series Forecasting

by Neural Network Using Conjugate Gradient Learning Algorithm and Multiple Linear Regression Weight Initialization. Department of Computing, The Hong Kong Polytechnic University Kowloon, Hong Kong, 1996

• Jovina Roman, Akhta Jameel. Backpropagation and Recurrent Neural Netwrok in Financial Analysis Of Multiple Stock Market Returns. Department of Computer Science, Xavier University of Louisiana, 1996

• en.wikipedia.org/wiki/Recurrent_neural_network• en.wikipedia.org/wiki/Neural_network

38

Thank you!

1 stock trend prediction using neural networks seminar presentation marco witzmann (11v101009) ravi...

Documents