Time Series –from Achieved to

Excellence

http://www.nzchildren.co.nz/child_poverty.php

AS: Time SeriesUsing the statistical enquiry cycle to investigate time series data involves:• using existing data sets• selecting a variable to investigate• selecting and using appropriate display(s)• identifying features in the data and relating this to the context• finding an appropriate model• using the model to make a forecast• communicating findings in a conclusion.

Excellence!Investigate time series data, with justification and statistical insight involves integrating statistical and contextual knowledge throughout the statistical enquiry cycle, and may include reflecting about the process; considering other relevant variables; evaluating the adequacy of any models; or showing a deeper understanding of models.

Statistical insight involves integrating statistical and contextual

knowledge

http://www.nzchildren.co.nz/child_poverty.php

Road to excellence?You need to:

understand/relate to the context

research it properly and write with insight.

They need a structure to work to in order to organise your brains.

They need to be familiar with the language of statistics.

http://www.point8td.com/perfection-vs-excellence

Fake it until you make it

Understand and relate to context

http://www.bbc.co.uk/blogs/theoneshow/consumer/2009/03/25/truancy-should-the-kids-or-the.html

Attendance in New Zealand Schools

2012Something you are familiar with

Attendance in New Zealand Schools

2012

Note the date of the data set

Why are we interested in this investigation?

“There has been increasing community, political, and education sector concern over absence from school.”(Mallari, Loader, 2013)

http://cityview.worcesterschools.org/modules/cms/pages.phtml?sessionid=&pageid=300890

Always use referencing

BackgroundA national survey of state primary and secondary schools in New Zealand in 1977 (Taylor, Sturrock and White 1982) reported that the unjustified absence rate in primary schools was 0.69%, and in secondary schools it was 1.4%. Berwick-Emms (1987).

http://studentwork.hss.uts.edu.au/wnm08/scars/source/prischool.html

Broader context – Underlying issue - Referencing

The problem of truancy is shared throughout the world (see Reid 1987, Andrews 1986). Whitney (1994:15), a British researcher, notes that ‘Truancy, like poverty, has a lengthy past history, and the two have always been closely related.

“Chronic absenteeism is most prevalent among low income students.”

Balfanz, 2012

Data Sourcehttp://www.educationcounts.govt.nz/publications

/series/2503

Survey DetailsThe Ministry of Education survey on attendance was carried out in the week 11-15 June, 2012

The response rate was 88%

Schools recording absences on the paper form were required to make their own judgement as to whether a student was absent for all or part of a day, and whether that absence was justified based on the definitions and instructions supplied.

Are comparisons valid?The survey was carried out in the week of 11-15 June 2012, close to the middle of the second school term. This week was the same week of term as the 2009 and 2011 surveys.

By analysing data from a similar time of year, factors such as winter illness would have been at broadly similar levels.

Perspective - numbersIn 2012, approximately 62,000 students were absent from school for all or part of a day during the survey week. Of these, 15,000 students were unjustifiably absent from school.

Tables to visual

Not very good for the messages we want from the data

2004 2006 2009 20110.0

2.0

4.0

6.0

8.0

10.0

12.0

14.0

Unjustified Absence Rate (%)Intermittent Unjustified Absence Rate (%)Justified Absence Rate (%)Total Absence Rate (%)Total Unjustified Absence Rate (%)

Who would be interested and why?

2004 2006 2009 2011 20120.0

2.0

4.0

6.0

8.0

10.0

12.0

14.0

Percentage absent

Unjustified Absence Rate (%) Intermittent Unjustified Absence Rate (%)Justified Absence Rate (%) Total Absence Rate (%)Total Unjustified Absence Rate (%)

Is this a real decrease or is it pressure on schools by the Ministry to deal with

absences?

2004 2006 2009 2011 20120.0

2.0

4.0

6.0

8.0

10.0

12.0

14.0

Unjustified Absence Rate (%)Intermittent Unjustified Absence Rate (%)Justified Absence Rate (%)Total Absence Rate (%)Total Unjustified Absence Rate (%)

Main features:Time periods are not at equal intervals

Total: Between about 10% and 12%Peak: ≈12% in 2009

Drop or leveling out since 2009

2004 2006 2009 2011 20120.0

2.0

4.0

6.0

8.0

10.0

12.0

14.0

Percentage absent

Unjustified Absence Rate (%) Intermittent Unjustified Absence Rate (%) Justified Absence Rate (%)Total Absence Rate (%) Total Unjustified Absence Rate (%)

Similarities/Differences/ReasonsWhat other questions should be asked?

Monday Tuesday Wednesday Thursday Friday0.0

2.0

4.0

6.0

8.0

10.0

12.0

14.0

Total Absence: Day of the week

200920112012

% A

bsen

t

What might be different in the previous graphs if we just looked at secondary?

Full Pr

imary

inclu

ding K

ura Te

ina

Interm

ediate

Secon

dary (

Year 7

-15)

Second

ary (Y

ear 9

-15) in

cludin

g TPU

and K

ura Te

ina0.02.04.06.08.0

10.012.014.016.0

School Type

Total Unjustified Absence Rate (%)Justified Absence Rate (%)

or Decile

1 2 3 4 5 6 7 8 9 100.0

2.0

4.0

6.0

8.0

10.0

12.0

14.0

Absence and school decile

Total Unjustified Absence Rate (%)Justified Absence Rate (%)

Decile

Gender and year level

Ethnicity

2006 2009 2011 20120.0

2.0

4.0

6.0

8.0

10.0

12.0

14.0

16.0

18.0

20.0

Ethnicity

NZ EuropeanMāoriPasifikaAsianOther*

Abse

nce

%

Regions – is there a link with poverty

2009 2011 20128

9

10

11

12

13

14

15

16

Regions absence rate

NorthlandAucklandWaikatoGisborneWellingtonCanterbury*Otago

Time Series

Wed Fri Tues

Thurs mon wed fri tue

sthu

rsmon wed fri tue

sthu

rsmon wed fri tue

sthu

rsmon wed fri

0

10

20

30

40

50

60

70

80

Number absent from School

Num

ber

Abs

ent

Always ask questions about it.What would it look like at our school?What might be different?

All births in the US 1978

12/2/7

31/1

/741/3

1/74

3/2/74

4/1/74

5/1/74

5/31/7

46/3

0/74

7/30/7

48/2

9/74

9/28/7

4

10/28

/74

11/27

/74

12/27

/741/2

6/75

0

2000

4000

6000

8000

10000

12000

Babies born in the US in 1978

Date

Num

ber

of b

abie

s bo

rn

A snippet

10/28/74 11/2/74 11/7/74 11/12/74 11/17/74 11/22/74 11/27/74 12/2/747500

8000

8500

9000

9500

10000

10500

A snippet

10/28/74 11/2/74 11/7/74 11/12/74 11/17/74 11/22/74 11/27/74 12/2/747500

8000

8500

9000

9500

10000

10500

Why is this Thursday lower

than usual?

Births in NZ 2011

11/18/10 1/7/11 2/26/11 4/17/11 6/6/11 7/26/11 9/14/11 11/3/11 12/23/11 2/11/120

50

100

150

200

250

Distinct number of Babies- 2011

September

8/30/11 9/4/11 9/9/11 9/14/11 9/19/11 9/24/11 9/29/11 10/4/110

50

100

150

200

250September

December births

11/28/11 12/3/11 12/8/11 12/13/11 12/18/11 12/23/11 12/28/11 1/2/120

50

100

150

200

250December

Researching and Referencing

http://csmaster.sxu.edu/caviles/images/

Why Reference? Referencing is necessary to avoid plagiarism.It allows others to follow up and read what other researchers (writers) have to say about the topic.

It will become part of youruniversity life.

http://writecite.com/swsi.nsw/

StyleI encourage my students to use APA referencing as it is often used in university courses.

http://owll.massey.ac.nz/referencing/apa-reference-list.php

Mobile Data UsageSeptember 9, 2013 September 16, 2013

Good site for starting

You can search by dates

www.nzherald.co.nz

Google search tools

Do not use sites like wikipedia, reddit etc., Go to the referenced sites.“Wikipedia acknowledges that it should not be used as a primary source for research.”

The main problem is the lack of authority.

http://downwithtyranny.blogspot.co.nz/2011/08/is-wikipedias-real-problem-really-that.html

Create references as you go.

Form of Assessment

Structure

BrieflyConcise sentencesPassive form (avoid “I”), use impersonal verbs.Correct tenseUse a writing frameVocabulary

PROBLEM and PLANUnderstanding and defining the problem.

Time series is essentially an investigation into ‘what has already happened and what then is likely to happen’ with consideration of how valid it all is.

IntroductionState the investigation.Research related to choosing particular

variables- not just general research.

BackgroundData sourceDescription of variablesImportant aspects of survey detailsMost important factors affecting trends (from

research)

Data and analysisOverviewTrendSeasonal EffectsResidualsIrregularitiesVariationForecast

Overview – add labels

OverviewStart with an overview of what they see.Can include maximum and minimum values and average increase / decreaseUseful words:Rapid/steady/gradual/plateau, increase/decrease, fallen/risen, weekly/monthly/quarterly/annual

Trend Monthly visitor arrivals – Holiday; Jan 2000 to Oct 2012

Description from left to

right

“The graph shows that the trend for the number of holiday visitors was increasing from about 35000 in the beginning of 2000 up to about 59000 visitors in the beginning of 2007. This means there is a rise of approximately 300 holiday travellers every month.”

Include numbers and gradients

Model of good writing

Reasons from research

“During this period, we noticed a sharp increase in the year 2000, this could be caused by multiple international events happening around that time, “Visitors to several international events - America’s Cup, APEC summit, World Netball Championship, Under-17 Soccer World Cup - contributed to this large increase” (as cited in External Migration January 2000).”

Detail – In-depth research

“The prominent increase in the end of 2003 could be partly contributed to the success of “The Lord of the Rings” trilogy which is completed in December 2003. This is reflected by the research “The International Visitor Survey from 2004 found that six percent of visitors to New Zealand (around 120,000 - 150,000 people) cited The Lord of the Rings as being one of the main reasons for visiting New Zealand.” (as cited in Marketing destination New Zealand through the Hobbit trilogy, 2012)”

Description from left to

right

“However, from the start of 2007 to the end of 2011, the trend remains to be relatively stable with a very slight decrease over time.”

Next section

“This change in trend could be explained by the global economical recession starting from roughly 2008, ………The change is understandable as people will first cut their budget in recreational activities like holiday travel.”

Insight!

Student’s own

thoughts about what

is happening

“At the beginning of 2012, especially in February, there was a sudden decrease in holiday visitors to New Zealand.”

Last section

Holiday effect“This could be due to a number of reasons, such as the moving holiday effect of Chinese New Year, “There were fewer arrivals from Hong Kong and China …….”

Seasonal Effects

“There is very clear seasonality in this series. The patterns can be clearly seen in

the following graphs.”

Identify and quantify high and low seasons in context with reasons.

“From the estimated seasonal effect, it shows that holiday visitors are considerably higher in January and February with the peak in February at about 35,000 visitors above the trend.”

Identify and quantify high and low seasons in context with reasons and insight.

“This is important for the New Zealand economy and tourism dependent industries, as that is the time where they can maximize their profits. Hence we can see that tourism industry in New Zealand is a heavily seasonally dependent market.”

Relate back to the investigative

question

Identify and quantify high and low seasons in context with reasons.

“….Moreover, we notice that the peak is normally in February: this is possibly due to the fact that New Zealand is sometimes visited after going to Australia in January.”

Use of language“An increasing population of Chinese holiday visitors to New Zealand also supports the February peak, as their holiday of Chinese New Year usually starts between early and mid February. This is justified by, “Tourism is set to recover from its current slowdown due to the continuing strength of Australia and a growing Chinese market.” (as cited in Forecast commentary, 2012)”

Your own thoughts on why

“…. The number of visitors troughed in June (about 25000 people below the trend line) but raised slightly in July. The trough in June can be caused by the decreasing temperature as New Zealand goes to winter and the increasing amount of rainfall which makes a holiday less favourable.”

Detail “July, however, seems to favour more visitor numbers than June; one would expect this because July is when the summer holiday of the Northern Hemisphere starts. Hence we would see an increase in holiday visitors from UK and China. This explanation is supported by …”

Unusual season(s)“…In particular, there is an outlier in the seasonality for September in 2011, which reaches to about 50,000 instead of the usual 30,000 visitors. This change is caused by positive influence brought by the Rugby World Cup of 2011.”

Variation and residuals

Variation and residuals“After a visual inspection of the graph, the residual is relatively small with most of the variations being below 10% of the overall range (±4000) However, at the beginning of 2011, there is a large residual of about 7500. This unusual residual was probably caused by….”

Components (ball park)

Variation in data: 98000 – 21000 = 77000Variation in Trend: 58000 – 36000 = 2200022/77 = 0.29 i.e. 29% of the variation in the data is the trend

Seasonal component

Variation in data: = 77000Variation in Seasonal Effects: 35000 + 25000 = 6000060/77 = 0.78 i.e. 78% of the variation in the data is the seasonal component

Residual Component

Variation in residuals = 1500015/77 = 19%

SummaryHoliday Visitors to NZ

Min (000)

Max (000)

Range (000)

Approx. % of Contribution

Raw Data 21 98 77Trend 36 58 22 29%Seasonal -25 35 60 78%Residual -5 10 15 19%

NOTE: These are ball-park figures read off the graphs and don’t add up to 100%. The main source of variation comes from the seasonal component which contributes around 78% of the overall variation in the data.

SummaryHoliday Visitors to NZ

Min (000)

Max (000)

Range (000)

Approx. % of Contribution

Raw Data 21 98 77Trend 36 58 22 29%Seasonal -25 35 60 78%Residual -5 10 15 19%

What we are interested in is what is driving this series- in this case the seasonal component.

Prediction Intervals

Prediction Intervals“After a visual inspection of the plot I am confident that the model provides a good fit as differences (white spaces) between the fitted data and the raw data are very small.”

Rounded values

Make an actual prediction“I predict that the average number of holiday visitors to NZ in August 2013 will be between 17400 and 44600. Hence, in the near future, my model predicts that there will be a decreasing trend in 2013.”

Check robustness of the predictionTake out the last 3 months of data, re-analyse and check against predictions.

“The model does not work particularly well for Sept 2012. There was an unusual decrease in visitor numbers, as opposed to the expected increase. The actual value of Sept 2012 does not even fall into the prediction interval.”

Limitations of forecasting“…the data captures a period of economical downturn at the near end, hence predictions are generally decreasing and this will be inaccurate if the economy becomes better in the future.”

“In addition, the data only covers the total number of visitors and it does not signify the visitor spending and the length of stay. Hence it cannot give a very accurate reading of the tourism’s contribution to the New Zealand economy.”

Discusses what the data does not tell you in relation

to the investigative

question

Etc. etc.Second analysis: Visitors of family and friends.

The student then compares the two series.

Similarities and differences with reasons

Etc. etcForms a new series and discusses the contributions made and effects of key events on the new series.

ConclusionYou give a concise summary of the investigation which links back to the original purpose of the investigation.