Frank Milthorpe

27 May 2011

Suite of Emme Macros to Enable Flexible Specification of Multiple Data Sources

Outline

1. Goals

2. Text registers within Emme

3. Implementation Approach

4. Conclusions

5. Listing of macros

Goals

• Flexible definition of input data

• Easy to distinguish differences between model runs

• Implemented within Emme macros

Overview of Desired Specification

Run 1

Standard Inputs

except

Employment_A

Run 2

Standard Inputs

except

Population_B

Employment_C

Text Registers

• t1 .. t9 are text registers available

• t9 is global

• t1 .. t8 local registers – can’t return arguments

• Can’t “load-up” registers with filenames from a lower macro

Implementation Approach

• Create a LIFO (last in first out) text file

• Write to text file with label and filename

• Read the text file in appropriate macro to retrieve and return filename via t9

• Also allows an audit trail

Example Data File Createdstart_default === 27-Apr-2011 15:23zonal_data M:\STM_Core\Inputs\ZonalMtx\emp_data M:\STM_Core\Inputs\Employment\2009_Oct\pop_adjust_file m:\stm_gma\lu_adjust\pop_adjust_default.mtxemp_adjust_file m:\stm_gma\lu_adjust\emp_adjust_default.mtxtimeskim_busonly_macro tskimfaccarskim_data M:\STM_Core\Inputs\Calibration_data\Dec_09\end_default ===pop_adjust_file M:\Myproject\PopFac_2011.mtxcarskim_data M:\Myproject\carskim_matrices\

Reading Data~/ ... Start of macro read_emp.mac %ms02% ...~#~t2=%t1%read_emp.rep~<ifexist_delete %t2%reports=%t2%~#batchin=?~#~# --- Find directory for the data~<get_data_source_item emp_data~<stm_log emp_data %t9%~#~t3=%t9%~#~# --- Non-Manufacturing employment: md3 ---~+;3.11;%t3%nonmanemp%ms02%.mtx;2;~#

Conclusions

• Macro suite designed and implemented to meet requirements

• Can be a filename or a directory

• Can be adapted for other purposes (eg specify assignment parameters)

• Text file provides an audit log of specifications

Core Macrosdelete_data_source_stack Delete the data_source_stack file

create_data_source_stack Create data_source_stack file

write_data_source Write an item to a data_source_stack file

get_data_source_item Find a return value for data item

strip_leading_blanks Trim the leading spaces from a text string

delete_data_source_stack.mac~:start~/ ... Start of macro delete_data_source_stack.mac ...~#~<ifexist_delete stmaudit\data_source_stack.dat~#~/ ... End of macro delete_data_source_stack.mac ...

create_data_source_stack.mac~:start~/ ... Start of macro create_data_source_stack.mac ...~#~<ifexist_delete stmaudit\data_source_stack.dat~#~#~# Write something to the data_stack to ensure other macros~# have a file to open.~#~<make_directory stmaudit~#~<get_date_time~<write_data_source data_stack_created %t9%~#~/ ... End of macro create_data_source_stack.mac ...

write_data_source.macExample macro call write_data_source %1% %2% %1% - source_name %2% - directory (or other item) eg ~<write_data_source pop_data m:\popdata\2008\

~:start~/ ... Start of macro write_data_source.mac %t0% ...~#~t1=stmaudit\data_source_stack.dat~t2=%t0%~#~# Strip leading blanks~:continue_stripping~t3=%t2.1%~+;~?!t3= ;~$>stripped~+;~t2=%t2.-1%;~$continue_stripping~#~:stripped~# Write to output file~+;~>>%t1%;~"%t2%;~>

get_data_source_item.mac (1)~$>start

Emme/2 macro "get_data_source_item.mac"

Macro to obtain answer (normally directory) for a data itemwritten to the data_source file. This is the last entry written.

N.B.If macro used the first entry this may not be the entry that is needed.For example if the model was rerun with different defaults and the stackwas not deleted, it would return the first (and wrong original entry).

Example macro call ~<get_data_source %1% %1% - source_name ~<get_data_source pop_data

Returns %t9% Item (directory or other data label)

get_data_source_item.mac (2)~:start~/ ... Start of macro get_data_source_item.mac %t0% ...~#~t1=stmaudit\data_source_stack.dat~#~#~# --- Find length of macro argument %1% store in %x%~t2=%1%~#~x=-1~:continue_counting~x+1~t3=%t2.1%~+;~?t3= ;~$>counted~+;~t2=%t2.-1%;~$continue_counting~#~:counted~#~# length of argument %1% is %x%~#

get_data_source_item.mac (3)~# --- Find line in file with source_name~#~# t8 - line of data read in~# t7 - line of data with a match~# t3 - substring of t8 for trial match to source_name %1%~# r1 - count of entries found (set to zero initially)~#~r1=0~#~:process_line~t8=~@~+;~?e;~$>end_of_file~+;~t3=%%%t8.%x%%%%~+;~?t3=%1%;~t7=%t8%~+;~?t3=%1%;~r1+1~$process_line~#~:end_of_file~#

get_data_source_item.mac (4)~# Was the source_name found?~+;~?r1>0;~$>found~#~:not_found~/ did not find %1% in get_data_source_item~t9=Error %1% Not found in get_data_source_item~$>end~#~:found~# --- Extract the item label (probably directory)~# remove first %x% chars~+;~t2=%%%t7.-%x%%%%~#~# remove leading blanks~# result is returned in %t9%~<strip_leading_blanks %t2%~# returned value is %t9%~:end~# Close file~>~/ ... End of macro get_data_source_item.mac %1% %t9% ...

Strip_leading_blanks.mac~:start~# Does this macro have an argument passed to it?~x=%0%~+;~?x>0;~$>valid_argument~# No argument - nothing to strip~t9=%t0%~#~$>end

~:valid_argument~# remove leading blanks if they exist~t2=%t0%~:continue_stripping~t3=%t2.1%~+;~?!t3= ;~$>stripped~+;~t2=%t2.-1%;~$continue_stripping~:stripped~t9=%t2%~:end~# returned value t9 is:%t9%~# ... End of macro stip_leading_blank.mac %t9% ...

Implementation Macrosset_default_data_sources Write the default data sources to

data_source_stack file

read_emp Macro to read employment matrices

(similar macros exist to read other data files)

set_default_data_sources.mac~/ ... Start of macro set_default_data_sources.mac ...~#~# Make sure directory where data_source_stack.dat resides exists~<make_directory stmaudit~#~# Delete data source stack if it exists (NOT IMPLEMENTED)~# Stack operates as a permanent record, use last occurance of keyword~#~<get_date_time~<write_data_source start_default === %t9%~#~# --- Set the default data sources~<write_data_source zonal_data M:\STM_Core\Inputs\ZonalMtx\~<write_data_source emp_data M:\Inputs\Employment\2009_Oct\~<write_data_source lu_internal_data m:\stm\march_2005_gma\

and etc as per user needs

read_emp.mac~:start~/ ... Start of macro read_emp.mac %ms02% ...~#~t2=%t1%read_emp.rep~<ifexist_delete %t2%reports=%t2%~#batchin=?~#~# --- Find directory for the data~<get_data_source_item emp_data~<stm_log emp_data %t9%~#~t3=%t9%~#~# --- Non-Manufacturing employment: md3 ---~+;3.11;%t3%nonmanemp%ms02%.mtx;2;~#

General Utility Macrosifexist_delete Delete a file if it exists

get_date_time Obtain and return the date and time

make_directory Create a DOS directory

stm_log Write to logbook and a DOS file

wait Wait a specified number of seconds

ifexist_delete.mac~:Start~#~!if exist %1% del %1%~<wait 2~#~:End~#/ ... End of macro IfExist_Delete.mac ...~#

get_date_time.mac (1)~:start~# --- Get date string~# Obtain current day - Global parameter p=2008~p=2008~x=%p%~# Obtain current month - Global parameter p=2007~p=2007~y=%p%~# Obtain current year - Global parameter p=2006~# Depending of status of switch 25~# This value may only be a 2-digit year.~p=2006~z=%p%~+;~?z<2000;~z+2000~# --- Obtain month character label~+;~?y=1;~t2=Jan~+;~?y=2;~t2=Feb . . . . .~# Consolidated Date String dd-mmm-yyyy~# If d is less than 10 need a 0 to pad the field~t1=%x%~+;~?x<10;~t1=0%x%~t4=%t1%-%t2%-%z%

get_date_time.mac (2)~# --- Get date string~# Obtain current hour - Global parameter p=2009~p=2009~x=%p%~# Obtain current minute - Global parameter p=2010~p=2010~y=%p%~# Obtain current second - Global parameter p=2011~p=2011~z=%p%~# Consolidated Time String hh:mm~# If hour or min is less than 10 need a 0 to pad the field~t1=%x%~+;~?x<10;~t1=0%x%~t2=%y%~+;~?y<10;~t2=0%y%~t5=%t1%:%t2% ~t9=%t4% %t5%~#~# t9 (Date Time) is %t9%~#

make_directory.mac~:start~/ ... Start of macro Make_directory.mac %1% ... ~#~!IF not exist %1% MKDIR %1%~#~/ ... End of macro Make_directory.mac %1% ...

stm_log.mac

Example macro call stm_log %t0% %t0% - text to write to log file / logbook

~:start~/ ... Start of macro stm_log.mac %t0% ...~#~t1=stm_log.rep~#~# Write to output file~+;~>>%t1%;~"%t0%;~>~#~# Write to logbook (with a max of 60 characters)c=%t0.60%~#~/ ... End of macro stm_log.mac %t0% ...

wait.mac~:Start~# current cpu time since creation of database in 1/10 secs~p=2005~x=%p%~# current sec (1/10) since database creation %x%~#~# Add %1% * 10 to current 1/10 second to get new target~r1=%1%~r1*10~r1+%x%~z=%r1%~#~#~# New target is %z%~:Repeat~# current cpu time since creation of database in 1/10 secs~p=2005~x=%p%~?x<%z%~$Repeat~#