SScore2 User Manual

Date: 05/08/01

Authors:

1

Table of Contents1. Introduction

1-A. Purpose of the Document1-B. Targeted Reader1-C. Document Conventions1-D. Reader Starting Points

2. Quick Start2-A. Input

A. Rescale to IntegersB. Integer Scale ScoresC. Population DistributionD. Item Parameter File FormatE. Columns For:F. Current FileG. ViewH. Select Item Parameter Input FileI. ProcessJ. QuitK. For 3PL onlyL. Show Percentiles

2-B. Output1. View Output2. Process Output

2-C. Constraints

3. System Overview3-A. Hardware Requirements

1. Hard Requirements2. Soft Requirements

3-B. Directory Structure3-C. Program Flow Diagram

4. Tutorial4-A. Getting Started

1. Download SScore22. Check for and/or Download Java Virtual Machine (JVM) 1.23. Read Documents4. Run the Program

4-B. Program Running1. How to activate the various buttons and what they do2. How to use text boxes (and acceptable data)3. How to use radio buttons and Check boxes 4. How to submit and process data

4-C. After Data has been Processed1. How to access the new data2. How to copy and paste the data??3. How to submit new data4. How to save the new data

5. Appendices5-A. System Requirements

2

5-B. Installation 5-C. Troubleshooting 5-D. Maintenance Procedures and Issues5-E. Contact Information (Group members, Client name, current emails, etc.)

6. Glossary

3

1. Introduction1-A. Purpose of the Document

The purpose of this document is to assist the user in understanding the features and functionalities of the SScore2. Included within this document is a tutorial to take the user through the entire process of execution. This document will also provide easy reference to the various functions used in the program.

It is assumed that the user is familiar with the data involved with the program and has an understanding of basic statistical terms. A glossary is included that defines the statistical terms used in the context of the program.

1-B. Targeted Reader

It is assumed that the reader of the document is familiar with the Windows operating system (95, 98, 2000, ME, XP, NT, … etc.). Furthermore, the user is assumed to be familiar with the Multilog or Bilog statistical software; as files generated by those programs are used as the input of this program. It is also assumed that the reader has a background in statistics so that the terminology and conventions used are not foreign.

1-C. Document Conventions

So far, the only conventions being used are to use Times New Roman font (the current font), and put the section headings in bold type. For specific keyboard commands, we will use Courier New font in green italics. Warnings or particularly important messages will appear italicized and in RED, such as warning. Other conventions will be added as necessary.

1-D. Reader Starting Points

The key starting point for the reader will be Section 2, the Quick start section. Enough information will be provided so the user can immediately begin to use the software as a statistical aid. However, for a more detailed explanation of the various features and their uses, the reader should look in Section 4.

4

2. Quick Start

2-A. Input

The following letters correspond with the GUI pictured above. After each brief description, the section of the full-walkthrough that describes each item in more detail is included. For example, the checkbox that corresponds with letter A is described in greater detail in section 4-B.2.4.

A. This portion of the interface sets the average and standard deviation and evaluates the data from the files accordingly. If the “Rescale to Integers with” checkbox is not filled, nothing placed in the “Average” or the “Standard Deviation” data fields will have any effect on the results. If “Rescale to Integers with” is checked, “Integer Scale Score” and its dependent fields become grayed out. Below is an example.

See Section 4-B.2.4

B. Indicates if you would like Scaled Scores to be displayed in the output. The scaling sets the average and standard deviation of possible scores. For example, to simulate results for the SAT or the GRE, the center (0.0) would be 500 and the standard deviation (s.d.) would be 100. This would produce a range from 200 to 800, as desired.See Section 4-B.2.2

5

C. Sets the desired values of and for population distribution for the underlying variable measured by the test. Modifications of these entries from their default values of 0.0 and 1.0 permit scores to be constructed for populations other than the one for which the items were calibrated.See Section 4-B.2.1

D. The File Format sets the file input type from one of 3 different types of file structures (Columns, Bilog, and Multilog). Choosing Bilog or Multilog will gray the Column input fields and prevent input to these fields.

This becomes the appearance of the Columns if Multilog or Bilog is selected.

Columns: See Section 4-B.3.1.1Bilog: See Section 4-B.3.1.2Multilog: See Section 4-B.3.1.3

E. Choose which columns represent the corresponding data Item Parameters a, b, and c. For use only with the Column format. If Column input is not selected, this portion of the Interface will be grayed out and unavailable for editing.See Section 4-B.2.5

F. Textbox showing currently selected input file. A check is done as to whether the file specified exists. If the file does not exist, the “Process” and the “View” buttons will be disabled and will look like the button below.

See Section 4-B.1.2, 4-B.2.3

G. View the selected input file. This button can only be clicked if there is a valid File selected.See Section 4-B.1.1

H. Clicking on this button will bring up a selection screen that is used to choose an input file.

See Section 4-B.1.2, 4-B.2.3

6

I. Processes the input file to produce the output file. This will take into account all the valid data entered in the form. This button can only be clicked if a valid file is selected.See Section 4-B.1.3

J. Button to quit the program.See Section 4-B.1.4

K. Choosing between the normal metric data calculation and the Logistic metric calculation. Multilog is always defined using normal metric regardless of user selection. Bilog and Column formats are affected by this selection. See Section 4-B.3.1.1

L. Button to report percentiles in output. If not selected, the percentile column will be blank in the output.See Section 4-B.3.1.2

2-B. OutputThe output of your program will vary depending on the options chosen. There are two outputs to this program: the output obtained from pressing the view button, and the output obtained from pressing the process button.

View OutputThe out of this screen is based entirely on the file selected. Regardless of the file type, the information stored in the selected file is displayed. Below is an example.

7

This particular image is of a column file.

Process OutputAgain the results of the output will vary depending on certain features. For example, the data will only have a percentile column if the “Show Percentiles” checkbox is checked. Below is a sample output.

Below are the heading types you will encounter and their description.Score: Represents the summed score, which ranges from the minimum to the maximum score.EAP: Expected A Posteriori and represents the score that has yet to be scaled.S.D.: Standard Deviation and is calculated based on the data provided.Prop.: Expected proportion for each summed score. Sscore: Scaled scores. For example, this would be from 200 – 800 on the GRE/SAT.SE: Standard error of the scaled scores.%ile: Only appears if the percentiles are requested on the GUI.

2-C. Constraints

With the exception of the “Current File” input field and the “Columns For” area, giving the interface unacceptable values will cause the program to give the problematic fields default values that are output to the screen. The “Columns For” input fields are given default values if anything other than numbers is inserted, however the default values are not placed onto the screen.

8

1) The user MUST input the correct values for the input fields in “Columns For” or the program may crash or yield incorrect results.

2) Correct files with correct file format selection must be selected or else errors and/or crashes may result. The program takes into account whether an indicated file exists or not, however it does not distinguish between different file types such as *.doc (a word document) and *.zip (a compression format).

9

3. System Overview3-A.1 Hardware Requirements

Any computer running on a Windows 95 platform or greater. For best results (maximum performance), we suggest any machine running on with a process speed of > 100 Mhz.

3-A.2 Software RequirementsWarning: The computer MUST have a Java Virtual Machine (JVM) of version 1.2 or greater. The program will not run on an earlier JVM. To see if you have the proper JVM or too get the necessary JVM, please see section 5-C.

3-B. Directory StructureAll the files should be saved in the same directory:

We suggest that you place the program in the following directory, but you may save the files into any desired folder.

Suggested: C:\Program Files\SScore2

The resultant data will be able to be saved into any file you choose. Again, we suggest the following, but the decision is up too the individual user.

Suggested: C:\Program Files\SScore2\Data

Here is some space to record where things are in relation to other files. Ex. Where to find the processed data.

____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

10

3.C Diagram of Data Flow

This document shows the user the various ways in which data can be processed in the GUI and the flow of the program.

11

4. TutorialThe tutorial will provide a full walkthrough of the program and how each function is intended to be used. Also included in the tutorial is how to download and install the program and necessary software to run the program. The tutorial is designed to compliment the Quick start section, acting as a source of more detailed information and explanations on the functionality of the program.

4-A. Getting StartedThe following section describes in detail how to download the SScore2 onto the user’s computer and obtain the necessary software updates.

4-A.1. Download SScore2

The first step that must be taken is too download the Software package from the following Internet address:

http://www.cs.unc.edu/~karlsson/Product/SScore2.zip

The Software can also be found by going to the product homepage and clicking on the link entitled: ‘Download SScore2’. The Internet site is:

http://www.cs.unc.edu/~karlsson

The program will be saved as a .zip file. When download is complete, unzip the contents of the file to the following directory:

C:\Program Files\SScore2

If the software was received via a software package on a CD-ROM disc, simply place the disc in the CD drive and save the contents to the above suggested file.

*Note: The files can be unzipped into any directory, but for ease of use we suggest the above file.

4-A.2. Check for and/or Download Java Virtual Machine (JVM) 1.2

The following step is paramount to successfully using the SScore2. The current computer you are using MUST have a JVM no earlier than version 1.2. For example, having JVM 1.1 would not be sufficient for running the program.

To check which JVM is currently installed on your computer, do the following:

GO TO SECTION 5-C.

To obtain the current version of the JVM, do the following:

GO TO SECTION 5-C.

12

4-A.3. Read Documents

Upon successful download and unzipping of the software, it is recommended to read through the remainder of the walkthrough or at least peruse the Quick start section before using the software. These documents were designed in order to make the software simple to learn and should answer any questions about the software and its use.

4-A.4. Run the Program

To run the program, find the folder were the program contents were unzipped. In that folder should be an .exe file entitled (SScore2.exe). Double click the mouse on the .exe file, and the GUI should pop up. The program is ready to be used.

4-B. Program Running

4-B.1. How to activate the various buttons and what they do

1.1 ViewThe View button allows the user to preview the currently selected input file and view the contents of that file. This button my only be pressed after the user has chosen an input file. After a file has been selected and the view button is pressed, a pop-up window containing the contents of the selected input file will be displayed. By viewing the input file, the user can determine which input format (Columns, Bilog, Multilog) should be used, and in the case of Columns, which Columns should be selected.

Note that the View button will not appear until an input file has been chosen.

1.2 Select Input Parameter Input FileThe Select Input Parameter Input File button brings up a menu that allows the user to select a file saved on their computer (or on a floppy disk) to process. Once selected, the file data and format can be viewed using the View button. After being selected, the rest of the GUI should be filled out and the Process button should be pressed to finish processing the data. To change the file at any time, you can press the Select Input Parameter File button again to select a new file. You may also type in the name of the file if you know where the file is located on your computer. For example:

C:\Program Files\SScore2\Data\Data.irt

Note that if the file you type in is invalid, you will not be able to process the data. The Process button only appears if the selected file is valid.

1.3 ProcessThe Process button should only be used once the GUI has been filled out in its entirety. Although the Process button may be pressed once an input file has been selected, it will only calculate data based on default values. Once pressed, the selected input file data is processed and a pop-up window containing the

13

output appears. (If there are any errors with the input file or the selected options, an appropriate error message will appear to state the error.)

Note that the Process button will not appear until an input file has been chosen.

1.4 QuitThe Quit button stops the program and closes the GUI window. Make sure to save all data files before pressing the quit button, as you will have to run the program again if the output data has not yet been saved. This button may be pressed at any point to stop the program. To restart the program, simply double click on the (SScore2.exe) file again.

4-B.2. How to use text boxes (and acceptable data)

2.1 Population Distribution: By changing the Pop. Dist. mu text field, the user specifies the mean of the population distribution on the underlying variable measured by the test. Usually, mu is 0.0, but it may be smaller or larger (in standard deviation units) to compute scores for populations that differ from that for which the items were calibrated. The values entered may be either integers or floating-point numbers. If no value is entered in the text field, then the default value for is 0.0.

: By changing the Pop. Dist. s text field, the user specifies the standard deviation of the population distribution on the underlying variable measured by the test. Usually, s is 1.0, but it may be smaller or larger to compute scores for populations that differ from that for which the items were calibrated. The values entered may be either integers or floating-point numbers. If no value is entered in the text field, then the default value for is 1.0.

2.2 Integer Scale ScoresTo use the Integer Scale Scores feature, the check box directly underneath the heading must by checked.To activate the Integer Scale Scores check box, simply click the mouse cursor in the check box (An X will appear). To deactivate the Integer Scale Scores check box, click on the box again (The X will no longer appear).

Scale 0.0 to: By changing the Scale 0.0 to text field, the user specifies the average value of the test scores on the reporting scale. For example, to obtain scores on the T-score scale, a value of 50 is entered. The values entered may be either integers or floating-point numbers. If no value is entered in the text field, then the default value is 0.0.

Scale s.d. to: By changing the Scale s.d. to text field, the user specifies the standard deviation of the test score reporting scale. For example, to obtain scores on the T-score scale, a value of 10 is entered. The values entered may be either integers or floating-point numbers. If no value is entered in the text field, then the default value is 1.0.

14

2.3 Current FileThis Current File text field works in conjunction with the Select Item Parameter Input File Button. The text that appears in this field may only be altered by either clicking the Select Item Parameter Input File Button and choosing a new file or by typing in the name of a valid file that is saved on the computer.

Note: you can alter the text field by selecting it and typing in the file you wish to select. However, the file MUST be found on the computer or the process button will not appear.

2.4 Rescale to IntegersTo use the Rescale to Integers feature, the check box directly underneath the heading must by checked.

To activate the Rescale to Integers check box, simply click the mouse cursor in the check box (An X will appear). To deactivate the Rescale to Integers check box, click on the box again (The X will no longer appear).

Average: By changing the Average text field, the user specifies the average value of the test scores on the reporting scale. For example, to obtain scores on the T-score scale, a value of 50 is entered. The values entered must be either integers or floating-point numbers. If no value is entered in the text field, then the default value is 0.0.

Standard Deviation: By changing the Standard Deviation text field, the user specifies the standard deviation of the test scores. For example, to obtain scores on the T-score scale, a value of 10 is entered. (Note that this effect differs from “Integer Scale Scores” in section 2.2 in that this rescaling produces scores that actually have the specified standard deviation in the sample of test scores; the same values entered in “Integer Scale Scores” in section 2.2 produce scores with a somewhat smaller standard deviation.) The values entered must be either integers or floating-point numbers. If no value is entered in the text field, then the default value is 1.0.

2.5 ColumnsEach of the Columns must be entered as an integer value. To ensure the correct column numbers, press the View button after selecting the file and check which columns correspond with the correct data.

a.: Enter the first column of data in the Column text field labeled a. Make sure that this is an integer value that corresponds with the column of data in the input file that contains the values for the a parameter of the three-parameter logistic model.

b.: Enter the second column of data in the Column text field labeled b. Make sure that this is an integer value that corresponds with the column

15

of data in the input file that contains the values for the b parameter of the three-parameter logistic model.

c.: Enter the third column of data in the Column text field labeled c. Make sure that this is an integer value that corresponds with the column of data in the input file that contains the values for the c parameter of the three-parameter logistic model.

Note: The Columns Text Fields can only be altered if the Columns Radio Button has been selected.

4-B.3. How to use radio buttons and Check boxes

3.1 Check BoxesThe various check boxes switch on and off different aspects of data calculation and output formats as described below.

3.1.1 Integer Scale ScoresCheck the Integer Scale Scores check box to (scale the data to the specified average and standard deviation). This button does affect the resultant data, so be careful when using this feature.

3.1.2 Rescale to Integer ScoresCheck the Rescale to Integer Scores check box to (scale the data to the specified average and standard deviation). This button does affect the resultant data, so be careful when using this feature.

3.1.3 Show PercentilesCheck the Show Percentiles check box to add a column to the output that shows the percentile of the resultant data. This button does not affect the resultant data in any way.

3.2 Radio Button The Radio button allows you to choose alternate between one of several choices for both the selected file format and the use of the 3PL feature. To alternate between the various options, use the mouse to click the circle in front of the desired option. A small black circle will appear in the selected circle to verify your selection.

Note: Be sure the selected file format is the same as the selected file. If they are not, errors will result in the calculations.

3.2.1 ColumnsSelect the file input data format as Column input.

3.2.2 BilogSelect the file input data format as Bilog input.

3.2.3 MultilogSelect the file input data format as Multilog input.

3.2.4 Normal Metric16

Select the Normal Metric option to process the data with.

3.2.5 Logistic MetricSelect the Logistic Metric option to process the data with.

4-B.4. How to submit and process data

The procedure to submit the data to the program for processing is simple. After entering all the relevant and necessary data in the various fields in the GUI, hit the Process button. An output window will appear with the data processed and formatted as specified in the program

4-C. After Data has been Processed

4-C.1. How to access the new dataOnce the new data has been created (Once the Process button has been pressed), a pop up text window will appear with the data already formatted. You may then choose to save the data file, copy and paste the data, or process a new data file.

4-C.2. How to copy and paste the dataTo copy and paste the new data, select the data you wish to copy by highlighting all of the data with the mouse. Then hit either Ctrl C or right click on the mouse and select Copy. Then open a new text window (Word, Notepad, etc…) and select Paste from the edit menu at the top.

4-C.3. How to submit new dataTo choose another input file to process, hit the Select Item Parameter Input File button and choose a new input file. Be sure to update any relevant choices (Changes to standard deviation and average, Different input file type, etc…).

4-C.4. How to save the new dataTo save the newly created data, click the File button on the menu bar of the Output window. Select the Save As option and save the data in the specified folder. We suggest saving the file as a text file or a word document.

17

5. Appendices

5-A. System Requirements Hard Requirements: The system must be running on an Operating System that is

Windows 95 or later and the system must have JVM (Java Virtual Machine) version 1.2 installed. To determine whether the computer has JVM installed, please see 6 – D. Troubleshooting.

Soft Requirements: The machine should have a processor (CPU) speed of 100+ MHz. A minimum of 16 MB of memory is also recommended.Currently, there are no other soft requirements.

5-B. Installation The file will be SScore2.exe file that will be run by the user anytime they wish to process data. It will be placed in the suggested directory

C:\Program Files\SScore2.exe

5-C. Troubleshooting To determine if the JVM file exists on the computer, the user may do a find files or folders for the file “jvm.cfg”. To do so…1> First click on the Windows “START” button (usually on the lower left corner of the

screen).2> Select “Find” from the Windows Menu that pops up after clicking on the “START”

button.3> Click on “Files or Folders…” from the “Find” menu.

18

4> When the find window appears, type on jvm.cfg and click on the “Find Now”

button.

5> If the file could not be found, be sure to look into all the drives that have data stored in it and not just the C: drive.

6> If jvm.cfg is not found in any of the drives, please refer to the next section on how to download jvm from the web.

To download the latest version of jvm.cfg from the web…1> Open up an Internet browser. (Internet Explorer or Netscape Navigator).2> Type in http://msdn.microsoft.com/visualj/downloads/wfcinfo.asp in the address bar

and click “enter”.3> This should take you to the Microsoft’s Visual J++ page with the title “Windows

Foundation Classes Update.”4> If the link has been updated, go to http://www.microsoft.com and type in JVM in

the search field. After the search results have appeared, click on Microsoft Visual J++ Information for WFC. This link should also take you to the previous mentioned page.

5> From the “Windows Foundation Classes Update.” Page, follow the instruction on how to Download the new Microsoft Java Virtual Machine and download it onto your computer.

Currently there are no plans to update or handle errors caused by the software. It is possible that should the use of this software become widespread, some form of support will be offered. However, there are no plans to do so for the first version of this software.

5-D. Maintenance Procedures and IssuesA detailed log will be kept of all known bugs and the solutions required to fix them. This will be updated as new defects/bugs are found, and should give a detailed overview of any problems that might occur from time to time.

5-E. Contact Information Current as of 28 April 2001

Client: Prof. David ThissenEmail: dthissen@email.unc.edu

Director: Chris Schenck Email: cschenck@email.unc.edu

19

Producer: Ian YuanEmail: ianyuan@email.unc.edu

Administrative Leader: Benjamin Chen Email: bchen@email.unc.edu

Librarian: Charles SkenderEmail: drgnrbrn@email.unc.edu

Quality/Test Leader: Eric Karlsson Email: karlsson@email.unc.edu

20

6. Glossary

Integer Scale Scores: The basic scaled scores are computed on a scale that has an average of 0.0 and a standard deviation of 1.0; this yields negative and decimal-valued scores. Test scores are usually reported as positive integers. That is accomplished by a linear transformation of the underlying scores, for example multiplication by 100 followed by the addition of 500 yields scores on the well-known College Board 200-800 scale. Because of the nature of the summed score to scaled score computations implemented here, the standard deviation of the standard Integer Scale Scores is smaller than the nominal value. “Rescaling” to a specified Average and Standard Deviation yields scaled scores that are predicted to have the specified average and standard deviation in the data from the test-takers.

Population Distribution : The mean, or average, score of the group of test-takers on the underlying variable being measured by the test, usually referred to as .

Population Distribution : The standard deviation of the group of test-takers on the underlying variable being measured by the test.

3PL: The Bilog (see below) program takes in information about the test and computes the parameters used in this Scaled Scores program. In calculating the parameters, Bilog uses a logistic equation to represent the item response function. This curve is calculated using two different formats: the normal metric and the logistic metric. These formats differ in that the logit of the curve calculation is multiplied by a different constant. Because the Bilog program output does not specify which format is used, the user must know and enter the format used so that curve calculations will be consistent across both programs.

Normal Metric: Using Bilog or Columns input, this multiplies the logit by 1.7.

Logistic Metric: Using Bilog or Columns input, this multiplies the logit by 1.0.

Input Parameter File Format: One of three possible types of input files. All three of the files contain the same parameters, but in different formats because the files are the output of previously run programs. Test data must have been run on one of the three programs before it can be manipulated in this program.

Bilog: This particular file format is the output from the Bilog program. The Bilog program calculates the difficulty and other item response function parameters for each question on a test that can have only two answers (right or wrong). This Scaled Scores program uses these parameters to assign the proper scores to an individual with a given summed score on the test.

Multilog: The Multilog software is very similar to Bilog in that it calculates the difficulty and other item response function parameters for each test question. The difference is that while Bilog calculate these parameters for questions with only two answers (right or wrong), Multilog can calculate these values for questions where the participant can receive partial credit (Ex: An essay question out of 10 points). Again this Scaled Scores program uses these parameters to assign the proper scores to an individual with a given summed score on the test.

Columns: The Columns file format is usually derived from the entry of item parameters into a database or spreadsheet system. The item parameters are listed in three of the columns of this format, thus the box that designates which columns contain parameters a, b, and c.

21

GUI: Graphical User Interface – This is the window that pops up when the program is run. It is large and gray and allows the user to enter information about the test and population to be scored.

22