aivazian-preview days evaluation project
DESCRIPTION
Russell Aivazian and Diego DonnaPreview Days Evaluation PlanDecember 2014TRANSCRIPT
-
Running head: GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 1
Graduate Assistant Preview Days Evaluation Plan
Russell C Aivazian & Diego Donna
Loyola University Chicago
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 2
Table of Contents Introduction 3 History of Preview Days 4 Inputs 6 Purpose and Assumptions 7 Format and Structure 10 The Buddy Program 10 Higher Education Program Information 11 Workshops and Panels 11 Campus Tours 13 Socials 13 External Factors 14 Stakeholders 15 Evaluation Approach 16 Quantitative Approach 18 Population and Sampling 18 Design and Instrument 18 Pilot Testing and Implementation 21 Analysis and Results Presentation 23 Qualitative Approach 24 Participants 25 Interview Protocol 26 Implementation 27 Limitations 29 Budget 30 Timeline 31 Next Steps 32 References 33 Appendices
A. Logic Model 34 B. Previous PD Schedules 35 C. Preview Days 2010 Evaluation Findings 38 D. 2013 Instrument 46 E. Survey Instrument 58 F. Construct Map 67 G. Email Templates 73 H. Interview Protocol 75 I. Interview Consent Form 78 J. Interviewers Notes 79 K. Email Templates 80 L. Initial Codes 81 M. New GA Compensation and Structure Email 82 N. Timeline 85 O. Budget 86 P. Presentation 87
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 3
Specifically for students looking to pursue a degree in student affairs, interview and
outreach programs are one of the ways students look to better understand campus culture and
make a more informed decision on the school where they want to pursue their masters degree.
Depending on the program, these programs are often used as requirements for admission or
consideration of an assistantship. At Loyola University Chicago (LUC), the Preview Days
program is a partnership between the School of Education, Academic Affairs, and the Division
of Student Development (DSD) to help recruit and fill Graduate Assistant (GA) positions across
various functional areas within the university. The GA positions filled during Preview Days
supplement the coursework in various masters programs across the university, including the
Master of Education in Higher Education program (Division of Student Development, n.d.).
Each assistantship allows students the ability to engage in professional development
opportunities, while receiving financial support for their graduate studies (in the form of tuition
remission and a stipend, depending on position). Graduate Assistants work in departments
across the University, including, but not limited to, Student Activities and Greek Affairs, Student
Diversity and Multicultural Affairs, Residence Life, Academic Affairs, and Campus Ministry.
GAs are typically offered ten month contracts, lasting the duration of their academic program
(two years for Higher Education masters students) and work about twenty hours per week, on
average (Division of Student Development, n.d.).
In addition to providing a space to conduct interviews for the various GA position
openings, Preview Days provides an opportunity for the Universitys partners to interact with
potential graduate students and help candidates articulate the Loyola GA experience.
Throughout the two days of the program, participants are invited to attend various lectures,
panels, tours, and socials in order to better understand the institution and the opportunities
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 4
available at LUC. Even though the Preview Days program generally caters to Higher Education
masters students, participants from other graduate preparation programs (Social Work, Pastoral
Studies, Business, etc.) are invited to attend as well.
History of the Preview Days Program
Prior to 2006, each department within the Division of Student Development (DSD) and
Academic Affairs used their own process to recruit, interview, and hire Graduate Assistants
(GAs) for their open positions. Due to the lack of consistency among the various departments,
GA candidates were met with a confusing process that required them to submit similar and
redundant application materials for each GA position. Jack McLean, JD, Assistant Vice
President for the DSD, notes that even though many departments shared application materials
and processes, there was little collaboration to create a cohesive process and hiring timeline
(personal communication, September 25, 2014). Driven by requests from individual departments,
the DSD was charged with creating a Preview Days experience that introduced candidates to
Loyola and reflected the changing demographic of the Higher Education masters program
(specifically, increased student interest from outside the Midwest). Under McLeans leadership,
the GA Recruitment and Selection Team was created to help streamline the entire process, which
included planning the components of the Preview Days program.
The original Preview Days program was first introduced in 2006 as the Graduate Intro
Days, which invited GA candidates to Chicago to interview for all open positions in a more
centralized process. As graduate enrollment increased in the Higher Education program and the
DSD expanded their GA program, the Division focused on creating a development plan for their
GAs. This included the creation of the GA Formation Council and the introduction of the GA
orientation, which was used as a supplement to the Intro Days program in 2008. In 2009,
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 5
under new leadership, the DSD revised the GA recruitment and selection process to increase
collaboration between the Higher Education program and the various divisional partners in order
to create a more meaningful experience for prospective Graduate Assistant (GA) candidates. It
was at this point that the Division of Student Development (DSD) changed the name to Preview
Days, signaling a renewed focus on introducing GA candidates to the University and the GA
experience. McLean explains that the initial vision for Preview Days was modeled after the
program at Seattle University. In this new model, interviews would be paired with sessions
about the Loyola GA experience and information about moving to and living in Chicago (Jack
McLean, JD, personal communication, September 25, 2014).
Since 2009, the GA Recruitment and Selection Team have worked to create a more
coordinated and centralized process by evaluating and assessing the various components of the
process (see Appendix C and D for a summary of the evaluation findings from 2010 and the
evaluation instrument used in 2013). One of the more significant changes is the move from
paper application materials to a fully electronic system of gathering and evaluating applicant
materials and demographic data (J. Curtis Main, personal communication, September 25, 2014).
Using this new online system, GA candidates are able to submit their applications in one
centralized location and indicate their intention to attend the Preview Days program. On the
administrative end, hiring supervisors (individuals selected by their departments to coordinate
the hiring process for their GAs) are able to gather candidate information and easily identify
possible candidates for their open positions. This also allows for more consistent communication
from the DSD and an easier process for GA candidates to inquire about the assistantship
offerings and Preview Days program. As a result of the Divisions efforts to create a more
streamlined process and training sessions offered by the Higher Education program to help GA
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 6
supervisors understand the graduate application process, the DSD has seen an increase in the
number of submitted applications and overall attendance at Preview Days.
Inputs to Preview Days Program
The Preview Days program requires collaboration between DSDs departments and
partners in order to create a positive experience for prospective GAs. As our logic model
(Appendix A) illustrates, the execution of the Preview Days program requires the interaction and
use of four groups of inputs: prospective Graduate Assistants (GAs), time, resources, and
personnel. Since the Preview Days program is planned for the benefit of prospective GAs, it is
important to consider their input and behaviors when considering the many components of the
program. Without the attendance of prospective GA candidates, the Preview Days program
would not achieved its desired goals and outcomes (discussed in the next section).
Jack McLean and J. Curtis Main (personal communication, September 25, 2014) both
expressed the considerable amount of time that goes into the planning and execution of the GA
recruitment and selection process. Beginning in late June, members of the Division of Student
Development (DSD) and the Higher Education program get together to evaluate the process from
the previous year and set internal and external deadlines for the departments and prospective GA
candidates. During this planning stage, the various departments work to define the dates for
Preview Days (usually late February) and the timeline for the planning and execution of the
program. In addition, individual departments set aside time to review applications and plan for
the potential openings in their departments for the next academic year.
The Preview Days program must also take into account the various resources needed to
provide an impactful experience for the potential GA candidates. During the program,
participants are also treated to a selection of food from various Chicago restaurants, which
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 7
require the commitment of monetary resources from the Division of Student Development
(DSD). In addition to the cost of food, the DSD must cover the cost of printing all materials
(Preview Days guide, nametags, and handouts) used throughout the program (J. Curtis Main,
personal communication, September 25, 2014).
Finally, the Preview Days program enlists the participation of the personnel within the
DSD in various capacities. In order to plan and execute the entire GA recruitment and selection
process, the DSD relies on two committees: the Graduate Assistant Recruitment and Selection
Team (GARST) and the Preview Days Planning Committee (PDPC). Whereas the GARST
works to plan the entire recruitment and selection process for the DSD, the PDPC works to plan
the various components of the Preview Days program. Both of these committees require the
DSDs personnel to commit time to carrying out the process and work in concert to create a
positive experience for prospective GAs. In addition to serving on one of the planning
committees, Divisional staff and current GAs are also invited in a number of capacities
(presenting, attending a social, leading a campus tour, etc.) to interact with the prospective GAs
throughout Preview Days. Since the Preview Days program is planned in partnership with the
Higher Education program, faculty members are also invited to interact with candidates and
provide an introduction to the Higher Education program (Jack McLean, JD, personal
communication, September 25, 2014). It is assumed that throughout this process, University and
Divisional personnel will be available and willing to dedicate their time and resources to the
execution of this program (Appendix A).
Purpose and Assumptions of the Preview Days Program
Even though the Preview Days program has existed in its current form since 2009, the
program lacks explicit learning outcomes to help guide the planning and execution of the
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 8
program (Jack McLean, JD, personal communication, September 25, 2014). When assessing a
specific program, stated learning outcomes are essential to understand the reasons why the
program exists and what the program aims to achieve. As Weiss (1998) explains, when goals
are unclear or ambiguous, more than the evaluation can be affected. Where there is little
consensus on what a program is trying to do, the staff may be working at cross purposes (p. 53).
In the absence of specific learning outcomes for the program, McLean explains that the primary
focus of the Preview Days program is to expose participants to Loyola and the Division of
Student Development (DSD) in order to assess whether or not Loyola will be a good fit for a
candidates graduate studies. The Preview Days program works to achieve this outcome by
partnering with the Higher Education program to expose students to the expectations of the
program, introducing the Jesuit mission of Loyola University Chicago, helping departments
coordinate their outreach to potential candidates, and expose candidates to the diverse pool of
applicants and staff within the DSD. By highlighting these specific components of the Loyola
GA experience, McLean hopes that prospective GA candidates will walk away from Preview
Days having a better understanding of their fit at Loyola as well as the unique aspects of the
DSDs vision for the GA experience.
When assessing the level at which these learning outcomes occur, it is important to
highlight the assumptions that can be made about the DSDs approach to the Preview Days
program (listed in Appendix A). As we will discuss later, there is no formal requirement that
each participant attend all of the programs offered during Preview Days. Candidates may either
be taking part in interviews or choose to not attend various programs. By structuring the
Preview Days program in this way, the Division of Student Development (DSD) assumes that
participants will attend all applicable workshops and sessions during their time at Preview Days.
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 9
Various factors may lead to a participants decision to attend a specific session during their
experience, which may change their perspective on the usefulness or impact of the Preview Days
program on their choice to attend Loyola for graduate school. This assumes that participants will
be aware of all the opportunities at Preview Days and will know what sessions to attend in order
to better understand the Loyola Graduate Assistant (GA) experience.
In addition, the program assumes that potential GA candidates will have little to no prior
knowledge about Loyola University Chicago, Loyolas GA experience, and the city of Chicago.
In our examination of the Preview Days schedule and program offerings, discussed later
(Appendix B), many of the sessions are planned and presented to candidates with the intent to
educate candidates about the unique opportunities at Loyola and in Chicago. For example, there
is time set aside in the schedule to talk about the mission and vision of Jesuit education (Jesuit
Education 101: Jesu-WHAT?). When planning this program, the assumption is that the
participants posses little understanding about the Jesuit educational practices. Even though many
of the participants may not have knowledge of the Jesuit education model, those that have prior
knowledge about the topic may not experience the same impact as a result of the session. Using
this framework to guide the participants experiences during Preview Days, there is an
assumption that the programs presented will meet the needs and retain the interest of prospective
students. In its current format, students are not asked prior to Preview Days about the questions
they may have regarding Loyola and the Loyola GA experience. Previous evaluations have been
conducted, focusing on the logistics (communication, flow of the schedule, etc.) and the
programmatic aspects (various sessions) of Preview Days (Appendix D). Absent in previous
evaluations is a focus on the participants overall experience (in relation to the learning
outcomes) as well as an assessment of the information participants wish they would have
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 10
received during the process. The Division of Student Development (DSD) does use previous
evaluations (see Appendix C) to frame their conversations about individual sessions, however
the evaluations are used to understand how the content should be presented during Preview Days.
Format and Structure of Preview Days
The Preview Days program, in its current form, takes place over two days and is held at
Loyolas Water Tower and Lake Shore Campuses in late February (see Appendix B for previous
Preview Days schedules). Candidates are invited to attend if they are offered an interview by the
hiring supervisors throughout the Division of Student Development (DSD) and are scheduled
according to the number of interviews offered to the candidate. Typically, the Preview Days
program begins with the Higher Education program introduction and dinner on the first day of
the program and ends once the candidates interviews have completed on the second day.
During the second day of the program, interviews are conducted simultaneously with the
programs planned by the Preview Days Planning Committee (PDPC). Over the past three years,
the opportunities available to Graduate Assistant (GA) candidates consist of the buddy
program, Higher Education program information, workshops and panels, campus tours, and
socials, which are spread throughout the two days of Preview Days (see Appendix A).
The Buddy Program
Unlike other institutional programs, candidates are not required to attend the Preview
Days program in order to be considered for a Graduate Assistantship or admittance into the
Higher Education program. For the last few years, PDPC has paired current GAs with applicants
as a way to bridge the gap between individuals who attend the program and those that choose not
to attend. Current GAs are paired with six to ten GA candidates and are instructed to reach out
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 11
via email and act as a resource if GA candidates have any questions about either the recruitment
and selection process or about Loyola.
Higher Education Program Information
During the first day of the program is the Higher Education Program Overview &
Dinner, which usually takes place at Loyolas Water Tower Campus (Appendix B). During this
session, candidates hear from program faculty about the various components of Loyolas
program and are encouraged to interact with faculty and current students over dinner. Since an
outcome of the Preview Days program is to be introduced to Chicago, the Preview Days
Planning Committee uses meal times as a way to expose candidates to the various food offerings
in Chicago.
In addition to the information session on the first night of the program, Higher Education
students are encouraged to attend an optional class during on the second day of the program. In
the past, students have typically attended the Student Development Theory class, but are asked to
RSVP in advance since there are limited seats in the classroom. This offers candidates another
opportunity to interact with the faculty and understand the academic experience at Loyola.
Since many of the Graduate Assistant (GA) positions do not require students to be
enrolled in the Higher Education program, candidates from other programs (Social Work,
Pastoral Studies, Business, etc.) are invited to interview during the Preview Days program.
These candidates are invited to the activities on the first night of the program, but are not
required to attend and encouraged to reach out to their respective programs of study.
Workshops and Panels
Throughout the Preview Days program, GA candidates are invited to attend various
sessions that help them to understand the Loyola GA experience. Over the years, the overall
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 12
framework of the sessions has remained unchanged (Appendix B), even though the content
presented may look different from year to year. Starting with the first night of the program,
candidates participate in a panel of current masters students in the Higher Education program,
who speak to their experience with the academic components of the program. The Preview Days
Planning Committee (PDPC) works to gather a diverse set of students who have varying
experiences with the Higher Education program (full and part time, current GAs, post-
undergraduate experiences, etc.) in order to offer different perspectives about the program. In
addition to the panel on the first night, an additional panel is offered during the second day,
focusing on the Graduate Assistant (GA) experience and the relationship between GAs and
University personnel. During both panel discussions, prospective GAs are encouraged to ask
questions that help to clarify their intent to join Loyola as a GA.
In addition to the panels offered during the Preview Days program, GA candidates are
also invited to attend sessions and workshops about the Jesuit and social justice mission of
Loyola as well as the opportunities available to GAs living in Chicago. Beginning with the
Division of Student Development welcome, usually delivered by the Vice President for Student
Development, candidates are exposed to the Loyola student GA experience. As an introduction
to the Jesuit identity of Loyola and the Jesuit education model, Jesuit Education 101: Jesu-
WHAT? is presented by staff members from Campus Ministry. Since many candidates come to
Preview Days with little knowledge about Jesuit education, this session serves as a way for
students to understand the characteristics of Jesuit education and understand how Loyolas Jesuit
identity may frame their work at the institution. In conjunction with faculty members in the
Higher Education program and divisional staff, candidates are also engaged in a discussion about
social justice and how it informs professional practice at Loyola. Given that the Higher
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 13
Education program has a focus on social justice, this session has historically allowed candidates
to engage with each other in conversations about social justice and introduces them to the types
of conversations and material covered within the classroom.
Finally, the Moving to Chicago presentation offers candidates an opportunity to learn
about living options in Chicago as well as the opportunities available in the city. Since most
Graduate Assistants (GAs) will live off campus, this session is intended to provide information
about living in the city and is usually presented by GAs and the Division of Student
Development (DSD) staff who live in the neighborhoods surrounding Loyola. All of these
programs are intended to help candidates assess their fit at Loyola as well as educate them
about the mission of Loyola.
Campus Tours
During the Preview Days program, participants have many opportunities to tour the Lake
Shore and Water Tower Campuses. During these tours, led by current GAs, candidates are
exposed to the physical layout of Loyolas campuses and are introduced to the various campus
resources graduate students use frequently. During the tour, candidates are able to see the
various facilities and become somewhat familiar with the campus. This is another opportunity
for participants to interact with current GAs and gain a better feel for the campus environment at
Loyola.
Socials
In addition to the Higher Education Program Overview and Dinner on the first night,
candidates are given the opportunity to interact with undergraduate and graduate students, DSD
staff, and faculty members. In previous evaluations, the Preview Days Planning Committee has
observed that candidates have requested more time set aside to meet with the members of the
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 14
Loyola community. Throughout the past few years, these social aspects have been included into
meal times, structured settings (Social with Grads and Undergrads), and throughout the
interview process. The Preview Days Planning Committee has worked to make sure candidates
can have their questions answered by staff members and current Graduate Assistants (GAs)
throughout out the process. DSD staff operate a Hospitality Booth throughout the second day
of the program and current GAs are able to volunteer to mingle with candidates in the interview
preparation space to calm nerves and serve as a resource.
External Factors
In addition to the assumptions made by the DSD about the achievement of the learning
outcomes, there may be various external factors, which may lead to obstacles and disruptions in
the execution of Preview Days (Appendix A). Since the program has seen an increase in the
number of candidates from outside the Chicago area, the cost of attendance has been a factor to
which Jack McLean and J. Curtis Main are sensitive (personal communication, September 25,
2014). Many candidates travel via different forms of transportation, which may be cost
prohibitive or highly susceptible to changes in weather. Since Preview Days normally takes
place in late February, candidate travel may also be impacted by the cold and unpredictable
weather in Chicago.
The extent that the Preview Days program impacts a candidate may also be affected by
their previous beliefs and assumptions about Loyolas program. For example, if a candidate
labels Loyola as more desirable than another institution (assuming their process includes
multiple institutions), they may respond more favorably to the experiences during Preview Days.
In contrast, candidates who may be on the fence may look at the Preview Days experience
more critically in order to assess their interest in the Higher Education program and GA
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 15
experience. In addition to these external factors, institutional factors may positively or
negatively impact a candidates experience. The programs outcomes may or may not be met
depending on the availability of University resources (physical, monetary, and/or human), for the
Preview Days program. The institutional climate (demographics, facilities, etc.) may not be
desirable for candidates, which could impact their view of the institution and their perceived fit.
Finally, a candidates choice to attend Preview Days or attend Loyola may be impacted by the
changes to assistantship positions and stipends. Within the next two years, the Division plans to
increase stipend amounts while decreasing the total number of assistantships across the Division,
which may impact the number of candidates who attend Preview Days (Appendix M).
Stakeholders
Before understanding our approach to the evaluation of Preview Days, it is important to
understand the various stakeholders involved in this program. Wholey, Hatry, and Newcomer
(2010) explain that stakeholders are individuals, groups, or organizations that can affect or are
affected by an evaluation process or its findings (p. 31). In our conversations with Jack
McLean, JD (personal communication, September, 25, 2014), it is clear that the primary
stakeholders for this program are the prospective GA candidates, Higher Education program
administrators, and divisional staff that plan and execute the Preview Days program.
Specifically for the Division of Student Development (DSD), our evaluation of the Preview Days
program directly influences the prospective GA candidates. As Loyola aims to be more
competitive with their processes and GA offerings, it is important for the (DSD) to evaluate the
program and implement changes that reflect candidates needs. If candidates needs are not met,
they will likely not attend Preview Days or apply for open GA positions. This relationship is
also important for the members of the Higher Education administration. Even though there is not
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 16
a formal requirement to hold a GA at Loyola in the graduate program, the Preview Days
experience is an important piece in a students decision to attend Loyola. This is not the only
factor, but must be taken into consideration as we begin to form and create an evaluation plan for
the program (Jack McLean, JD, personal communication, September 25, 2014).
Finally, the personnel in the Division of Student Development who use the process to
recruit GA candidates would also be impacted by an evaluation of the program. Much like the
Higher Education program faculty, the Preview Days experience is an important step in selecting
the most desirable candidates for the open Graduate Assistant (GA) positions. As we mentioned
earlier, the absence of this streamlined and cohesive process caused confusion for GA candidates,
which may have persuaded them to consider other institutions for their graduate studies. If
candidate needs are not taken into account or met, hiring supervisors may not have a reason to
recruit their GAs through this process (or hire a graduate student for the position at all). As we
begin to understand the design of our evaluation, it will be important to consider these
stakeholders throughout the entire process.
Evaluation Approach
Our evaluation will be formative in nature as we seek to provide information that could
be used to make improvements. A formative evaluation is defined as one whose primary
purpose is to supply information for the improvement of a program (Fitzpatrick, Sanders,
Worthen, p. 20, 2010). Based on our conversations with stakeholders, we will be using a
combination of process and outcomes approach to the evaluation of Preview Days. The reason a
process approach is helpful when evaluating Preview Days is due to the purpose of a process
study. Such studies may focus on whether the program is being delivered according to some
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 17
delineated plan or model or may be more open-ended, simply describing the nature of delivery
and the success and problems encountered (Fitzpatrick et al., 2010, p. 26).
Since we are looking into the effectiveness of Preview Days we must look at the various
panels, workshops, and events that occur throughout the program. Given that a process study is
based on the delivery of a program and also takes into account the success and difficulties of the
program, this approach would be advantageous to the needs of the Preview Days program. An
outcome approach is defined as one that takes into account the changes in an audience as a result
of a program (Fitzpatrick et al., 2010). As we have previously mentioned there are several
learning outcomes that aid in describing the purpose of Preview Days. Since the nature of our
evaluation is focused on the delivery and execution of Preview Days we use the learning
outcome that McLean expressed to help guide our evaluation. The main focus of Preview Days
according to McLean is to paint a picture for participants of the culture here at Loyola and within
the DSD. By providing participants with this, Preview Days hopes to help participants discern
whether or not Loyola is a good fit for their graduate studies. In evaluating Preview Days and
its execution, it is vital that we keep this learning outcome in the forefront as we progress
through the evaluation.
The combination of process and outcomes approaches is appropriate as we seek to
address the effectiveness of the Preview Days program and the value it offers to the Loyola GA
experience. Since the process aspect looks at the delivery of the program, we will be looking at
the influence of individual components in relation to the learning outcomes to expose
participants to Loyola and DSD in order to assess whether or not Loyola will be a good fit for
a candidates graduate studies. As the outcome approach examines changes in an audience of a
program, we will be surveying those individuals who attended Preview Days. This also
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 18
addresses the learning outcome of exposing participants to Loyola and DSD. The strength of this
dual approach is that the process aspect lends itself to looking at both the individual components
and their execution in relation to Preview Days as a whole. The outcomes aspect of this
approach illustrates the effectiveness of Preview Days and allows to us to see the affect such a
program has on its participants. This way if any changes need to be made we can find out what
exactly needs to be change and how to make the necessary adjustments in order to ensure a better
delivery.
Although both process and outcome approaches provide strengths to this evaluation, there
are also weakness to each aspect. Even though the process approach lends itself for a closer
examination of individual components of the Preview Days program, it does not take into
consideration outside influences and variables that potentially affect each piece. The challenges
of an outcome approach is the willingness participants to speak to their experiences and the
potential for varying answers from year to year that would leave have an effect on any changes
that would need to be made. Taking this into consideration, we will explain the various
components and approaches to our evaluation plan.
Quantitative Approach
Population and Sampling
Our intended population of study will be the candidates in attendance during the Preview
Days (PD) program. We selected this group because the focus of this evaluation entails
observing a change within a selected population as a result of attending a program. By surveying
the candidates, we can begin to evaluate the delivery and effectiveness of Preview Days. Since
Preview Days is designed and executed with the candidates needs in mind, it is also important to
note that they are one of the primary stakeholders. Their input and feedback will provide insight
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 19
into any changes that need to be made and other areas of improvement. Since Loyola aims to
become more competitive in the search and hiring process of Graduate Assistant (GA)
candidates, the information gained by this survey will prove beneficial.
As a result of surveying the candidates who attend Preview Days, we will conduct census
sampling. On any given year, there are about 50-75 candidates who attend Preview Days. The
reason we have selected census sampling is that we are choosing all the individuals who attend
PD. In this case, since those attending PD will have their email accounts already on file, it will
be easy to reach out to them and send out invitations. Another reason that census sampling is
best serves our evaluation is because the design and implementation of our evaluation focuses on
the experience of the individual participant throughout PD. One of the strengths of using census
sampling is that it allows for researchers to select participants who are able to speak to their
experience, in this case attending PD, and will also be able to answer the questions posed in the
survey surrounding their experience. We define our population as people who attend PD in order
to look at specific outcomes that arise as a result of their attendance. When using census
sampling, one weakness can be the limited ability to make generalizations and conclusions due
to the small scope of the population sample. Additionally, another limitation to our survey is the
assumption that all the participants that are in the intended survey sample will answer the survey
in a timely manner presents a weakness and potential challenge to our evaluation. Our
anticipated response rate is at least 50 percent, which reflects our realistic goal based on previous
response on the assessment for Graduate Assistant professional development sessions. Even
though this number is less than the optimal achievement 70 percent identified by Wholey et al.
(2010), it will still yield results that will allow us to make conclusions about the Preview Days
program.
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 20
Design and Instrument
We will be utilizing a descriptive cross-sectional design in our evaluation of the Preview
Days program. As Creswell (2009) notes this is a pre-experimental design where we study the
participants and view the effects the Preview Days program has on them afterwards. It is our
belief that cross-sectional best serves our needs as we seek to distribute the survey once all the
candidates have finished attending Preview Days. Cross-sectional studies measure a specific
point in time, in this case at the conclusion of PD, in order to accurately evaluate the delivery of
a program. As we collect our responses, one of the limitations to this research design is the
validity of responses entered. Our challenge in using a cross-sectional design is that we are not
controlling any previous knowledge, experiences, or expectations surrounding Loyola and
Preview Days, which would hinder the strength to which we are able to assess the impact PD had
on the participants. Our intention is to deliver the survey as quickly as possible with frequent
reminder emails (Appendix G) to allow our participants easier time recalling their experience.
We hope to engage our participants in a deeper reflection in their experience through interviews
in our qualitative analysis that shall be discussed later. Our quantitative approach reflects the
assumptions outlined in our logic model (Appendix A) that our participants enter PD with little
or limited surround Loyola and the GA experience. Our goal in administering this survey is to
evaluate the extent to which the outcomes, given the assumptions, are reached within the
delivery of the PD experience.
A majority of our survey focuses on the extent to which participants believe they were able to
achieve the outcomes of the PD experience. This is assessed through a set of statements that
asked participants to rate their level of agreement using a Likert scale (See Appendix E for
survey instrument). Our Likert scale allows our participants to choose between five choices
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 21
including: strongly agree, agree, neither agree or disagree, disagree, and strongly disagree. One
of the strengths of the implementation of a Likert scale is that it allows us to assess general
trends and distribution of the data collected. It also paints a general picture of the level of
agreement respondents had towards the statements we made. Our challenge in using a Likert
scale is that the difference between each choice in the response scale is dependent on the readers
understanding of the scale itself and statement made. Since we are assessing the experience of
PD, we are concerned with the distribution of the responses rather than the average response
given that a numerical value would not have much significance. Another concern of this
approach may be that participants may pick the extremes or the middle of the response scale in
interest of time. We hope to mitigate this concern with the use of a large response rate and
evaluation of the descriptive statistics the survey produces.
We are also utilizing open-ended questions to provide a unique space for participants to voice
their opinion and clarify their responses. Unlike the Likert scale, participants are able to
articulate their response in a more complex and unique way that allows researchers better
understand their experience. Our open-ended questions ask respondents to identify any lingering
questions, experiences that were valuable or could have been improved, and a space for
comments and suggestions. A limitation to using these types of questions is the complexity and
level of detail that different participants may employ. Variety, detail, and the unpredictability of
responses can also be a limitation as it may be a burden on researchers to analyze quickly.
Additionally, to many open-ended questions can lead lower responses rates and incomplete
answers as survey fatigue may set in (Schuh & Associates, 2009). The responses to these open-
ended questions will be compiled and given to stakeholders in order to make more informed
decisions surrounding any changes or improvements to the program.
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 22
Pilot Testing and Implementation
In the implementation of our current form of the survey, it is best to pilot test it in order to
ensure the success of our instrument. We plan on pilot testing our survey with current GAs who
attended Preview Days. Our goal is to gain feedback from at least ten people in order to ensure
that our questions are readable and that the participants do not have any confusion regarding both
the questions and responses. Another reason for pilot testing our survey, as noted by Creswell
and Associates (2009) is determining if we need to make any improvements in the questions we
ask and if there are areas where we should add or remove questions. In pilot testing our survey,
we will also be testing our delivery method, administering the survey through the web in an
email format. The current GAs who will be pilot testing the survey for us will be asked to take
down any notes or suggestions that they see fit and email those suggestions once they have
completed the survey. If we need some clarity on their comments and suggestions then we will
email them to follow-up.
The timing of the distribution of the survey as noted by Creswell (2009) involves multiple
waves and reminders. The survey will be administered by the Division of Student Development
(DSD) Graduate offices email and that office will serve as the main contact. All the emails will
be sent out using the DSD email of [email protected], and the first wave will be an invitation
email sent out the following Wednesday at the conclusion of Preview Days (Appendix G). The
tentative date of the first email will be the 25th of February, and the first reminder will be the
following Monday March 2nd. The subsequent reminders will be as follows: March 4, March 6,
and March 9, with the survey closing on March 11. These reminders are important as they help
encourage the participants to complete the survey. An added benefit of distributing the survey
through email is that it requires minimal effort in accessing the survey, as it will already be
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 23
linked to the participant. Two limitations to distributing the survey through email is the potential
for an incorrect email being used and user error in accessing the survey.
Analysis and Results Presentation
All the data we gather from the survey be organized and describe. The only statistical
analyses we will be using will be descriptive univariate statistics. These kinds of statistics are
those intended to summarize information on a single variable (Wholey et al., 2010, p.
455). As previously noted, many of the questions are tied to an outcome listed in our logic
model (Appendix F) and will provide interval data. For our demographic and general
information questions we will be analyzing the data using frequency of responses. For our Likert
questions we will be analyzing the data using means and medians to help determine the
frequency and distribution of responses. Along with using means and medians to help describe
the data, we will be using ANOVA and r-square statistics in order to measure the interaction
among variables and the significance of variables to their full experience. In addition to these
statistics we will also run a Chi-square test to determine if there is a statistically significant
difference between the Preview Days experience and their demographic characteristics. We
anticipate using SPSS as our statistical software to translate the data into these measures (which
is available on University computers).
We will be presenting these results in our final report using tables and figures that will
contain both the questions and the corresponding scores. There will also be an executive
summary to help the reader grab a big picture of the results and make it easier for them to
comprehend the results. In the executive summary there will be information containing our
purpose, several points that the reader should take notice of, and our own recommendations as
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 24
part of the evaluation. Our recommendation will be a result of our own interpretation of the data
collect and organized.
Qualitative Approach
In combination with the survey conducted at the conclusion of the Preview Days program,
we will conduct interviews with participants in order to better understand the achievement of our
outcomes (Appendix A). Utilizing a qualitative approach will allow us to better understand the
experience of candidates and explain phenomena that are presented during the administration of
our survey. Recognizing that our quantitative approach attempts to describe the experience of
Preview Days participants as a whole, our qualitative measure will help identify the nuances of
each students experience with the program. Our qualitative approach will use semi-structured
interviews in order to engage participants in a reflection about their experience and identify their
interaction with specific aspects of the Preview Days experience. Semi-structured interviews
combine directed interview questions with supplemental questions (known as probes) that ask
the participant to further explain their experience (Wholey et al., 2010). This approach allows
the interviewer to engage with the participant and clarify specific experiences that may be
essential to understanding an individuals experience with the Preview Days program.
We identified a semi-structured interview approach for the Preview Days program
because of the ability to conduct interviews easily while being able to have the freedom to ask
follow-up and probing questions to better understand their experience. In contrast to focus
groups, semi-structured interviews are conducted with one participant at a time (Wholey et al.,
2010). Even though focus group interviews allow participants to interact with each other and
speak about shared and individual experiences, semi-structured interviews will allow us to
quickly identify and interview participants without creating a focus group with participants
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 25
across the country. Using this one-on-one interview format, we will be able to easily schedule
interviews with participants and control for any technical difficulties that may arise from
conducting a focus group via telephone or videoconference.
Participants
The participants for the interviews will be selected from those who complete the survey
administered after the conclusion of Preview Days. While participants are completing the survey,
they will be asked if they would be willing to participate in an interview regarding their
experience. If they select yes, they are asked to provide an email address that they can be
contacted in order to schedule the interview. After the survey closes on March 11th, we will
begin to contact participants in order to schedule interviews for mid-April to mid-May. This
timeframe will allow us to analyze the data and edit our interview protocol, as appropriate. We
anticipate interviewing five Preview Days candidates and will compensate interview participants
with a $10 Amazon.com gift card.
Given that our interviews will last approximately 30 minutes, our sample size was
selected in order to conduct the interviews and analyze the data in order to make changes and
improvements to the next Preview Days program. The size of our sample for our qualitative
approach is a limitation to our assessment. Since we are not interviewing candidates that exhibit
a specific characteristic and relying solely on self-selection, we limit the generalizability of our
findings. However, we hope to use the interviews to not only understand the experience of
candidates during Preview Days, but to bring light to phenomena that may present itself during
the analysis of the quantitative data. Additionally, this evaluation component was chosen in
order to quickly gather information and recommendations and results in order for the Division to
consider any changes to the Preview Days program. If the Division of Student Development
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 26
wishes to better understand the experience of candidates who exhibit a specific characteristic
(demographic, programmatic, etc.), the qualitative approach would be the place to better
understand those variables. Our goal is to use the interviews in order to enhance the
interpretation of the quantitative data, while providing tangible examples to explain areas of
improvement for the program.
Participants will be selected randomly once the email addresses have been collected from
the survey instrument. Each email will be assigned a random number and will be placed in
ascending order. Once this has been completed, we will begin to contact participants in order of
their assigned number. When participants accept the invitation to be interviewed, we will work
to schedule their interview and send them the consent form (Appendix I), which outlines the
purpose of the interview. Additionally, the consent form will include demographic information
(similar to the information included in the survey instrument), which will be used during data
analysis after the interviews have been conducted. If participants decide to decline the offer for
the interview, we will select the next participant on the randomized list.
Interview Protocol
An interview protocol (Appendix H) will be used to conduct the 30-minute semi-
structured interviews. The overall structure of the interview will include an introduction, a set of
three foci with probing questions, and a conclusion. Prior to the interviews, we will conduct a
pilot test of our questions with current Graduate Assistants. Much like the quantitative approach,
pilot testing will allow us to make sure that the questions are constructed without any confusion
and will draw out directed responses. During the introduction of the interview, the interviewer
will explain the rationale of the interview and ensure that the participant has electronically signed
and returned their interview consent form (Appendix I). The interviewer will also reiterate that
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 27
the participants answers will be kept confidential and will not be traced back to their name or
candidate file. Since we are conducting these interviews during the timeframe when
assistantship offers are typically made, it is important that we ensure that the participants can be
honest with their responses, without any effect on their assistantship offer status. This timeframe
does present a limitation, but our goal is to ensure that the information provided during
interviews will not be traced back to their candidate file.
After the interviewer has explained the interview and asked for the participants verbal
agreement for continuing the interview and audio recording, the interviewer will ask the
questions outlined in the protocol. Given that we only have 30 minutes to complete the
interview, we have focused our time for the participant to answer three foci (all open-ended) with
additional probes provided for the interviewer. The interviewer will not be obligated to ask all of
the probing questions, but will be provided a list of probing questions in order to ensure that
participants are providing the desired response. Including the probing questions also allows
participants to provide more focused responses that will aid in our ability to analyze and interpret
the data. Our foci are intended to speak to our outcomes outlined in our logic model (Appendix
A), with specific emphasis on the medium and long-term outcomes. After the interviewer has
asked the three focused questions, they will be instructed to thank the participant and offer to
answer any questions about the interview. Additionally, participants will be asked to confirm
their email so that they can receive their compensation.
Implementation and Analysis
A current graduate assistant in the Division of Student Development (DSD) who is
recruited by the Preview Days Planning Committee will facilitate the interviews for this portion
of the assessment plan. Utilizing a current graduate assistant helps ensure the validity of the
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 28
responses because they are minimally involved in the hiring process for Graduate Assistants and
also able to relate to participants experiences during Preview Days. In partnership with the DSD,
the selected Graduate Assistant will work to schedule the interviews. All interviews will be
conducted via telephone and the interviewer will be asked to take notes regarding any major
themes or ideas that emerge during the interview (Appendix J). All notes will be collected in
order to be analyzed by the researchers, but will not weigh heavily in the final interpretation of
the data.
All interviews will be audio recorded, saved electronically, and transcribed by a graduate
student who will receive compensation (see budget in Appendix O). Once the interviews have
been transcribed, we will then engage in the process of descriptive coding in order to analyze the
data. Descriptive coding allows researchers to create categories as they examine transcripts,
while using initial codes developed in relation to the outcomes of the program (Wholey et al.,
2010). The initial codes we have developed (Appendix L) relate specifically to the learning
outcomes outlined in the logic model (Appendix A) and will be amended if specific phenomena
appears from the quantitative survey that becomes of interest to the DSD. Utilizing descriptive
coding for the interviews is helpful since this is the first time the DSD will take on a qualitative
approach, which involves coding. Descriptive coding allows the researchers to be flexible with
their coding structures and focus more on the specifics described in the interview. This type of
coding will be time intensive for the researchers doing the coding and those interpreting the
results (Wholey et al., 2010), but will help the DSD create a coding structure for future
interviews. We will also compare the candidates information with their choice to accept of
decline an assistantship using back-end coding once we have compiled all of the transcripts and
information.
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 29
After the transcripts have been coded, we will comb through the data using pawing in
order to observe the themes in the data as they emerge. As we examine the development and
interaction of the codes and the data, we will be able to look for specific themes and re-analyze
the data to identify other areas where those themes may emerge. Once we have examined the
data from the quantitative and qualitative measures, we will ensure validity through a process of
triangulation. As Wholey et al. (2010) explain triangulation helps to limit the bias that results
from relying on one source of information to make conclusions. Because we are asking
questions in both instruments about the outcomes and goals of the Preview Days program, we
can match the results of the evaluation approaches to identify any inconsistencies. If the data
does result in any inconsistencies, we will flag them and find ways to evaluate those phenomena
in future evaluations (Wholey et al., 2010). In addition to triangulation, we will also perform
member checking with the participants of our interviews. Member checking involves taking
initial interpretations findings, and descriptions back to the participantsto determine whether
they are accurate from the participants point of view (Schuh & Associates, 2009, p. 169). This
will ensure that our observations match the experience of the participants. Once the interviews
have been examined and validated, the conclusions will be paired with the quantitative data,
using direct quotes in order to support the conclusions within the final report.
Limitations
This evaluation plan will allow the Division of Student Development and the Preview
Days Planning Committee valuable information in order to make changes to the Preview Days
program to meet the needs of the Division and assistantship candidates. Even though the intent
of this evaluation is to aid in the execution of the Preview Days program, we must acknowledge
that there are a few limitations to our approach. Since we will be conducting the survey using an
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 30
online platform, our response level (50 percent) will be lower than what is optimal for
quantitative instruments (70 percent; Wholey et al., 2010). This will limit the generalizability of
our findings; however, we believe the 50 percent response rate will still let us make general
conclusion about the experiences that candidates have during Preview Days. We will hope to
increase our response rate through announcements during Preview Days and multiple reminder
emails after the program.
In addition to the response rate on the survey instrument, our small sample size for our
interviews presents further limitations. We only anticipate interviewing five participants, which
is about less than ten percent of the total participants in the Preview Days program. This size
was selected given that the DSD does not have any current qualitative evaluation structure for
the Preview Days program. Additionally, we are looking to gather results and make
recommendations before planning for the next Preview Days program. Selecting a small sample
size will be realistic and will provide quotes and observations that can be added to the
quantitative results.
Budget
As outlined in Appendix O, our total anticipated cost for this evaluation plan is $350.
The DSD already has access to CampusLabs and printing services, which will help deliver the
survey and print any paper materials and final report. The bulk of our budget will be spent on
the surveys in the qualitative plan. Participants will be compensated for their participation in the
interview with a $10 Amazon.com gift card. We anticipate the total cost of gift cards to be $50
and they will be distributed via email. Since our qualitative plan requires transcription and
coding, we will recruit a current graduate student to complete this step of our plan. Our
estimates indicate that it will take approximately 10 hours for the students to transcribe the two
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 31
and a half hours of audio recording (audio recorder provided by digital media services).
Additionally, we estimate coding to take a similar amount of time. We will pay our graduate
students $15 per hour, which we believe to be an acceptable incentive to complete the
transcription and coding. The Division may be able to explore options for students to perform
this work as part of an internship or summer project, if they wish to be cost conscious about the
evaluation program.
Timeline
Throughout the evaluation plan, a timeline has been outlined in order to provide further
direction for the evaluation of the Preview Days program. Our specific timeline is further
explained in Appendix N. After the completion of our evaluation plan, we will discuss the
findings with Jack McLean and J. Curtis Main to discuss our findings and understand the
feasibility of the study. We anticipate meeting with Jack and Curtis in December in order to
understand how to implement the evaluation plan within the Division. Once we have met with
Jack and Curtis, we will begin pilot testing our survey and interview protocol in order to
understand if any further changes need to be made to our plan. Based on this feedback, we
anticipate completing our survey instrument at the beginning of February, in anticipation of
Preview Days, which takes place later in the month. Once Preview Days is completed,
participants will be emailed the survey (with subsequent reminders) and be instructed to
complete the survey before March 11.
Once the survey is completed, we will email those participants who indicated that they
were open to being interviewed (after random selection is completed). We will schedule the
interviews and hold the interviews during April and May. The data will be transcribed and
coded after the completion of the interviews. Initial conclusions will be made based on the
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 32
evaluation results and sent to the participants who were interviewed for member checking. Once
we have received information back from the participants, we will compile the information into
the final report and inform the DSD of our findings and suggestions for changes to future
Preview Days programs.
Next Steps
As was mentioned earlier, we will meet with Jack and Curtis to discuss our findings and
understand the feasibility of our evaluation plan. We will send Jack and Curtis our plan
electronically and anticipate scheduling a meeting face-to-face with them to understand their
general reactions to the plan and any considerations we should take moving forward. Our hope
would be to assist the Division with this evaluation (in any capacity) and extend our help for the
planning and implementation of Preview Days 2015.
If the Division wishes to continue with this evaluation plan, it should consider finding
ways to perform an evaluation of students who are offered interviews but who choose not to
attend Preview Days. This information will allow the Division to assess any barriers to
attendance and better tailor the program to be available to a wider audience. Since the scope of
this evaluation plan is the execution of the existing program, we did not address this area in our
plan. Finally, the Division should also work with Preview Days stakeholders in order to better
target the quantitative and qualitative questions and understand how each office perceives the
Preview Days experience.
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 33
References
Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed methods
approaches. Thousand Oaks, California: SAGE Publications.
Division of Student Development (n.d.). Graduate assistantships. Retrieved from:
http://www.luc.edu/studentdevelopment/assistantships/
Fitzpatrick, J. L., Sanders, J. R., Worthen, B. R., & Worthen, B. R. (2011). Program evaluation:
Alternative approaches and practical guidelines. Upper Saddle River, NJ: Pearson
Education.
Schuh, J. H. & Associates (2009). Assessment methods for student affairs. San Francisco, CA:
Jossey-Bass.
Weiss, C. (1998). Evaluation: Methods for studying programs and policies (2nd ed.). Upper
Saddle River, N.J.: Prentice Hall.
Wholey, J. S., Hatry, H. P., & Newcomer, K. E. (Eds.) (2010). Handbook of practical program
evaluation (Third Edition). San Francisco: Jossey-Bass.
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 34 Appendix A: Logic Model
Program: Graduate Assistant Preview Days Logic Model
Inputs Outputs Outcomes -- Impact Activities Participation Short Medium Long Prospective Gradate Assistants Time Resources: -Housing -Food -Preview Days guide -Facilities -Food Personnel: -Current Graduate Assistants (GAs) -Committees -Divisional staff -Hiring supervisors -Higher Education program faculty
Buddy System Interviews Programs: -Socials -Campus tours -Jesu-What? -Social justice introduction -Divisional overview -Classroom Experience -Workshops and panels
Prospective GAs Current GAs Divisional staff Higher Education program faculty Hiring supervisors
Understand and articulate the Division of Student Developments GA expectations and experience. Gain a better understanding of the assistantship offerings and experiences at Loyola University Chicago. Be able to select most desirable assistantship experience (specific department).
Accept an offer for a GA position at Loyola. Discerning whether Loyola is a good fit for the candidate.
Their Loyola GA experience matches their expectations as a result of attending Preview Days. Sharing their experience with their network of peers and professionals. Deeper level of investment and a stronger sense of community among GAs.
Assumptions: 1. Students will attend Preview Days and attend all applicable
workshops during Preiview Days 2. Prospective GAs will have little to no prior knowledge of Loyolas
GA experience and the city of Chicago 3. The programs chosen and presented will meet students needs and
help students discern institutional fit. 4. The Preview Days programs will be able to keep students interest
throughout the experience 5. University and Divisional partners are willing to take part in the
Preview Days experiencea
External Factors: 1. Prospective GAs expectations of the Preview Days experience 2. Number of attendees and cost of attendance 3. Availability of staff and resources (specifically monetary) 4. Weather and travel obstacles 5. Divisional and University power structure and politics 6. Institutional demographics (staff and students) 7. Existing interest in Loyolas programs and GA experience 8. Higher Education enrollment and application process 9. Reduction of assistantship offerings
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 35
Appendix B: Previous Preview Days Program Schedules
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 36
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 37
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN
38 Appendix C: Preview Days 2010 Evaluation Findings
Logistical Information
What is your current educational status? Current graduate student at Loyola 6 Do not plan on attending Loyola in Fall 2010 4 Incoming graduate student, Fall 2010 5 Still deciding on graduate school 9
How did you travel to Loyola?
I am local to the Chicago area and drove or took public transportation. 12
I drove to Loyola from out of town. 3 I flew into O'Hare/Midway. 8 Train 1
The logistical information (schedule, directions, map) was received in a timely manner. Agree 12 Disagree 1 Strongly agree 8 Strongly disagree 3
I feel that the logistical information provided prepared me for Loyola's preview day. Agree 14 Disagree 2 Strongly agree 8
Scheduling my interview(s) for March 3, 2010 was an easy process. Agree 10 Disagree 2 Strongly agree 12
I was well informed of my interview schedule. Agree 11 Disagree 2 Strongly agree 10 Strongly disagree 1
Comments
The timeliness of receiving the information about the two days could improve. The interview format was confusing and not explained, and the communication about the interviews was very poor. Many of us did not know if we were only interviewing with one or all of the assistantships we applied for, in addition to
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN
39 who the main contact person was for the interviews and explaining that process.
I feel that the process of receiving responses was very confusing. We were given contradictory information as to when and how we would hear back from GA providers. This could be more clear in the future
Lodging
Lodging information was provided in a timely manner.
Agree 9 Disagree 3 Strongly agree 9 Strongly disagree 3
Did you stay with a student host or in Baumhart Hall?
No 22 Yes 2
Was the housing registration process effective?
Yes 2 Were you able to contact and sync up with your host(ess)/check in to Baumhart Hall in a timely manner? Yes 2 Was your host(ess)/residence hall accommodating during your stay? Yes 2 Did you know there was an option to stay with a current student/residence hall on Tuesday evening? No 11 Yes 11 Had you known, would you have utilized this option? No 9 Yes 2
Comments I was told not to count on student housing accommodations, and I did not receive any follow up once I
made an inquiry. I wish I would have heard further in advance because I made other arrangements. It would have been nice to have had internet access in Baumhart Hall.
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN
40 School of Education Reception Did you attend the reception? No 11 Yes 13 Why didn't you attend? I had class during the reception time. 6 I was not aware of the reception. 1 I was still in transit to Chicago/Loyola. 4 The day of the reception worked well with my schedule (Tuesday) Agree 5 Disagree 1 Strongly agree 7 The time of the reception worked well with my schedule (4:30 p.m. - 6:00 p.m.) Agree 6 Disagree 1 Strongly agree 6 The length of the reception was sufficient to obtain the information I needed (1.5 hours) Agree 7 Strongly agree 6 The dinner following the reception was beneficial. Agree 7 Strongly agree 6 I had ample time to talk with Students Students Staff Faculty
Agree 4 6 8
Disagree 1 2 1
Strongly agree 8 5 4
Learning about the faculty and their research was beneficial to better understand the higher education program. Agree 6 Strongly agree 7
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN
41
I felt welcomed at this event. Agree 4 Disagree 2 Strongly agree 7 *No comments provided for those that disagreed. Ironically, they agreed/strongly agreed with all other components of Tuesday evening. Please indicate your level of agreement with the following statements: - My experience on Tuesday left me with a good overall impression of the Loyola academic program. Agree 5 Disagree 1 Strongly agree 7
Comments:
Tuesday was wonderful! To improve it though, I wish there was a panel of student including some who did not have assistantships and how they were coping economically
It was a good night, highly informative and scheduled well. I did not realize the importance of attending the event on Tuesday evening. After attending other Preview
program, I believe meeting the professors and getting to know more a about the program could have impacted my decision on whether to attend Loyola. I also thought the Interview day would provide me with this information but sadly that was what the night before was all about.
I would have liked to know more about the class option before I arrived. I did not know about it and would have planned to attend had I known.
I would have liked to talk to more current students in the program. I thought it was an excellent way to start prior to the GA interview day following the reception. It provided
some opportunities to get to know a few of the other applicants, which helped reduce some nerves. I would continue to do a dinner reception in the future.
I felt we should have had more time to speak with current students. No one really moved around at the social; would it be possible to have dinner in one location and the social
in another? Prospective students just remained in their seats and it was really awkward to try to speak to a professor or staff member who was sitting at a full table.
Great help and made me feel at ease about the whole process and people in the program It was a great evening. It really provided me with a hands-on look at the academic program. This truly
enabled me to make a better overall judgment of whether or not the program was right for me.
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN
42 Social Justice Class Did you attend the Social Justice class? No 20 Yes 4
By attending this class, I was able to gain a better
understanding of the higher education curriculum. Strongly agree 4
I felt welcomed in the class.
Strongly agree 4
I felt my time was well spent attending class. Strongly agree 4
Comments:
Loved it-loved the way the professor expected us to participate-and we did!
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN
43 Wednesday: Interview Day
Please rate how effective the following logistical elements of interview day were:
Location (Lake Shore Campus) Effective 11 Ineffective 1 Very effective 12 Day of interview (Wednesday) Effective 12 Ineffective 4 Very effective 8 Interview time (e.g., 10:00 a.m., 11:00 a.m.) Effective 14 Ineffective 1 Very effective 9 Interview duration (30 minutes) Effective 13 Ineffective 1 Very effective 10 Length of day (8:00 a.m. - 4:30 p.m.) Effective 14 Ineffective 5 Very effective 4 Very ineffective 1 Room layout Effective 10 Ineffective 8 Very effective 5 Very ineffective 1 The following presentations were beneficial to my understanding of Loyola University Chicago Welcome (Dr. Rob Kelly) Agree 9 Strongly agree 15 Icebreaker Agree 4 Disagree 12 Strongly agree 6 Strongly disagree 2 What is Loyola and the Jesuit Mission? (Jake Jacobson) Agree 8 Disagree 1
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN
44 Strongly agree 14 Strongly disagree 1 The Division of Student Development (Dr. Rob Kelly) Agree 11 Strongly agree 13 Moving to Chicago Information Session (Cliff Golz) Agree 16 Disagree 2 Strongly agree 4 Strongly disagree 2 Panel of Current Graduate Assistants Agree 18 Disagree 1 Strongly agree 4 Strongly disagree 1 I was able to efficiently fill my time between interviews. Agree 8 Disagree 6 Not applicable 1 Strongly agree 6 Strongly disagree 3 The amount/type of programming offered between interviews was sufficient. Agree 4 Disagree 10 Not applicable 2 Strongly agree 5 Strongly disagree 3 Comments:
A tour around campus, the student affairs offices, etc. A more structured way to meet/visit with everyone More on the classes, getting to know the prospective students as well as the current students in a more
familiar basis More structure for what do while waiting for interviews Campus tour! After going through this survey, I realized I signed up for interviews prior to receiving the schedule.
Because of this, I missed a session I would have preferred to attend, moving to Chicago. In the future it may serve the applicants better to see the structure of the day prior to being offered interviews. That way, applicants will see the structure/layout of the day and may be able to better accommodate their needs.
What to do if you do not have an assistantship For someone who is already a Loyola graduate student, none of the day's events aside from the interview
were helpful; however, I can see their relevance to those who are not already enrolled.
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN
45 More information about the program - I am still trying to make sure I find a good fit school and program
wise. The dates of the reception and interview days were very inconvenient. I missed two days of class in order
to travel and attend the events. Holding these events later in the week, such as a Thursday or Friday, would be much more convenient than a Tuesday and Wednesday. I was not able to attend the panel of current students.
More interaction with current students and faculty. The conference room for us to sit in between interviews was too small and it was crowded. Great day. The presentation on What is Loyola and the Jesuit Mission? was quite overwhelming. It was too pushy
and religion focused. I really love and appreciate Loyola's mission, it's truly inspiring, but for non-Christian and especially atheists and agnostics the presentation was too strong. I believe it scared my non-Christian peer.
At a glance
Logistical information should be provided earlier for students to make flight arrangements and properly schedule interviews.
We should avoid scheduling sessions (i.e. Off-Campus living, grad panel) during interview times or hold them multiple times throughout the day. Students may have scheduled their interview not knowing about the session and then had to miss the session to interview.
We need to do a better job scheduling interviews for folks that have multiple interviews so they can be closer together. Maybe we can centralize this?
Providing a program for first-year Loyola students could be beneficial but does not appear to be required. Would be helpful to better communicate expectations to first-year students.
Offering lodging with current students would be helpful but not required. Finding graduate students to host students is very difficult.
The Tuesday evening reception was very well received! Only improvement would be to have people mingle more rather than staying seated. If current student, faculty, or staff were at a full table, it was difficult for prospective students to speak with that person.
The option to attend the Social Justice class was also very well received! Art did a fantastic job integrating the students into discussion and making them feel welcome.
Interview day: Overall well received. Improvement could be seen with the downtime students had. Offer campus tours, have graduate students on staff to speak with perspectives.
Could we hold the interviews in CFSU either in Bremner or in individual offices? Also, use the Hague as a gathering area for folks between interviews?
Can we offer campus tours or coffee with current grads during the downtime?
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 46
Appendix D: Preview Days 2013 Evaluation Instrument
Page - 2013 Division of Student Development Graduate Assistantship Preview Days Evaluation We would greatly appreciate your feedback on what we could do to improve the Preview Days experience. Please take a few minutes to tell us what you think!
Required answers: 0 Allowed answers: 0 Q1 Is Loyola the only Interview/Preview Day program for which you will participate? Yes[Code = 1] No (What other institutions have you/will you visit?)[Code = 2] [Textbox]
Required answers: 1 Allowed answers: 1 Q2 Which Preview Days session(s) did you attend on Sunday, February 24, 2013? (Check all that apply) Water Tower Campus Tour[Code = 1] Higher Education Orientation Dinner[Code = 2] Current Graduate Student Panel[Code = 3] Optional activities (please specify)[Code = 4] [Textbox] None of the above[Code = 0] [N/A]
Required answers: 1 Allowed answers: 5 Q3 Which Preview Days session(s) did you attend on Monday, February 25, 2013? (Check all that apply) Breakfast[Code = 1] Welcome, Introductions, Professional Development Opportunities[Code = 2] Overview of the Day[Code = 3] Jesuit Education 101: Jesu-What?[Code = 4] Moving to Chicago[Code = 5] Lake Shore Campus Tour(s)[Code = 6] Informal Social[Code = 7] Lunch[Code = 8] Social Justice Literacy[Code = 9] Grad Research: Scholar and Practitioner[Code = 10] Staff Panel[Code = 11] Lake Shore Campus Tour(s) #2[Code = 12] Optional Class: Student Development Theory[Code = 13] None of the above[Code = 0] [N/A]
Required answers: 1 Allowed answers: 14 Q4 Did you take advantage of the on-campus housing accommodations offered during Preview Days? Yes[Code = 1] No[Code = 2]
Required answers: 1 Allowed answers: 1
Next Page: Sequential
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 47
Page - 2 How would you rate the housing accommodations on the following components?
Q5 Information received regarding housing accommodations prior to your arrival Excellent[Code = 5] Good[Code = 4] Satisfactory[Code = 3] Poor[Code = 2] Unacceptable[Code = 1] Did not attend/Does not apply[Code = 0] [N/A]
Required answers: 1 Allowed answers: 1 Q6 Check-in process Excellent[Code = 5] Good[Code = 4] Satisfactory[Code = 3] Poor[Code = 2] Unacceptable[Code = 1] Did not attend/Does not apply[Code = 0] [N/A]
Required answers: 1 Allowed answers: 1 Q7 Room provided Excellent[Code = 5] Good[Code = 4] Satisfactory[Code = 3] Poor[Code = 2] Unacceptable[Code = 1] Did not attend/Does not apply[Code = 0] [N/A]
Required answers: 1 Allowed answers: 1 Q8 Check-out process Excellent[Code = 5] Good[Code = 4] Satisfactory[Code = 3] Poor[Code = 2] Unacceptable[Code = 1] Did not attend/Does not apply[Code = 0] [N/A]
Required answers: 1 Allowed answers: 1 Q9 Parking Excellent[Code = 5] Good[Code = 4] Satisfactory[Code = 3] Poor[Code = 2] Unacceptable[Code = 1] Did not attend/Does not apply[Code = 0] [N/A]
Required answers: 1 Allowed answers: 1
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 48
Display if Q4='Yes' Q10 Why did you not take advantage of the on-campus housing accommodations offered during Preview Days? [Code = 1] [Textbox]
Required answers: 0 Allowed answers: 1 Display if Q4='No'
Please rate the following aspects of your Preview Days experience:
Q11 Communication with DSDGrads Excellent[Code = 5] Good[Code = 4] Satisfactory[Code = 3] Poor[Code = 2] Unacceptable[Code = 1] Did not attend/Does not apply[Code = 0] [N/A]
Required answers: 1 Allowed answers: 1 Q12 Information you received regarding Preview Days Excellent[Code = 5] Good[Code = 4] Satisfactory[Code = 3] Poor[Code = 2] Unacceptable[Code = 1] Did not attend/Does not apply[Code = 0] [N/A]
Required answers: 1 Allowed answers: 1 Q13 Division of Student Development Graduate Assistantship Website Excellent[Code = 5] Good[Code = 4] Satisfactory[Code = 3] Poor[Code = 2] Unacceptable[Code = 1] Did not attend/Does not apply[Code = 0] [N/A]
Required answers: 1 Allowed answers: 1 Q14 Online application process Excellent[Code = 5] Good[Code = 4] Satisfactory[Code = 3] Poor[Code = 2] Unacceptable[Code = 1] Did not attend/Does not apply[Code = 0] [N/A]
Required answers: 1 Allowed answers: 1 Q15 Division of Student Development Staff Excellent[Code = 5] Good[Code = 4]
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 49
Satisfactory[Code = 3] Poor[Code = 2] Unacceptable[Code = 1] Did not attend/Does not apply[Code = 0] [N/A]
Required answers: 1 Allowed answers: 1 Q16 Current Graduate Students/Assistants Excellent[Code = 5] Good[Code = 4] Satisfactory[Code = 3] Poor[Code = 2] Unacceptable[Code = 1] Did not attend/Does not apply[Code = 0] [N/A]
Required answers: 1 Allowed answers: 1 Q17 Food selections Excellent[Code = 5] Good[Code = 4] Satisfactory[Code = 3] Poor[Code = 2] Unacceptable[Code = 1] Did not attend/Does not apply[Code = 0] [N/A]
Required answers: 1 Allowed answers: 1 Q18 Session locations Excellent[Code = 5] Good[Code = 4] Satisfactory[Code = 3] Poor[Code = 2] Unacceptable[Code = 1] Did not attend/Does not apply[Code = 0] [N/A]
Required answers: 1 Allowed answers: 1
Next Page: Sequential Page - 3 Preview Days Sessions on Sunday, February 24, 2013 How valuable were the following aspects of your Preview Days experience?
Q19 Water Tower Campus Tour Extremely valuable[Code = 5] Very valuable[Code = 4] Moderately valuable[Code = 3] Not very valuable[Code = 2] Not valuable at all[Code = 1] Did not attend/Does not apply[Code = 0] [N/A]
Required answers: 1 Allowed answers: 1
-
GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 50
Display if Q2='Water Tower Camp