Paper ID #7538

Measuring intercultural sensitivity: A case study of the REU program atUPRM

Dr. Saylisse Davila, University of Puerto Rico, Mayaguez Campus

Dr. Davila research interests includes the application and development of data mining methods in theearly detection of anomalies. She is currently working on the development of a variety of methods in-volving feature selection and pseudo-permutation tests in the early detection of disease outbreaks. Hercurrent work also targets high-dimensional approaches to characterize anomalies with applications topublic health surveillance and statistical process control. Other interests for Dr. Davila include provid-ing under-represented populations with research opportunities at an undergraduate level. To date, shehas mentored more than ten undergraduate students, and worked as the co-principal investigator for theResearch Experiences for Undergraduates (REU) site at University of Puerto Rico at Mayaguez. As adoctoral student, Dr. Davila received prestigious fellowships from the National GEM Consortium/IntelCorporation, Tau Beta Pi, and Awards for Research College Scientists. She is also member of the Instituteof Industrial Engineers and honor societies such as Golden Key, Alpha Pi Mu, Tau Beta Pi, and OmegaRho.

Dr. Viviana I. Cesani, University of Puerto Rico, Mayaguez Campus

Dr. Viviana Cesani is a professor of Industrial Engineering at the University of Puerto Rico at Mayaguez(UPRM). She completed her Ph.D. degree in Manufacturing and Production Systems at the Universityof Wisconsin-Madison in 1998. Her areas of interest in teaching and research include production plan-ning and control, supply chain management, engineering economy, project management, and engineeringeducation. She is currently the department head of the IE department at UPRM. Dr. Cesani is a seniormember of IIE, President of the UPRM-Delta Chapter of the International Organization for Women Ed-ucators, and member of the Professional College for Engineers and Land Surveyors of Puerto Rico. Shewas recognized as UPRM Distinguished IE professor in 2003 and 2010, and as Distinguished IndustrialEngineer for the Year 2010 by the College of Engineers and Land Surveyors of Puerto Rico.

Dr. Alexandra Medina-Borja, University of Puerto Rico at Mayaguez

Alexandra Medina-Borja earned her Ph.D. and M.S. in Industrial and Systems Engineering from VirginiaTech. and holds a Production Engineering degree from the Federal University of Sao Carlos in Sao Paulo,Brazil. Medina-Borja has concentrated her work in areas related to the effective design and analysis ofservice delivery systems. Her main research contribution has been to advance a model for the performanceevaluation of nonprofit social services by adapting Data Envelopment Analysis formulations to accountfor the effects of the operating environment on units functioning under different market and geograph-ical conditions. In 2002, the American Red Cross adapted Medina-Borja’s performance measurementframework to evaluate chapter operations, thus making it one of the largest centralized nonprofit evalua-tion systems in the world. A paper relating this experience based on her doctoral dissertation work wasawarded the 2007 Goodeve Medal by the British Operational Research Society for the most significantcontribution to the theory and practice of OR in the given year. In 2006, she founded the InternationalService Systems Engineering Research Lab in the Department of Industrial Engineering at the Univer-sity of Puerto Rico at Mayaguez where she is an associate professor. Through ISSER she has been ableto secure more than $1.5 million in federal funding for her research, mainly from the National ScienceFoundation (NSF) and mainly to study the inclusion of behavioral and cultural considerations in modelingof large service systems. She is a member of the System Dynamics Society, IIE, INFORMS, and AlphaPi Mu and a board member of the Society for Engineering and Management Systems. In January 2012,Medina-Borja started a rotation at NSF in Arlington, Va., where she was appointed as the first director ofevaluation and assessment for the Directorate for Engineering. In this new role, Medina-Borja’s respon-sibilities include designing and implementing a state-of-the-art evaluation and assessment system for thewhole directorate that encompasses close to 100 programs in five different divisions with a budget of 800

c©American Society for Engineering Education, 2013

Page 23.890.1

Paper ID #7538

million dollars. Outcome indicators collected through this system will be used to demonstrate the impacton society of NSF’s investments in engineering research. She was nominated for the UPRM EngineeringDistinguished Faculty Award in 2006 and won the award in 2007, 2008 and 2011.

c©American Society for Engineering Education, 2013

Page 23.890.2

Measuring intercultural sensitivity: A case study of the REU program at UPRM

Abstract The National Academy of Engineering’s call to educate global engineers has sparked a number of globalizing programs and coursework at different institutions. Nevertheless, there is a need to quantify the extent to which the ability of recent graduates to understand, appreciate, and accept differences among cultures is being influenced by globalizing programs and coursework. In other words, how do these emerging programs improve the intercultural sensitivity scale of engineering graduates? In 2009, NSF funded a Research Experiences for Undergraduates (REU) site at the University of Puerto Rico at Mayagüez (UPRM) named Educating a culturally-sensitive Industrial Engineer: A complex interdisciplinary systems perspective to global IE issues. The summer component of this REU site takes advantage of the unique location of Puerto Rico, with a distinct Hispanic-Caribbean culture and a highly industrialized manufacturing environment, to expose intercultural teams of participants to some of the challenges associated with working in a global economy. In addition to research activities, this REU includes cultural and professional activities aimed at providing students with an opportunity to foster their culturally-sensitive management skills. By the end of the summer internship, students gained not only critical scientific knowledge and expertise in different research areas, but also the experience of working in intercultural teams. Nevertheless, did this exhaustive intercultural program have an effect in the students’ intercultural skills? Are there any differences in the way it impacted students coming from US-based institution versus the way it impacted UPRM students? To answer all these questions, we used the Intercultural Sensitivity Scale (ICSS) by Chen and Starosta1 on a cohort of 43 students from the 2010-2012 UPRM-REU summer programs. Feature selection, using analysis of variance, indicates that the factors that contribute more heavily toward changes in the students’ intercultural sensitivity are (1) the experience of going through the REU summer program and (2) whether the student went to school at UPRM or not. Both of these factors also have a statistically significant two-way interaction with the different ICSS constructs. Non-parametric paired analyses were used to test whether the experience had a positive effect in the participant’s intercultural scale. At a 0.05 significant level, every REU summer program was able to trigger a statistically significant improvement in the interaction confidence and interaction enjoyment constructs of the ICSS. Overall, the REU program also had a statistically significant (𝛼 = 0.05) effect in the interaction engagement and respect for cultural differences constructs of the ICSS. Lastly, results indicate that the UPRM program has been responsible for the overall improvement in the intercultural sensitivity scale of 21 participants of the REU program. Background In the mid-1990s, the University of South Australia introduced the concept of graduate qualities (GQs). These graduate qualities, according to this institution, are the inherent qualities that all graduates must possess. Among these qualities, they outlined the ability of a student to demonstrate an international perspective as a professional and as a citizen.2 In 2007, the American Society for Engineering Education started a report based on a year-long discussions

Page 23.890.3

within the Society on the role and importance of educational scholarship to ensure the long-term excellence of U.S. engineering education. This report3, released in 2012, was entitled “Innovation with Impact: Creating a Culture for Scholarly and Systematic Innovation in Engineering Education” sought to catalyze conversations on creating an engineering academic culture for scholarly and systematic innovation to ensure that the U.S. engineering education enterprise keeps pace with changes in the engineering profession and in the world. The report recognizes among many other topics that engineering careers have become increasingly collaborative, multidisciplinary, entrepreneurial, and global; and, as the pace of change of technology has accelerated, the expectations for engineering education have expanded including all sorts of professional skills beyond disciplinary depth which include team building skills with a global perspective. The report also recognizes that while good intentions are the driver for a lot of faculty-driven innovations in classrooms, the issue is a need for more educational innovations that have a significant impact on students. The report advocates widespread and efficient implementation of proven practices, including those that prepare students for a globalized workforce. Proven practices being those that have documented statistically significant improvements in a skill-set but also improvement in the satisfaction and attitudes of the students that experienced the intervention. Clearly, developing students with an international perspective requires institutions to adjust their curriculum to the current needs of graduates joining a global workforce. In 1997, Knight4 described this as “integrating an international/intercultural dimension into the teaching, research and service functions of the institution.” Shortly after, the National Academy of Engineering (NAE) Committee on Engineering Education (CEE) got on board in this call to arms. Its report entitled The Engineer of 2020: Visions of Engineering in the New Century5 discusses plausible scenarios of the challenges facing the engineering profession by 2020. One of the main challenges being that the new engineering workforce must be prepared to interact with consumers, employees, colleagues, and leaders from a variety of cultural backgrounds. Furthermore, the NAE Engineer of 2020 report emphasizes the need not only for a continued emphasis on soft and hard skills, but newer emphasis on business and leadership skills, creativity and ingenuity, and abilities in change readiness and management.5 ABET’s A-K6 student outcomes required for engineering program accreditation outlines in outcome h that programs need to provide “the broad education necessary to understand the impact of engineering solutions in a global, economic, environmental, and societal context” of which one could argue national culture is certainly a societal context affecting the implementation and consequent impact of engineering solutions. But are industrial engineers ready; either because of their experience or education, to deal with evermore evident sociological aspects of engineering brought about by economic globalization? According to NAE’s 2007 executive summary to Congress7, in order to sustain its share of high-technology jobs, America must prepare for a new wave of change for which engineering is essential only if it is able to continue to adapt to the above new trends and educate the next generation of students so as to arm them with the tools needed for the world as it will be, not as it is today. In 2009, the NSF funded a Research Experiences for Undergraduates site at the Department of Industrial Engineering at the University of Puerto Rico-Mayagüez (UPRM) to address this important question.

Page 23.890.4

The UPRM REU site, “Educating a culturally-sensitive Industrial Engineer: A complex interdisciplinary systems perspective to global IE issues,” has two main goals. As any other REU program, it provides funding for undergraduate students to participate more actively in research activities. In particular; this research program addresses cross-disciplinary problems for which the intellectual focus centers on utilizing IE tools and analytical techniques such as simulation, statistical analysis, network analysis, system dynamics, survey design and survey research, linear programming, and qualitative data analysis to solve complex problems. However, at its title implies, what makes the UPRM REU site unique is its strong intercultural focus aimed at: (1) providing an opportunity for students to work in intercultural teams and present their work at an international conference, and (2) promoting the development of culturally-sensitive management skills. In this work, we will use an instrument called the “Intercultural Sensitivity Scale”1 to measure the impact of the REU’s intercultural component on its participants. In the REU program at UPRM section, we will provide a description of the cohort of students we used in this work. The REU site intercultural component section discusses some of the intercultural activities that are used to promote the development of culturally-sensitive management skills in the participants of the REU program. In the Intercultural sensitivity section, we will provide further information on this instrument as well as a formal definition on intercultural sensitivity. On Results, we will discuss the impact of each of the summer programs in the intercultural sensitivity scale of its participants. We will provide results at different levels of aggregation. That is, we will measure the impact of each REU summer program at the different constructs of the intercultural scale and the intercultural sensitivity scale as a whole. In addition, we will validate the internal consistency of the ICSS on our sample. Finally, in the Conclusions section, we will provide some concluding remarks on the implications of our statistical analysis.

Assessing the impact of international experiences

While there are several international REUs, there is little published information on formal evaluations of the impact that this international research experience had on students. There is much more information on study abroad programs in which students spent at least a semester or 18 weeks abroad taking classes that count towards their degrees. This is a different setting than a 6-10 weeks summer REU focused on engineering research and intercultural teaming, and, therefore, we are only mentioning it as a reference on the available findings in the literature. For example, Parkinson8 presented the results of a survey of engineering study abroad programs that gave light to some best practices and assessment methods of undergraduate international experiences. Lohmann et al.9 described a quasi-experimental research effort to measure the effectiveness of study abroad programs using an instrument developed by the Intercultural Communications Institute10. The Handbook of Intercultural Competence11 acknowledges, however, that developing reliable instruments for this complex construct is challenging due to the influence of so many external factors and recognizes the efforts of few institutions, including Lohmann et al.9, in measuring impact. Further, Deardoff11 is an advocate of triangulation in which assessment instruments are complemented by exit interviews and other data collection methods. All published interventions had some degree of improvement in intercultural competence, but they did not use the same constructs or instruments. P

age 23.890.5

There is also at least one report that evaluates the impact of international research experiences on faculty. Yating et al.12 evaluated three constructs: technical impact, professional impact, and global/transcultural impact of NSF’s international research experiences programs (IREE). Two of the most interesting findings in the transcultural construct are that the program in fact encourages the innovative development of “shared work space” to accommodate cultural difference and fuels the emergence of best practices for transcultural collaborations. These authors provide a series of lessons learned, but, of course, the subjects of research have different set of conditions and characteristics that substantially differentiate their experiences from that of undergraduate students.

REU program at UPRM The REU program at UPRM targets undergraduate students in the field of industrial engineering and/or operations research from any US-based institution, including the University of Puerto Rico at Mayagüez. Participants are selected based, not only on their academic merits, but also considering their cultural background. For detailed program application requirements refer to Figure 1.

Since its establishment in the fall semester of 2009, the REU site at the University of Puerto Rico at Mayagüez (UPRM) has run year round. This REU site has served 57 Industrial Engineering (IE) undergraduate students with a staff of 14 distinct IE professors as mentors. During the spring and fall semesters, all participants were recruited from the IE undergraduate program at UPRM. Nonetheless, the REU summer program consisted of a blend of local students and students from a variety of institutions within the continental United States (US). Every summer program, intercultural research teams were created by pairing a local student with a student from a US-based institution. In most research teams, there were also graduate students working along with REU students, performing a mentoring role. Most UPRM students completed two terms in the REU program, one in the summer and the other one either in the spring or fall semester. Students from US-based institutions completed only the summer term in the program, which is strictly the term of the REU program on which we are focusing in this work. Nonetheless, both local students and students from US-based institutions were sponsored to present their research projects at conferences such as the Industrial and Systems Engineering Research Conference (ISERC), INFORMS Annual Meeting, and Great Minds in STEM.

Page 23.890.6

1 - Be currently enrolled in an Industrial Engineering or Operations Research bachelors degree-granting program.

2 - Have successfully completed their sophomore year and have at least one more semester left.

3 - Have a minimum GPA of 3.00 on a 4.00 scale.

4 - Complete the online application process at the UPRM REU website.

5 - Submit an official transcript and two letters of recommendation from professors or professional mentors.

6 -Write an essay showing genuine interest in pursuing research and embracing an intercultural experience.

Figure 1 UPRM REU Site application requirements

Please refer to Figure 2 for a summary of the number of participants and mentors involved in the REU program from fall 2009 through summer 2012. In this period, the program impacted thirty-nine students from UPRM and one student from each of eighteen different US-based institutions. Refer to Figures 3 and 4 for a breakdown of the different ethnic backgrounds of the REU participants and their home institution. Notice that Figure 3 shows an unusually large number of participants with a Latin background, and that is due to the fact that throughout the academic year, the program only recruits students currently enrolled in the IE program at UPRM. The REU summer program, nevertheless, consists of roughly 50% students from the continental US and 50% students from UPRM.

Figure 2 REU staff and participants for the years 2009-2012

Page 23.890.7

Figure 3 REU participant ethnic background Figure 4 REU participant home institution REU site intercultural component In order to develop culturally-sensitive management skills, this REU combines a series of technical workshops/industry visits with a variety of socio-cultural activities. On the first day of the program, the students play the Barnga intercultural awareness game.13 In this game, the students are exposed to scenarios characterized by having different people play by different rules, which leads to a discussion on how this dynamic can be applied in the context of intercultural interactions. Within the first weeks of the program, all students participate in a workshop on Puerto Rican cultural scripts. This workshop is addressed to immerse the US students in the Puerto Rican culture. In addition; the students participate in a variety of cultural and scientific excursions including international exhibits at the Ponce Art Museum, guided tours of El Yunque National Rainforest; and trips to Old San Juan, the National Astronomy and Ionosphere Center Arecibo Observatory, and the research laboratories from the UPRM Marine Sciences Department at Isla Magueyes. To complement the experience, the students are also exposed to a variety of industrial settings, such as IBM, Hewlett Packard, Stryker Corporation, Amgen, Ethicon, among others. During these plant tours, the company’s designated host discusses the challenges of working in a global environment and the use multicultural project teams from an engineering and human resource perspective. Furthermore, some of these companies also provide students with interactive technical workshops on IE topics such as the Lean workshop at Amgen. In addition, some of our REU summer students had to the opportunity to participate in other socio-cultural activities such as the Centro American and Caribbean games, which took place in the city of Mayagüez in the year 2010. Similarly, the 2012 participants were able to participate in several workshops, sociocultural activities, and industry visits in the company of a group of IE students and professors from Texas A&M University (TAMU), who visited our campus during the fourth week of the 2012 REU program.

Page 23.890.8

Intercultural Sensitivity The importance of the study of intercultural communication competence has been increasing because of the impact of globalization on human society. According to Portalla and Chen14, intercultural communication competence (ICC) can be conceptualized as an individual’s ability to achieve their communication goal while effectively and appropriately utilizing communication behaviors to negotiate between the different identities present within a culturally diverse environment. ICC is comprised of three dimensions, including intercultural awareness, intercultural sensitivity, and intercultural effectiveness.15

Intercultural awareness represents the cognitive process by which a person comes to know about their own and others’ cultures. Intercultural sensitivity is the affective aspect which not only represents the ability of an individual to distinguish between the different behaviors, perceptions, and feelings of a culturally different counterpart, but also the ability to appreciate and respect them as well.16 Intercultural effectiveness dictates the behavioral aspect of ICC, which refers to the ability to attain communication goals in intercultural interactions.

Research studies from different disciplines have tried to conceptualize the three ICC dimensions (intercultural awareness, intercultural sensitivity, and intercultural effectiveness) and, in order to do so, several standardized instruments were developed17-21. However, the study of ICC continues to be plagued with problems of conceptual ambiguity and the lack of valid instruments for measuring the concept14. Although more and more scholars have made an effort to reduce this confusion11, 15, more research is still needed.

While there is increased interest in promoting intercultural management skills in engineering graduates, there is only a limited number of instruments in the literature that actually measure the impact of programs leading to improving these skills. One of these few instruments available is the one developed by Chen and Starosta1 called the Intercultural Sensitivity Scale. According to its authors, the Intercultural Sensitivity Scale (ICSS) measures the affective aspect of intercultural communications. Intercultural sensitivity is, then, “the active desire that motivate themselves to understand, appreciate, and accept differences among cultures.” 1 The authors make it clear that intercultural sensitivity is a closely related but different construct than intercultural awareness and intercultural communication competence.

Intercultural sensitive persons have the ability to integrate themselves into a new environment, achieving a level of dual identity, being capable of enjoying themselves gradually overcoming the denial or concealing of cultural differences that according to the authors’ research has shown people facing a new culture experience. To achieve this, the authors propose that individuals need to exhibit six affective elements: self-esteem, self-monitoring, open-mindedness, empathy, interaction involvement, and suspending judgment. The authors created 73 questionnaire items to measure these affective elements and used factor analysis to reduce the dimensionality of the questionnaire. They came up with five factors or constructs with eigenvalues of 1 or higher and only kept items with loadings of at least 0.50 with secondary loadings no higher than 0.30. This procedure lead to a 24-item questionnaire whose items can be grouped into the following constructs: (1) interaction engagement, (2) respect for cultural differences, (3) interaction confidence, (4) interaction enjoyment, and (5) interaction attentiveness. Refer to Table 1 for a complete list of the items included under each construct of the ICSS.

Page 23.890.9

In a latter study, the authors validated the instrument against a series of standardized questionnaires commonly used in the literature22-25. The Cronbach alpha26 reliability coefficient for the ICSS was 0.86, implying that higher scores of this measure are indeed suggestive of the ability of an individual to be more interculturally sensitive. One weakness recognized by Chen and Starosta1 is that the sample used to develop the instrument was almost in its entirety composed of white college students in the United States. Therefore, they make the disclaimer that they do not know whether the items would group in the same way if the sample would have been more heterogeneous.

Table 1 Intercultural Sensitivity Scale by Construct1

1 - Interaction Engagement I enjoy interacting with people from different cultures. I tend to wait before forming an impression of culturally-distinct counterparts. I am open-minded to people from different cultures. I often give positive responses to my culturally different counterpart during our interaction. I often show my culturally-distinct counterpart my understanding through verbal or nonverbal cues.

2 - Respect for Cultural Differences I would not accept the opinions of people from different cultures. I think people from other cultures are narrow-minded. I think my culture is better than other cultures. I don’t like to be with people from different cultures.

3 - Interaction Confidence I feel confident when interacting with people from different cultures. I am pretty sure of myself in interacting with people from different cultures. I find it very hard to talk in front of people from different cultures. I always know what to say when interacting with people from different cultures. I can be as sociable as I want to be when interacting with people from different cultures.

4 - Interaction Enjoyment I often get discouraged when I am with people from different cultures. I often feel useless when interacting with people from different cultures. I respect the ways people from different cultures behave. I respect the values of people from different cultures. I get upset easily when interacting with people from different cultures.

5 - Interaction Attentiveness I am very observant when interacting with people from different cultures. I try to obtain as much information as I can when interacting with people from different cultures. I am sensitive to my culturally-distinct counterpart’s subtle meanings during our interaction. I avoid those situations where I will have to deal with culturally-distinct persons.

Results By the end of the UPRM-REU summer program, students are expected to have gained not only critical scientific knowledge and expertise about particular areas of IE, but also the experience of

Page 23.890.10

working in intercultural teams. Unfortunately, due to the distinct nature of each of the summer research programs, there was no single instrument we could use to directly quantify critical scientific knowledge and expertise. Still, the UPRM REU projects were presented at national and international conferences for a total of 31 oral and poster presentations and 10 conference proceedings. In addition, UPRM REU projects were the topic of four refereed journal articles; and four REU participants received honors and awards such as: first place in the 2011 HENAAC Great Minds in STEM Technical Poster Competition, second place in the 2012 Emerging Researchers National Conference in STEM, and 1st and 3rd place in the Technical Paper Contest at the IIE 2012 Regional Conference. But, did this program have an effect in the participants’ intercultural sensitivity scale? In this section, we will discuss the Intercultural Sensitivity Scale (ICSS) questionnaire1 results obtained in a cohort of the REU summer participants for the years 2010 through 2012. Note that, since we only had intercultural teams during the summer, we strictly measured the change in the students intercultural sensitivity scale during this term. We administered the 24-item to the participants of the summer program. Those REU summer students willing to participate in the study completed the ICSS both at the start and at the end of the summer program. As the ICSS was validated on a homogeneous sample, we used Cronbach’s alpha26 to validate the internal consistency of the instrument on our non-homogeneous sample of students from the continental US and Puerto Rico. Cronbach's 𝛼 is defined as

𝛼 = 𝑛𝑛−1

�1 − ∑ 𝑉𝑖𝑖𝑉𝑡�, (1)

where, in our analysis, 𝑛 represents the number of items in the ICSS, 𝑉𝑖 is the variance of question 𝑖 in our sample, and 𝑉𝑡 is the variance of the observed total ICSS scores.26 Cronbach's alpha is a number between zero and one, and it commonly increases as the intercorrelations among test items increase. As intercorrelations among test items are maximized when all items measure the same construct, Cronbach's alpha is widely believed to indirectly indicate the degree to which a set of items measures a one-dimensional construct or the ICSS of the REU participants, in our case. Although the internal consistency of the instrument was slightly below the acceptable mark of 0.70 at the start of the 2010 REU summer program, it was at least acceptable for all other periods. In fact, its internal consistency was proven to be excellent at both iterations of the 2012 summer program. The overall analysis also shows a mark of acceptable for both the stages of the REU program. As a result, we can conclude that although our sample is non-homogeneous, the internal consistency of the instrument still holds.

Page 23.890.11

Table 2 Cronbach’s reliability coefficient for the internal consistency of the ICSS at the start and end of the REU summer program throughout the years 2010-2012

Year Start End 2010 0.645 0.709 2011 0.76 0.785 2012 0.95 0.904

Overall 0.732 0.713 After we proved the internal consistency of the ICSS, we proceeded to use the questionnaire information to analyze the responses of all REU summer participants at different levels of aggregation. First, we used a Wilcoxon signed rank test to evaluate whether for the group of REU participants, as a whole, there was a significant improvement at each of the constructs after going through the REU program. Results for this analysis are shown in Table 3. At a 0.05 significant level, interaction confidence and interaction enjoyment are significant for the years 2010-2012. The overall analysis also shows that interaction engagement and respect for cultural differences are also statistically significant at a 0.05 significant level. Nonetheless, interaction attentiveness was only statistically significant at a 0.05 significance level on the summer program of 2011. Table 3 Results for the Wilcoxon Signed Rank test at the construct level (1 = interaction engagement, 2 = respect for cultural differences, 3 = interaction confidence, 4 = interaction enjoyment, and 5 = interaction attentiveness).

Year Construct

1 2 3 4 5 Legend:

2010 0.0713 0.0605 0.0018 0.0492 0.1356 p-value significant at

2011 0.0045 0.2895 0.0002 0.0046 0.0022 0.1 2012 0.0234 0.0005 0.0049 0.0024 0.5859 0.05

Overall 0.001 0 0 0.0005 0.379 0.01 On the other hand, we used a paired sign test to determine how many students had a statistically significant improvement in their intercultural sensitivity scale. Detailed results for this analysis are shown for all 2012 summer participants. A summary of the number of participants with a statistically significant increase in their ICSS across the board is shown in Table 4. To date, the UPRM REU can be linked to a statistically significant improvement (𝛼 = 0.05) in the intercultural sensitivity scale of 21 of the REU summer participants (see Table 5).

Page 23.890.12

Table 4 Paired sign test results for testing increases in the ICSS of individual participants in the year 2012.

Student Group Gender UPRM student p-value 1 Puerto Rican Female Yes 0.2120 2 Asian Female No 0.5000 3 Puerto Rican Male Yes 0.0065 4 Caucasian Male No 0.0000 5 Puerto Rican Female Yes 0.0059 6 Caucasian Female No 0.2120 7 Puerto Rican Female Yes 0.2539 8 Puerto Rican Female Yes 0.0010 9 Puerto Rican Male Yes 0.0009

10 Caucasian Male No 0.0106 11 Hispanic, not from Puerto Rican origin Female No 0.0193 12 Puerto Rican Female Yes 0.1051

Table 5 Number of students with a statistically significant increase (𝜶 = 𝟎.𝟎𝟓) in their overall ICSS. The numerator in the last row shows the total number of students with a statistically significant increase in their overall ICSS, while its denominator indicates the total number of participants in the 2010 – 2012 REU summer programs.

Student 2010 2011 2012 NON-UPRM 1 5 3 UPRM 3 5 4 TOTAL 4 / 14 10 / 17 7 / 12

Lastly, we used an ANOVA analysis to determine the factors that most significantly influence the ICSS of the REU participants. In this case, we studied the factors described in Table 6. We considered the year of the REU program, whether the student went to UPRM or not, the participant’s gender, at which point of the program we collected the responses to the ICSS questionnaire, and the construct to which each of the questions belongs to. Table 7 shows all the factors and the two-way interactions that are statistically significant at a 0.10 significant level. Figure 5 shows the plots where we check the validity assumptions for the model’s residuals. After applying a power transform on the response, we can clearly see that the assumptions of normality, equal variance, and independence are satisfied. Figures 6 - 8 show relevant main effect and interaction effect results from the ANOVA analysis. In Figure 6, we see how the ICSS, on average, increases as the participants go through the REU experience. In Figure 7, we see how UPRM students score higher, on average, on all constructs except for interaction engagement. As the US students are the ones being fully immersed into a new culture, it only makes sense that they are trying harder to engage and adapt to the new culture. Finally, Figure 8 shows how the average score for each construct goes up as the students go through the REU experience. Note, however, that interaction engagement and interaction attentiveness increased at a lower rate than the rest of the constructs. P

age 23.890.13

420-2-4

99.9

99

90

50

10

1

0.1

Standardized Residual

Per

cent

40302010

4

2

0

-2

Fitted Value

Stan

dard

ized

Res

idua

l

43210-1-2

40

30

20

10

0

Standardized Residual

Freq

uenc

y

200180160140120100806040201

4

2

0

-2

Observation Order

Stan

dard

ized

Res

idua

l

Normal Probability Plot Versus Fits

Histogram Versus Order

Residual Plots

Table 6 Factors considered in ANOVA analysis: when the student participated in the REU summer program (Year), whether the student was a student at UPRM or not (UPRM student), the participant’s gender (Gender), whether the ICSS was measured at the start or end of the REU program (Period), and the ICSS construct (Construct).

Table 7 Results for the analysis of variance. Results indicate that the year of the REU intervention, the term of the REU program (start vs. end), whether the student was a student at UPRM or not, and the second order interactions between the ICSS construct and the factors period and UPRM student are statistically significant(𝜶 = 𝟎.𝟏𝟎).

Figure 5 Graphical check of the validity of the ANOVA assumptions for residuals. Results indicate that; after applying a power transform on the response, the assumptions of residual normality, constant variance, and independence are reasonable.

Factor Levels Values Year 2 2010, 2011, 2012 UPRM student 2 Yes, No Gender 2 F, M Period 2 (1) Start of REU, (2) End of REU Construct 5 1, 2, 3, 4, 5

Source DF Seq SS Adj SS Adj MS F P

Year 2 1379.45 1444.51 722.26 19.63 0.000 Period 1 3279.16 3279.16 3279.16 89.12 0.000

Construct*Period 4 777.44 777.44 194.36 5.28 0.000

Construct*UPRM Student 4 342.31 342.31 85.58 2.33 0.058

UPRM Student 1 146.92 146.92 146.92 3.99 0.047

Error 183 6733.29 6733.29 36.79

Total 199 16116.56

Page 23.890.14

Figure 6 Main effects plot for the REU period (1 = Start, 2 = End). Results indicate that the overall ICSS increases, on average, as the students go through the REU summer program.

Figure 7 Interaction plot for factors UPRM Student and Construct. Except for Interaction Engagement, UPRM students have a larger average score for all constructs of the ICSS.

21

35

34

33

32

31

30

29

28

27

26

REU Period

Mea

n

Main Effects PlotFitted Means

Page 23.890.15

Figure 8 Interaction plot for factors Construct, REU Period, and UPRM Student. Results indicate that the average score for all ICSS constructs increases as the students go through the REU summer program. Conclusions Although the ICSS was developed with a homogeneous sample of white students, the internal consistency of the instrument rendered acceptable on our extremely diverse sample. Overall, the six-week research summer program can be linked to a statistically significant improvement (𝛼 = 0.05) of the Intercultural Sensitivity Scale of 21 participants of the summer program. Also, it can be shown that the UPRM-REU program had a statistically significant improvement in all constructs of the Intercultural Sensitivity Scale except for interaction attentiveness. Students improved their perception about people from other cultures being narrow-minded and, in general, their confidence communicating in front of diverse audiences and all aspects of their social interaction. It must be noted that Puerto Rican students tended to be more tolerant regarding certain aspects of the cultural interaction, but; in general, were more afraid of communicating as the program was conducted in English and, while it is expected that all UPRM students are bilingual, fluency varies greatly across students.

Last but not least, we need to make an important disclaimer. The participants of the UPRM-REU program represent a self-selected sample. In particular, the US students are individuals that are willing to adventure outside their known realm and approximately half of them were already international travelers and/or were interested in learning foreign languages. As a result, we are aware that there is an implicit bias in our analysis. Future work In order to address the bias in the estimation of the effect of the REU program on the participants’ intercultural sensitivity, future work must consider additional factors that would allow us to better describe the nature of each participant. For instance, we might need to consider whether the student has some exposure to intercultural skill development. For example, we could consider factors such as whether the student has traveled internationally in the past, whether the student speaks more than one language, whether the student has had a previous

Page 23.890.16

internship experience, etc. In addition, we could also consider the student’s age and what is the student’s academic standing in his degree program as an assessment of how the maturity level of the individual influences his capability to become more culturally sensitive. Acknowledgements Funding for this research was provided by the NSF Research Experiences for Undergraduates site at the University of Puerto Rico at Mayagüez – “Educating the culturally-sensitive Industrial Engineer: A complex interdisciplinary systems perspective to global IE issues.” (Award Number EEC-0851879). References 1. Chen, G. M., & Starosta, W. J. (2000). The development and validation of the intercultural communication

sensitivity scale. Human Communication, 3, 1-15. 2. Barker, S. & Crichton, J. (2008). Assessing and evaluating intercultural teaching and learning: an information

management case study. Journal of International Education in Business, 1(1), 29-42. 3. American Society of Engineering Education (2012). Innovation with impact: creating a culture for scholarly

and systematic innovation in engineering education. June, Washington D.C. 77 p. 4. Knight, J. & De Wit, H. (Eds). (1997). Internationalization of higher education in the Asia Pacific countries.

Amsterdam: European Association of International Education, 5–19. 5. National Academy of Engineering, The engineer of 2020: visions of engineering in the new century (2004)

(NAE) http://www.ijee.dit.ie/ articles/Vol13-5/ijee996.pdf, accessed January 2005. 6. ABET (2011). 2012-2013 Criteria for accrediting engineering programs, Baltimore,

MD. http://www.abet.org/uploadedFiles/Accreditation/Accreditation_Process/Accreditation_Documents/Current/eac-criteria-2012-2013.pdf, accessed March 2013.

7. National Academy of Engineering. (2008). Grand challenges for engineering. Washington, DC: National Academies Press.

8. Parkinson, B. (2007). Engineering study abroad programs: formats, challenges, best practices. Proceedings of the American Society of Engineering Education Annual Conference, Paper 2007-422.

9. Lohmann, J., Rollins, H. & Hoey, J.J. (2006). Defining, developing and assessing global competence in engineers. European Journal of Engineering Education, 31(1), 119-131.

10. Intercultural Communication Institute, http://www.intercultural.org/kozai.php, accessed March 2013. 11. Deardorff, D.K. (Ed). (2009). The SAGE Handbook of Intercultural Competence. Thousand Oaks, CA: Sage, p.

370. 12. Chang, Y., Atkinson, D. & Hirleman, E.D. (2009). International research and engineering education: impacts

and best practices. Online Journal for Global Engineering Education, Vol. 4: No. 2, Article available at: http://digitalcommons.uri.edu/ojgee/vol4/iss2/1.

13. Thiagarajan, S. & Thiagarajan, R. (2006). Barnga: A Simulation Game on Cultural Clashes, 3rd Edition, Intercultural Press.

14. Portalla, T. & Chen, G.M. (2010). The development and validation of the intercultural effectiveness scale. Intercultural Communication Studies XIX, 3, 21-37.

15. Chen, G.M., & Starosta, W.J. (1996). Intercultural communication competence: A synthesis. Communication Yearbook, 19, 353-384.

16. Chen, G.M., & Starosta, W.J. (1997). A review of the concept of intercultural sensitivity. Human Communication, 1, 1-16.

17. Bennett, M.J. (1986). A developmental approach to training for intercultural sensitivity. International Journal of Intercultural Relations, 10, 179-196.

18. Chen, G.M., & Starosta, W.J. (1998). A review of the concept of intercultural awareness. Human

Page 23.890.17

Communication, 2, 27-54. 19. Hanvey, R.G. (1987). Cross-culture awareness. In L. F. Luce & E. C. Smith (Eds.), Toward internationalism

(pp. 13-23). Cambridge, MA: Newbury. 20. Turner, C.V. (1968). The Sinasina “big man” complex: A central cultural theme. Practical Anthropology, 15,

16-22. 21. Hammer M.R., Bennett, M.J. & Wiseman, R. (2003). Measuring intercultural sensitivity: The intercultural

development inventory. International Journal of Intercultural Relations, 27, 421–443. 22. Cegala, D.J. (1981). Interaction involvement: A cognitive dimension of communicative competence.

Communication Education, 30, 109-121. 23. Wheeless, E.W & Duran, R.L. (1982). Gender orientation as correlate of communicative competence. Southern

Speech Communication Journal, 48, 51-64. 24. Lennox, R.D. & Wolfe, R.N. (1984). Revision of self-monitoring scale. Journal of Personality and Social

Psychology, 46, 1349-1364. 25. Davis, M.H. (1996). Empathy: A social psychology approach. Boulder, CO: Westview. 26. Cronbach, L.J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16 (3), 297-334.

Page 23.890.18