The paper will be organized in the following manner. First, the development of the InCoRe model will be described. Next the theoretical and pedagogical underpinnings of the model will be shared. Third, the student survey methodology, data and results related to self assessment will be provided. Finally, a discussion of the results and implications will be offered.
INCORE MODEL
The shift toward embedding reflection in the internship program began in 2007. Faculty and administrators at the university were conducting a major audit of program outcomes in preparation for an accreditation visit when they discovered a lack of formal connectivity and student reflection from one academic year to the next between experiential, liberal arts, and professional outcomes within a group of undergraduate degree programs. They concluded that an opportunity existed to map outcomes from one year to the next, link these outcomes through designated milestones and foster student reflection on learning and outcomes attainment at each of these milestones. The idea gained further acceptance after a second group of university administrators refined the idea while working to improve the experiential learning programs at the institution. This resulted in the development of a perspective that student reflection during their internship experience should be outcomes-based whereby students consider the outcomes required of them during their internship while reflecting on the experience itself. Administrators, including the Dean of Experiential Education and the Associate Provost, and the Dean of the College of Business became champions of the idea and work began in earnest to develop a final set of internship outcomes and methodology to assess student self-perceptions of competency.
A hypothesis that learning will be enhanced through required intervals of student-based reflection evolved from the audit and an operational definition was adopted from the literature to assure a common understanding: "Reflection is the process through which students step back from their experience and consciously learn from that experience" (Bailey, Hughes, Moore, 2004, pp. 215).
Further discussions with members of the College of Business faculty and administration resulted in the creation of InCoRe. InCoRe is an acronym for integration, coordination and reflection. Along with the creation of the acronym, the authors created a model of the InCoRe concept that graphically illustrates integration, coordination, and reflection for a four year degree (see Figure 1). Note that this paper is focused on student reflection in relation to learning outcomes and does not provide a discussion of the integration and coordination aspects of InCoRe. InCoRe is a theoretical map for assuring intervals of reflection in a four year degree program across three dimensions; during courses in the major, courses in general studies and during work integrated learning experiences or internship. For InCoRe to function successfully, student learning and reflection must be coordinated across the degree-granting school or college, departments providing core general studies courses, and those that manage and coordinate work integrated learning.
InCoRe Model
INCORE THEORETICAL PERSPECTIVES AND PEDAGOGY
The intent of this section is to provide an overview of the major theoretical underpinnings of the InCoRe model. Careful consideration was given to the perspectives and theories of major thinkers who have written about program design from the fields of philosophy, instruction, and reflection. InCoRe is built, in part, from these perspectives. This section will highlight the major theorists that influenced the InCoRe model, starting with Spencer and moving to Bruner, Tyler, and Kolb. Discussion of the theoretical underpinnings of the model will help the reader understand the evolution and development of InCoRe and, more importantly, provide a broader view of curriculum theory.
The pedagogical origins of the InCoRe model are structural in nature and begin with the educational philosophy of Herbert Spencer and follow with the influential perspectives of Jerome Bruner, Ralph Tyler and David Kolb. In his seminal work, Education: Intellectual, Moral, Physical, Spencer (1891) proposed that knowledge should have some practical benefit to society in the development of thinking skills in the learner who undergoes an education:
Acquirement of every kind has two values— value as knowledge and value as discipline. Besides its use for guidance in conduct, the acquisition of each order of facts has also its use as mental exercise; and its effects as a preparative for complete living have to be considered under both these heads. (p. 37)
To Spencer, the proper design of a curriculum or learning experience provided both practical application and the development of thinking. He suggested that consideration be given not only to what to teach but how to teach it and for what purpose.
Writing nearly one hundred years after Spencer, Jerome Bruner (1960), attempted in part to answer Spencer"s essential question of "what knowledge is of most real worth" (p. 26) by adding perspectives on "how and when" teaching should occur. One of Bruner"s key concerns in exploring the question was teaching for transfer. Bruner proposed that learning should be active and that the active use of learning is a precursor to transfer. Like Spencer, Bruner felt that learning should move from simple to complex, and that such learning should be discipline-specific-building or layering skills and knowledge within a discipline over a period of time until fluency is established and the potential for transfer developed. Transfer, in this case, refers to the ability of a learner to use knowledge and skills within one context to navigate or solve problems within another similar or, in some cases, dissimilar context. Layered above the notion of transfer is an individual"s self efficacy or belief about his or her ability to produce specific levels of performance (Bandura, 1994) that come with experience, practice and expanded intellectual ability.
This concept serves as the root of what has come to be known as Bruner"s spiral curriculum: a curriculum that, over a period of time, moves from general to specific, simple to complex; from broad to specific. The authors acknowledge that not all learning fits neatly within the model. However, the model emphasizes a progressive approach to learning that fits well with the way students learn as they progress from one year or grade to the next.
InCoRe, as presented in Figure 1, is directly based on Bruner"s model of a spiral curriculum. Bruner"s (1960) view that "learning should not take us somewhere; it should allow us later to go further more easily" (p.17) supports the tenets of the InCoRe approach, and InCoRe has the potential to fulfill Bruner"s expectation. Bruner proposed that learning is an active and intentional process where learners develop new knowledge by building on current and prior knowledge. This implies that prior and current learning can be explicitly linked when appropriate, and this intentional linking is a potential method for improving learning.
The InCoRe model is also highly influenced by Ralph Tyler"s Basic Principles of Curriculum and Instruction (1949). Tyler is widely recognized in curriculum and teaching circles for the Tyler Rationale. The rationale proposes four questions that must be answered when developing a curriculum or program of study. These key questions are:
- What educational purposes should the school seek to attain?
- What educational experiences can be provided that are likely to attain these purposes?
- How can they be organized?
- How can we determine whether these purposes are being attained? (p. 1)
For InCoRe to function as intended, program outcomes have to be well established and written with precision in response to Tyler"s first question about the educational purposes a school, or in this case, a program of study, should seek to attain. InCoRe, as a model, allows for exploration of Tyler"s second question about the experiences that will fulfill the purposes listed in response to his first question. Experiences are seen in holistic form and, following Bruner"s spiral curriculum, mapped out over a program of study. InCoRe allows for the identification of pivotal moments within a four year program of study where, through reflection, learning in fulfillment of program outcomes can be enhanced. The ability to link learning across courses and years of study is not only enhanced but mapped according to the InCoRe model allowing students, faculty members and experiential learning advisors advance notice of when reflection on learning should occur.
For InCoRe to function as intended, program outcomes have to be well established and written with precision in response to Tyler"s first question about the educational purposes a school, or in this case, a program of study, should seek to attain. InCoRe, as a model, allows for exploration of Tyler"s second question about the experiences that will fulfill the purposes listed in response to his first question. Experiences are seen in holistic form and, following Bruner"s spiral curriculum, mapped out over a program of study. InCoRe allows for the identification of pivotal moments within a four year program of study where, through reflection, learning in fulfillment of program outcomes can be enhanced. The ability to link learning across courses and years of study is not only enhanced but mapped according to the InCoRe model allowing students, faculty members and experiential learning advisors advance notice of when reflection on learning should occur.
Reflection is the most important aspect of the InCoRe model, particularly as it relates to the pre-determined touch points prior to, during, and after the work-integrated learning experience. In this sense, reflection refers to the process of linking current experience with prior experience in an attempt to increase the attainment of program outcomes within a given discipline and self efficacy. Students are provided with a program map of where reflection about their academic and work-integrated learning will occur. During these points of reflection, faculty and staff members guide students as they attempt to construct meaning from their experiences, develop personal and professional insights and future goals.
Kolb"s (1984) four-part model of experiential learning (concrete experience, observation and reflection, formation of abstract concepts, and testing in new situations) provides a theoretical basis for the reflective part of the InCoRe model. However, the most important part of Kolb"s influence is the ongoing and cyclical nature of his four-part model. The spiral InCoRe model is as much built on Kolb"s view of cyclical learning and development as it is Bruner"s. The facilitated reflection step included in the model is derived from the theoretical work of Kolb. However, the approach used with students is based more on the writings of Boud, Keogh and Walker (1985) who offer a working definition for reflection germane to the InCoRe model, stating that "reflection is an important human activity in which people recapture their experience, think about it, mull it over and evaluate it. It is this working with experience that is important to learning" (p. 19). InCoRe follows a similar construct but with a greater emphasis on debriefing and reflection. Students would focus debriefing and reflection on attainment of the learning outcomes in place for their specific discipline in relation to a recently completed work-integrated learning activity. The faculty member facilitating the reflection with his or her student uses the context of the student"s work-integrated experience as the basis for reflection on attainment of learning outcomes and self efficacy.
InCoRe represents an evolutionary construct built on the prior work of great academic theorists. It is unique not because of the theory it is based upon but because the institution where InCoRe is being piloted is committed to employing the model as a means for student transformation. The team that developed InCoRe, due to its composition, understood that working together to improve student learning was both feasible and necessary and that InCoRe represented a simple construct that could guide this interdisciplinary work in a meaningful and planned way.
During the latter phases of InCoRe development, a new system for collecting student feedback related to the internship experience was also being developed. The InCoRe model influenced the student feedback project and data regarding students" self assessment of internship learning competencies was collected. The development of a student feedback instrument for internships was the first in a number of planned subsequent steps to the InCoRe model and this paper reports on the findings of this first step. The implications and significance of the creation and implementation of this student survey are broader than the InCoRe model. More important than the specific results of the pilot was the new mechanism to collect consistent student input to be used to improve internship programs, career services, and the overall student experience. Perhaps most important, the student self-reporting on the experiential education competencies at a specific milestone (internship) will have the most impact once aggregate data over the course of an academic year is analyzed and fully realized.
METHODOLOGY
An important component of the university experiential education re-engineering efforts was the development of a feedback mechanism for internship program, process improvement, and student self assessment of established competencies. Information obtained from both internship students and internship employers about the internship experience can be used to positively affect the internship program design, internship site quality, and services and resources offered by the university. Additionally, the results from the assessment have potential to be used as a feedback mechanism for the academic units and curriculum enhancement.
Survey Instrument Development
Prior to 2008, the university used a variety of student surveys for internships. With a goal of creating a common student survey evaluation instrument to be used by the four campuses of the authors" home institution, the following steps and methodology were used to develop the instrument. Subsequently, a new internship student survey instrument was developed and implemented in the spring of the 2010 academic year, yielding promising results.
Student internship survey instruments from all four campuses, academic colleges, and academic programs were collected, yielding sixty-five unique student survey instruments. Historically, these survey instruments consisted of both hard copy paper forms and Microsoft Excel or Word documents that were emailed or faxed from employer and student to the university career services office. The questions from each survey instrument were compiled yielding over 400 student survey questions. Once compiled, the survey questions were coded and organized into categories. The major categories that emerged were a) "services" provided by the experiential education and career services department, b) internship "program" related information, c) "skills" related questions, d) internship "site" related information, and e) "value-added" information. Building from the five survey categories, a list of potential survey questions was developed. Ultimately, a final set of survey questions was developed using the following question: What student feedback or data is essential for improvement of the internship program or to provide the university with an understanding of how to enhance the student experience? Once developed, a draft of the survey instrument and questions was shared with the directors of Experiential Education & Career Services and staff at each campus and the Vice President of Experiential Education & Career Services for input prior to finalizing the pilot survey instrument.
Through a collaborative effort with the university office of institutional research, a discussion ensued about creating the most efficient process for distributing the new student survey instrument and collecting student responses. Instead of developing the survey instrument in Microsoft Excel or Word format and distributing the survey instrument via hard copy or email, as done previously, an electronic, on-line version of the survey was created using the Survey Monkey software. The on-line link to the Survey Monkey software was shared with work-study students from each campus in the spring of 2010. Student feedback about the content, clarity of survey questions, general process using the Survey Monkey software, and time to complete survey was collected. Based on this feedback, the survey instrument was modified and finalized.
Instrumentation
The survey was developed to assess student satisfaction with services, self reflection, and self-reported attainment of competency. Specific validity and reliability analyses were not conducted on the survey instrument; however, feedback was collected from a sample group regarding the language of the questions, thus adding to the content validity of the instrument
Administration
In week 9 of the 11 week spring 2010 term, the student survey pilot was officially launched. The student survey was administered to internship students at each campus using Survey Monkey, a web-based survey design and data collection tool. Electronic invitations to participate with introductory and explanatory messages were emailed via Survey Monkey. Electronic links to the survey with an introductory and explanatory message were sent to internship students. One week later, students who had not completed the survey were identified and a follow-up email was sent to these students.
Sample
A total of 1217 male and female students in the second, third, and fourth year of study depending on their major (n = 1217) were sent the survey via email. A total of 943 respondents participated (n = 943) resulting in a response rate of 77.5 percent. Table 1 provides a profile of the respondents by school and college.
| School/College | Respondents (n) | Proportion (%) |
|---|---|---|
| College of Business | 133 | 14 |
| College of Culinary Arts | 525 | 56 |
| Hospitality College | 259 | 27 |
| School of Technology | 26 | 3 |
| Total | 943 | 100 |
RESULTS
Data analysis was conducted using SPSS. Not all of the results of the pilot survey are relevant to the scope of this article. Table 2 contains the responses and percentages for the students" ratings of their internship experience. Students were surveyed regarding their proficiency in the university-identified experiential education competencies; disciplinary knowledge and skill, professionalism and work ethic, problem solving, communication, and collaboration. Ninety-three (93) percent of students surveyed agreed that he or she demonstrated competency in their discipline during their internship experience. Eighty-five (85) percent of students agreed that he or she demonstrated competency in professionalism and work ethic during their internship. Ninety (90) percent of internship students agreed that she or he demonstrated competency in problem solving. Ninety-three (93) percent of internship students reported that she or he demonstrated competency in communication while 91 percent of students agreed that she or he demonstrated competency in collaboration during their internship. In addition, 81 percent of the surveyed students agreed that they regularly engaged in a process of self-reflection throughout the internship experience, although 16 percent of students reported a neutral response. Furthermore, students report that they are able to adapt to their internship site, that they regularly engage in self-reflection and that their coursework prepared them for their internship experience. Regarding the value-added survey items, the following results emerged. Eighty percent of students agreed that the internship program strengthened his or her confidence in academic program or career choice. Seventy-six percent of students surveyed agreed that they feel more prepared for their career after completion of the internship.
DISCUSSION AND IMPLICATIONS
This paper presents the evolution of the InCoRe model with the goal of transforming the model from conceptual to practical. However, the movement from conceptual to practical is a multi-phase process. The results of the student survey data presented address the first phase of moving the InCoRe model from conceptual to practical. Specifically, this involved creating a survey instrument to capture student feedback related to attainment of established competencies during the internship. The next phase, which is currently underway, is redesigning the internship assignments to include increased critical reflection and student self-assessment, particularly related to pre-determined competencies. A third phase, also underway, involves capturing internship employer assessment of student competency through a similar survey instrument, allowing employers to evaluate student performance during the internship. The employers are evaluating students against the established experiential education competencies noted earlier. The survey, supported by the comparative results from both student and employer, especially for the experiential education competencies, will be useful tool to validate the InCoRe model during its implementation. As described earlier, the InCoRe model outlines a theoretical map where students are prompted to reflect on their attainment of learning outcomes at designated pre-determined milestones. The scope of this paper highlights the designated milestones of the second and fourth year internship experience and provides the results of a student survey asking students about a number of related items including a self-assessment of attainment of student learning outcomes for their internship experience.
The high percentage of students that report engaging in self reflection (Q3b) exceeded the results anticipated by the authors and indicates that students are prone to self reflection and that the quality of reflection could potentially be improved if students are provided with guidance. The authors were also surprised by the high percentage of students who agree that they demonstrated competency across all outcomes categories. Although a student self report is an indirect form of assessment compared to a direct assessment of student competency, the results suggest that students feel they have competency and, perhaps, a certain degree of self efficacy. Overall, the positive results reported by students indicate that the approach to integration, coordination and reflection proposed by the authors in concert with the assessment methodology used may prove to be an effective way of engaging in program and outcomes assessment for four-year programs that include some form of experiential learning.
We anticipate that the findings presented in this article will further the understanding of internship programs on student learning and development, particularly in relation to program competencies or learning outcomes. The results will also serve as evidence of successful completion of the goals set forth in the university strategic plan FOCUS 2011.
Practitioners involved in university-wide strategic initiatives or planning related to experiential education or workintegrated learning programs may also find the following implications helpful. First, the creation of a conceptual model grounded in experiential and learning theory provided university stakeholders a visual depiction of a multidimensional construct, InCoRe. Development of a visual model provides a central reference point for stakeholders to refer to over a long-term strategic initiative process. Second, conceptual models are limited in benefit to university leadership unless they can offer data to support the effectiveness of the model. In this case, we have strategically decided to extract a particular part of the conceptual model, collect data, and are preparing to report the data to the necessary stakeholders. Practitioners may find this a benefit to understanding the transition from conceptual to practical.
Finally, the results of the student survey have many implications, particularly related to program improvement and academic preparation prior to the internship experience; however, this is not within the scope of this paper. Future research will investigate the expansion of student reflection based learning within courses in the major and the general studies core.
SUMMARY
The intent of this paper is threefold. First, the paper outlined the developmental process of a conceptual model called InCoRe, an acronym for "integration," "coordination," and "reflection." This model was central to the re-engineering efforts of a university-wide experiential education program that included the incorporation of student self-assessment of established competencies. Included in this section were the theoretical foundations for the model. Second, the paper described the first steps in implementing part of the conceptual model, specifically capturing student feedback related to assessment of the previously mentioned competencies. Third, the paper shared the results of the student feedback described above. As further implementation of the InCoRe model takes place, the authors anticipate the model itself, as well as the usefulness of the data obtained from future studies, will evolve
| Survey Question | Agree n (%) |
Neutral n (%) |
Disagree n (%) |
Total n (%) |
|---|---|---|---|---|
| Q1d. Internship strengthened my confidence in my career/academic program | 749 80% |
134 14% |
55 6% |
938 100% |
| Q1e. I feel more prepared for my career after completion of internship | 717 76 |
162 17 |
62 7 |
941 100 |
| Q1f. The internship experience affected my educational and career goals positively | 726 77 |
159 17 |
53 6 |
938 100 |
| Q2a. At my internship, I demonstrated competency in my discipline | 876 93 |
56 6 |
9 1 |
941 100 |
| Q2b. At my internship, I demonstrated competency in Professionalism/Work Ethic | 894 93 |
45 5 |
3 0 |
942 100 |
| Q2c. At my internship, I demonstrated competency in Problem Solving | 850 90 |
83 9 |
7 1 |
940 100 |
| Q2d. At my internship, I demonstrated competency in Communication | 875 93 |
56 6 |
7 1 |
938 100 |
| Q2e. At my internship, I demonstrated competency in Collaboration | 854 91 |
74 8 |
11 1 |
939 100 |
| Q3a. I was able to successfully adapt to my internship site | 868 93 |
59 6 |
9 1 |
936 100 |
| Q3b. I regularly engaged in a process of self-reflection throughout my internship | 762 81 |
151 16 |
25 3 |
938 100 |
| Q9. My Johnson & Wales University coursework appropriately prepared me for my internship experience | 678 73 |
197 6 |
56 6 |
931 100 |