Lorna Hayward, EdD, MPH, PT, Associate Professor
Department of Physical Therapy, Northeastern University, Boston
Betsey Blackmer, EdD, PT, Bouve College of Health Sciences
Northeastern University, Boston
Joe Raelin, PhD, Asa Knowles Chair
Center for Work and Learning, Northeastern University, Boston
This study examined the impact of teaching physical therapist students a Model of Reflective Practice, a schematic to describe the process of learning from experience. Participants used the Model to guide their journaling during a cooperative education (co-op) experience. Using a pre-post design, learning outcomes were measured through the administration of a newly developed outcomes survey, the Practice-Based Learning Outcomes (P-BLO). Eighty-six students completed the P-BLO. Forty-three of these students received the reflective intervention prior to going on their first co-op experience. Results indicate that the reflective practice intervention had a significant impact upon P-BLO origination or developing new knowledge from work experience through inquiry with others. The findings also demonstrate that academic achievement or grade point average (GPA) is related to P-BLO extension. Extension refers to a student's ability to use existing knowledge to make sense of the workplace. The study supports the value of teaching students a process for reflective learning prior to co-op. Students with higher GPAs may be better equipped to integrate their existing academic knowledge with new challenges presented to them in the workplace.
Students on cooperative education (co-op) or internship may feel limited in what they believe they can learn in the workplace if they are not provided with tools to guide their learning process. For example, in the classroom the syllabus is one tool that provides a blueprint for learning goals. Furthermore, students are often socialized in the classroom to be passive recipients of knowledge, and this socialization may extend into the workplace where students may expect that supervisors should teach and show them how to do things. This belief implies that the student is the recipient of knowledge while the supervisor is the dispenser of knowledge. Prior research demonstrated that when students are passive recipients of knowledge rather than active, self-directed learners they are likely to become disengaged from the learning process. This, in turn, may result in boredom, lack of involvement, or even disruptive behavior (Hayward, Noonan & Shain, 1999).
Including opportunities for reflection has been demonstrated to help students learn during new work experiences (Hayward, 2000; Leigh, 2004). In particular, the development of a habitual reflective thought process results in self-directed professional growth and initiative on the job (Hayward, Noonan, & Shain, 1999; Hayward, 2000). Many authors in the teaching profession support the importance of reflection as a strategy or tool for promoting self-directed professional growth (Adler, 1993; Belenky et al., 1986; Brodkey, 1993; Guillaume & Rudney, 1993; Lampert, 1991; Lytle & Cochran-Smith, 1992; Osterman, 1990).
Cooperative education professionals use many strategies to stimulate reflective thinking in students about co-op or internship experiences. The strategies include: journaling (Hayward, Krantz, Evans, & DiMarco, 2001), reflective papers (Keen, 2001; Leigh, 2004), learning communities (Howard & England-Kennedy, 2001; Leigh, 2004), portfolios (Lampe & Rothman, 2004), virtual dialogue (Canale & Duwart, 1999), mentoring (Hayward, DiMarco, Krantz, & Evans, 2001; Ricks & Van Gyn, 1997; Van Gyn & Ricks, 1997), case studies (Van Gyn, 1996), debriefing (McRae, 2005), and concurrent learning teams (Raelin et al., 2006).
The target population for this study is constituted of physical therapist students enrolled in a 6 ½-year clinical doctorate of physical therapy (DPT). Prior research has indicated that inexperienced physical therapist students on experiential education placement have concerns about survival on the job and self-adequacy as practitioners (Jensen, Shepard & Hack, 1990; Jensen & Denton, 1991). This finding is not surprising especially given that success within an organization is highly dependent on an individual's ability to decode cultural expectations (Schein, 1992). It can take six to nine months for a student to understand a culture and feel comfortable in a working situation. It is known that reflection can promote self-directed professional growth and innovative thinking among novices (Berkley, Curtis, Minnick, Zietlow, Campell, & Kirschner, 1990; Calderhead, 1989; Freiberg & Waxman, 1990; Guillaume & Rudney, 1993; Jensen, Shepard, Gwyer, & Hack, 1992; Sparks-Langer & Colton, 1991; Zeicher & Liston, 1987).
Reflective critique is one method for conducting self-directed thinking about the outcomes of an experience to guide future work (Huber & Hutchings, 2005). However, while providing students opportunities for reflection, they may not understand why or how reflection actually impacts their learning. Intentional learning requires students to understand their own learning. Reflection on learning how to learn makes students more purposeful in their studies. Once a student gains an appreciation for how they learn, they can consciously take control, make choices, and forge connections that may increase their depth of understanding (Huber & Hutchings, 2005).
The view taken in this study is that intentional learning can be facilitated in students if it is actively and explicitly taught. The investigators examined whether teaching a Model of Reflective Practice (Hayward, 2000), based on guided journaling, enabled students on co-op to make explicit the learning outcomes that may impact their future actions. To measure any changes in practice-based learning outcomes due to the reflective process, a new survey instrument was developed and administered. For the purposes of this study, practice-based learning outcomes refers to learning in the workplace (practice setting) through the process of reflective judgment. The prevailing proposition was that students who have a framework for reflection on experience coupled with reflective journaling would be better prepared to recognize and assimilate the learning benefits from co-op.
Dewey's (1933) definition of reflection, in combination with Kolb's Experiential Learning Theory (1984) and the work of Schön (1983; 1987) provided the conceptual framework for the Model of Reflective Practice. Elements from each author's work contributed to the model and resultant conceptual framework for the current study.
Dewey defined reflection as the "stepping back from perplexing experience (to halt impulsive thought) which allows one to think and generate a more comprehensive plan of activity" (1933, p 64). He also described the process of learning as being continuous and based on personal experience (Dewey, 1938). Schön (1983) maintains that reflection is required for professional development. He proposed that professional practice could be enhanced if novices were provided with opportunities to reflect on their professional actions (Schön 1983; 1987).
Kolb (1984, p. 30) defines learning as the process in which knowledge is created through experience that is contrary to learner expectations, thereby facilitating a change in perspective. Experiential learning is the dynamic process in which ideas are reworked continuously through an individual's experience. Kolb's cyclical four-stage model of experiential learning includes: a) concrete experience--learners become involved in new experiences; b) reflective observation--learners observe and reflect on experiences using multiple perspectives; c) abstract conceptualization--learners create concepts that integrate observations into theories; and d) active experimentation--learners use these theories to make decisions and solve problems.
Learning through experience has also been described as a deepening of knowledge that is dynamic and changeable. Hutchings and Wutzdorff (1988) maintain, furthermore, that the integration of knowing and experience transcends application. They believe that learning is an ongoing interactive process in which both knowledge and experience are reciprocally transformed. Reflection is critical to this process as a means of fostering deeper, more integrated knowledge (Huber & Hutchings, 2005). Reflecting on the process of inquiry can lead to an understanding of experiences that may have been overlooked in practice (Raelin, 2000; 2001).
Schön (1983) describes the process of problem identification and how to apply past knowledge to a new situation. He describes how this process allows professionals to deal with unique or unexpected problem situations that might occur in a work setting. For novices especially, a mentor or coach may assist with the learning using reflective dialogue to examine past action. The expert uses the novice's knowledge to reframe the experience and suggest alternative approaches. At this point, the student can experiment with the suggestions and adapt the new situation to evolving conceptual frameworks. Schön also describes reflection-in-action in which an individual reflects on a perplexing incident while in the midst of experience itself.
Figure 1 in Appendix A (Hayward, 1995) depicts the Model of Reflective Practice that is a synthesis of the theoretical concepts described above. The starting point for a novice engaged in self-reflection is an experience. For Kolb (1984), this would be the concrete experience stage that requires the learner to participate in a new experience. According to Dewey (1933), an experience must be perplexing, or, as described by Kolb, contrary to learner expectations. To document the perplexing incident, the individual gathers information about the situation using methods of reflection. At this point, a mentor could assist a novice with the problem identification process (Schön , 1983).
Next, the novice must step back and think about the data gathered on the experience (Dewey, 1933). Reflecting on observation captured through journaling corresponds to the second stage in Kolb's (1984) model, reflective observation. Then, based on past experience and knowledge, the novice may draw inferences about the incident and generate a hypothesis for why it occurred. Ideally, knowledge is gained from the incident and is integrated into the novice's thinking. This point in the Model corresponds to Kolb's stage of abstract conceptualization, which describes how the individual integrates observations into existing theories. For Schön, (1983) this is where the novice determines if a situation fits the expert's frame. Finally, new knowledge may influence subsequent action. This stage corresponds to Kolb's (1984) final stage of active experimentation and the point where Schön's (1983) mentor or coach would suggest an alternative approach that the novice experiments with in practice.
In summary, the process of learning through experience is dynamic as well as personal. The learning process involves reflection either on past action or inaction, where the individual assumes the role of researcher who assesses his/her practice and applies what is learned to future action. Reflection and learning from experience promotes professional development that is meaningful and relevant for the individual.
The investigators proposed a null hypothesis that the taught model of reflective practice enhanced through journaling would produce no incremental effect on practice-based learning outcomes between two time periods spanning a six-month work experience of co-op.
The physical therapy (PT) program at Northeastern University (NU) is a 6 ½ year clinical doctorate in physical therapy (DPT). Each class within the DPT program contains approximately 70-100 students; the NU PT program is one of the largest physical therapy programs in the country.
All DPT students participate in two six-month cooperative education experiences (co-op). Co-op I occurs upon completion of the first two years of the program and co-op II during year four. While on co-op, students typically work as physical therapy aides under the supervision of licensed practitioners. Sixth year students participate in a 32-week clinical education component in which they assume the role of the PT under the supervision and instruction of an assigned clinical education instructor.
Participants in the study were 86 third year DPT students enrolled in two of four sections of a required course, PTH U212 Foundations of Physical Therapy. PTH U212 provides instruction to students on basic PT skills in preparation for working as an aide on co-op I.
The NU Office of Institutional Compliance reviewed the project and classified it as exempt because it was conducted in an established educational setting (classroom-based, regularly offered, required course) and examined the impact of an instructional technique. All student participants were oriented to the project and provided with an informed consent form that described the project purpose.
The NU's Center for Work and Learning created and pilot tested a new survey instrument, The Practice-Based Learning Outcomes Scale (P-BLO) -. The constructs for the instrument were original and designed to measure practice-based rather than classroom-based learning outcomes. Evaluating practice incorporates not just context, but the process or exercise of personal reflective judgment.
The items were each derived from theoretical expositions from writers who had documented their constructs focusing on meta-level reflective learning from experience (Bandura, 1977; Billett, 2001; Boud, 1998; Isaacs, 1999; King & Kitchener, 1995; Lave & Wenger, 1991; Mezirow, 1981; Piaget, 2000; Raelin, 2000; Vygotsky, 1962). Thus, all items are original, though substantiated from the literature. A 6-point Likert scale was provided for each item: 1 never, 2 small extent, 3 moderate extent, 4 large extent, 5 all the time, 9 not applicable. The scale developed contained 15 items, five each on three dimensions: "Soliciting Knowledge from Experience" (e.g., being aware of inconsistencies between one's beliefs and actions), "Developing Knowledge from Experience" (e.g., using existing knowledge to process new or unknown situations), and "Sharing Knowledge from Experience" (e.g., using knowledge to suggest improvements in the work environment). The number of items in the scale was doubled from 15 to 30. The purpose for the doubling was to determine if the three proposed subscales persisted when reverse-scored. Reverse scoring is a method that is often used to counter the effects of response bias (Baumgartner and Steenkamp, 2001). In addition to the 30 items, demographic information was collected on gender, ethnicity, socioeconomic class (SES), age, grade point average (GPA), graduation year, most immediate work experience, and previous experience with journaling.
The study consisted of a pre-post test design. The P-BLO survey was administered to all 86 participants during a PTH U212 class meeting in June 2005 just prior to co-op I. In addition, all participants were asked to keep a reflective journal on co-op I that documented their experiences, observations, learning, ethical dilemmas, and interesting patient cases. Students were informed that their journal entries would be used for assignments in class upon their return from co-op.
The 86 students enrolled in PTH 212 were randomly assigned by the course instructor to one of the four sections of the course. The sample was one of convenience, and two of the four sections were randomly allocated as the experimental group. The Model of Reflective Practice (Figure 1) was taught to 43 students, who were in two experimental sections of PTH U212. The instruction consisted of a half-hour presentation by two of the principal investigators who provided a definition of reflection, guidelines for how to do reflection, and a description of the .
Information about the Model included a description of how a perplexing experience can be understood by collecting data through journaling. Contemplation of the data can lead to new thought patterns that provoke a change in future action. The students were provided with a copy of the Model and guidelines for reflection. The guidelines asked the students to answer three questions related to any observed clinical event: What experience was important to their learning? Why was it was important? How they would use their new knowledge in the future? While the experimental group was on co-op, they were emailed as a group once a month for a total of five times and reminded to record and reflect on their observations in their journals.
At the end of the six-month coop experience, during a lecture associated with a class entitled, PTHU305, PT Professional Seminar 1, all students were asked to complete the survey again, as a post-test. Of the 86 original students, 68 completed the second survey. Eighteen students were either absent or refused to complete the post administration. In addition, seven surveys were not fully completed and were discarded, producing a final count of 61 successful survey pairs. This produced a response rate of 71 percent, representing a comparatively robust return for longitudinal surveys.
Once the quantitative data were coded and entered they were submitted to full-scale statistical analysis. The initial P-BLO scale using the pre-survey data was submitted to a series of exploratory maximum likelihood factor analyses using oblique rotation with Kaiser normalization. In an exploratory study of this nature, oblique rotation has greater flexibility in searching out patterns because there is no assumption about the multicolinearity of the data. The analysis was initially designed to test the three-way multi-dimensionality of the scale originally proposed: soliciting, developing, and sharing knowledge from experience. Finally, analyses of variance were run on all the scale items using the demographic data to determine if there were any initial relationships at the pre-test stage that may have caused variation in the scale responses prior to the experimental conditions of co-op combined with the reflective practice intervention.
Following co-op I, the post-survey was administered to 61 pairs of student respondents. T-tests were run on the scale data to determine, first, if the full sample between the two time periods, pre and post, differed on the basis of the work experience alone. A second t-test was run to determine if there were differences between the 61 paired samples at the second time period – those in the experimental group who received the reflective practice intervention and those in the comparison group who did not. Finally, a multi-level regression model was applied to the post data to address the study hypothesis: whether a unique effect existed due to the reflective practice intervention, controlling for any alternative explanations through the demographic variables.
Descriptive data revealed a characteristic profile of the sample as follows: 60 percent of the class was 20 years old, whereas 30 percent were 19. Eight-two percent were female and 86 percent were Caucasian in ethnic identity. All but seven percent considered themselves to be middle class in socioeconomic background, half of which said they were middle rather than lower or upper middle class. Forty-seven percent had recently held a part-time job, whereas 30 percent said their most recent work experience was a summer job.
The pretest data were first analyzed to determine the reliability and validity of the P-BLO scale. The P-BLO instrument directed the respondents to answer the survey questions based on their most recent work experience. Using maximum likelihood exploratory factor analysis, the results did not support the three-dimensional factor model originally proposed. Furthermore, the reverse-scored items did not load in any consistent way with the matched positive scale variables. However, two factors, incorporating eight of the original 15 items, loaded successfully, producing a goodness-of-fit of 4.95 with 13 degrees of freedom (see Table 1). The first factor had five items, which together produced an adequate internal consistency reliability estimate (alpha = .72). The second factor had just three items, producing a modest alpha reliability estimate of .62, considered to be acceptable in an exploratory analysis. These two sub-scales present the opportunity to use a parsimonious measure to examine practice-based learning outcomes.
Upon closer scrutiny, the two subscales appear to correspond to the two cognitive structures described by Piaget (1969) and adopted in other constructivist perspectives of learning (Bruner, 1966; Vygotsky, 1962). Constructivism stipulates that the purpose of teaching is not to transmit information, but to encourage knowledge formation and development (Bruning, Schraw, & Ronning, 1995). Piaget's process of assimilation involves the interpretation of events in terms of existing cognitive structures. The process of accommodation, meanwhile, refers to changing one's cognitive structure to make sense of the environment.
The first subscale, labeled extension, corresponds loosely to Piaget's assimilation concept, in which student learners attempt to use and also extend an existing cognitive structure to make sense of and inquire about workplace phenomena. The second subscale, origination, corresponds loosely to Piaget's accommodation concept, in which student learners attempt to change existing cognitive structures by inquiring with others to make sense of workplace phenomena (see Table 1 in Appendix B).
Following the construction of the two new practice-based learning outcomes subscales, analysis of variance procedures were run to determine if the two subscales were related to any of the demographic variables prior to the co-op experience and to the experimental reflective practice intervention. A few of the demographic variables obtained significant F values for some of the component measures of the two new scales. However, the two subscales as composite measures (produced from summing the components items) did not have any significant relationship to the demographic measures. Thus, at the outset, the sample was determined to be reasonably homogenous when it comes to practice-based learning.
The post-survey data were examined to determine if any observed differences could be accounted for by the six-month co-op experience. The sample was then split on the basis of the treatment effect of reflective practice. Any differences were tested using the Student's t-test in which the calculated mean differences and their associated standard deviations are examined in relationship to their real measures (Table 2).
Data in Table 2 illustrate that the extension scale did not differ between the two time periods nor was any difference explained by the experimental effect. The origination scale, measuring how students develop new knowledge from their workplace experience, did produce significant differences both within the overall sample and for the experimental group. However, the difference disappeared when examining the comparison group data alone. Thus, there appears to be an effect on origination practice-based learning from co-op experience that might be explained by reflective practice. Such a conclusion, however, would require a multivariate analysis (see Table 2 in Appendix C).
The post-survey data were submitted to a general linear model (GLM) least squares analysis to determine if the dependent variable scales could be explained by a combination of either classification or continuous variables. GLM essentially represents a multiple analysis of variance supplemented by the introduction of continuous variables, such as academic achievement, measured by grade point average. In this multivariate analysis, the demographic measures were added to the treatment measure to determine if there might be alternative explanations for any observed differences. It should be noted that the demographic measures used were those at the pre-survey stage. Only two of these measures were susceptible to change during the study period: academic achievement and journaling. However, when their time-2 counterparts were entered into the analysis in place of the time-1 measures, no changes were noted.
Table 3 contains data for two multivariate models, one for the extension subscale as the dependent variable; the other for origination. The scores for each of the subscales are difference scores; i.e., the calculation of the difference between the respondents' answers at the pre- and post-survey stages. In both instances, the R-Square value, which refers to the fraction of the variance explained by each model, is approximately .27. Given the very few explanatory variables entered, this is a reasonable indicator. The respective root square mean errors or RSMEs suggest that the origination model is closer to the fitted line than the extension model. For both models, two sum of squares (SS) statistics are provided: Type I and Type III, each carrying an F value to determine explanatory power. The Type I or sequential sum of squares indicates for each term how much the residual sum of square for the model is reduced. The Type III sum of squares determines how much the residual sum of squares is reduced by adding particular terms that contain all the other terms in the model statement. Hence, the latter holds a higher threshold.
In the first equation for the extension (difference) subscale, the only significant F is for academic achievement (GPA) at a P value of .04 for the Type I SS and at .09 for the Type III SS. Further analysis was performed on the categorical variables to see if any categories produced significant t values, but none were found.
For the origination (difference) subscale, the only significant F is for the experimental treatment of reflective practice at a P value of .05 for the Type I SS and .11 for the Type III SS. When examining the categorical variables, one category emerged at a t value significant at the .04 level: from the variable, most immediate work experience, having had part-time work prior to co-op was found to be a predictor of origination (See Table 3 in Appendix D).
This exploratory analysis, given its small sample size, has potentially generated new information relative to cooperative education research. First, the work has provided a first-step validation of a new measure of practice-based learning outcomes (P-BLO). Up to now, traditional co-op outcomes have been measured via economic (increase in wages upon obtaining a first job) or vocational (obtaining a job in one's major) indicators (Howard & England-Kennedy, 2001; Mundhenk, 2004). Learning outcomes have been traditionally relegated to standard measures of achievement such as grade point average or aptitude, using SAT or GRE scores. There is value in adopting familiar scales in wide use that have proven validity. However, programs of cooperative education must discover methods for assessing the learning outcomes attributed to experiential education that is specifically practice-based in character. This study has developed two new parsimonious scales to measure practice-based learning outcomes. The first scale, extension, assesses the student's ability to use existing knowledge to make sense of and inquire about workplace phenomena. The second scale, origination, assesses the student's ability to develop new knowledge through interaction with others to inquire about and make sense of workplace phenomena. Initial tests of reliability and validity suggest that, though this was an exploratory study using a relatively small convenience sample, these scales may add to the practical literature in assessing experiential learning.
This study also adds to the growing body of knowledge that suggests the value of engaging students in reflective practice while they are involved in off-campus experiential activities, such as co-op. Associated with the tradition of contextualized learning theory, reflection-in-action proposes that learning can occur in the midst of practice (Lave & Wenger, 1991). Instead of banking knowledge into one's mind (Freire, 1989), knowledge can be viewed as an interactive dialogue among a community of inquirers who share meanings, interpretations, and ideas (Dewey, 1938). In our case, students were asked to reflect on interactions with professionals in their field of study
In particular, students were asked to keep a reflective journal that documented their experiences, observations, and learning while on co-op. Furthermore, they were encouraged to communicate during co-op with their co-op advisors via e-mail or phone. Half of the students were instructed in a reflective process using a theoretical model. This study examined whether the reflective practice model had any effect on the students' practice-based learning. The results suggest that the treatment effect of reflective practice did not influence practice-based learning as characterized by extension. Indeed, there was little effect on this type of learning by co-op itself except for one important exception. Those students who exhibited high academic achievement were more likely to exhibit extension in their experiential learning than those with lower academic achievement. This finding may dispute common knowledge that academic achievement does not differentiate students vis-à-vis the benefits of co-op. Our results suggest that students' with higher grade point averages use their academic knowledge to take advantage of and use the knowledge available to them within the workplace.
When it comes to the practice of origination or developing new knowledge from work experience, the treatment effect of reflective practice has a discernible influence. This point is supported by the data in Table 3, origination difference. By teaching and exposing students to methods to acquire knowledge in the workplace using reflective practices, students were apparently better prepared to learn new things in the workplace especially through inquiry with others. The findings from this study also suggest that those students who had a previous part-time job were also more predisposed to originate knowledge from experience.
Through our research we have attempted to examine how teaching a model of reflection impacts student learning while on co-op. . This study was exploratory and was based on a small sample size of 61 respondents. In addition, the study was conducted with physical therapist students, whose experience on co-op may be different from students in other fields. Although the estimates of differences achieved statistical significance, the overall explanatory power of the least squares model was not overly robust. The results could change under tighter sampling conditions. Conversely, the findings might receive validation under such conditions. The authors encourage other researchers to replicate these findings and experiment with the newly developed P-BLO instrument. We hope this study will thus inaugurate a new wave of studies that seek to assess learning outcomes from off-campus learning activities that are specifically reflective in character.