COMPARING THE IMPACT OF TWO INTERNSHIP EXPERIENCES ON STUDENT LEARNING

The Problem

Cooperative education programs generally enjoy faculty support where there are clear links between vocational training, career development and job experience. Liberal arts faculty tend to be dubious about the value of internship programs that displace significant amounts of coursework, questioning whether the educational opportunity costs are offset by what is learned in the field. Even where faculty realize that the goals of liberal education and experiential education coincide, e.g., in putting theory into practice, acquiring a deeper understanding of how organizations operate, and developing higher order cognitive skills (Couto and Zuberer, 1988), they are reluctant to support full-time internships as the best way to achieve these goals ( Gore and Nelson, 1984). Thus in most universities, internship programs are marginal to the academic program (Migliore, 1990).

We know that students like internships and the chance they provide for 'real world' challenge and we are fairly confident that they contribute to affective goals such as increased self-esteem, tolerance and sense of responsibility and to career development (Williams, 1990; Caccese, 1984; Heck and Weible, 1978; Hursh and Borzak, 1979). We have less evidence, however, of the impact of field experience on learning. Do internships deepen understanding and facilitate transfer of learning? Are there approaches to supervision that will enhance such outcomes? This study addresses these questions.

What We Know About Impact of Internships on Learning

The problem of inert knowledge, the failure to apply knowledge learned in the classroom to real world problems, is well documented (Bransford, 1992). Even well-trained graduate students may fail to recognize problems that they have the tools to solve (Feynman, 1985). The transition from espoused theory or inert knowledge to knowledge in use, is facilitated by practice-in-context - by repeatedly applying concepts in diverse real world contexts and with role models who can help make the intellectual processes involved explicit (Bransford, Goldman and Vye, 1991; Resnick and Klopfer, 1989). The trick is to tum a field placement into a cognitive apprenticeship in which reflection is truly integrated with action and problem finding is accorded the same status as problem solving. The special virtue of the college internship is that it can follow the problem oriented action-reflection-action process that characterizes adult learning in professional contexts. Students can take the initiative to seek out experts, materials and other opportunities for knowledge as they pursue solutions to real organizational problems. Further, they can be assisted by guided reflection in the midst of intense and productive involvement in the organizational culture (Eyler, 1993). Interns who systematically try out and modify ideas learned in their courses should make them their own; they should spontaneously identify situations where these concepts are helpful and they should demonstrate a more subtle understanding of their application. While many authorities on college internships have identified this process as the strength of such programs (Althoff, 1979; Caccese, 1984; Hagerty, 1981; Hursh and Borzak, 1979; Kantrowitz, Mitchel, Davidson, 1982; Moore, 1983; Pataniczek and Johanson, 1983; Van de Creek and Fleischer, 1984), there have been few attempts to assess program impact on learning.

Where assessment of learning has been attempted, it has often been limited to students' impressions. Students generally claim to 'have learned more' in the field than the classroom (Kantrowitz, Mitchel and Davidson, 1982; Starke, 1985); although some were slow to see a connection between academics and experiences in the field (Hursh and Borzak, 1979). Internships have generally not shown increases in factual knowledge (Balutis, 1977; Eyler and Halteman, 1981) and attempts to link internships with general academic achievement have been mixed (Williams, 1990).

A study of political interns in the Tennessee legislature does suggest a fruitful direction for assessing the impact of internships on student learning and use of knowledge. While all groups gained factual knowledge, political interns were significantly more likely to develop a sophisticated understanding of the legislative process than students studying it in the classroom, or internship applicants who were not selected (Eyler and Halteman, 1981). Where there are clear academic goals for field experience, it is possible to capture movement towards those goals. It is however, a big jump from identifying and measuring cognitive changes in a single subject area to assessing similar changes that might occur in the diverse placements of many college internship programs.

The Vanderbilt Internship Study

This study explores how we might track the impact of a full-time college internship in helping students transform knowledge from the classroom into knowledge in use. Will internships help students develop a deeper understanding of what they have learned in the classroom about people and organizations? Will it increase their recognition of opportunities to apply such knowledge? Will the impact of the internship be affected by the quality of the supervision and instructional context?

Program and Sample. The Human and Organizational Development program is an interdisciplinary major that focuses on people and organizations. Such generic skills as speaking, writing, and analytic thinking are pursued across the core curriculum. The program is committed to active learning and all core courses include projects and experiences that actively involve students in the learning process. The full semester internship in the senior year is designed to integrate knowledge and experience gained in the program.

The three groups of students studied here were all placed in full time business or social service internships in the U.S. during fall of 1988, 1989 or 1990. The curriculum was the same for all 71 students, but there were major differences in the way the internship was defined, supported and supervised for group I and for groups II and III. While all groups were involved in sessions designed to help students reflect on their experience, there was a dramatic difference in the quality of these integrative experiences between Group I and Groups II and III.

Group 1. Before 1989, resources were directed towards logistical support, ie., securing sites, negotiating contracts, and dealing with problems. The 17 1988 interns kept journals and were brought together weekly to share experiences; this reflects widespread field placement practice.

Groups II and III. The second two groups of interns (30 in 1989 and 24 in 1990) participated in a systematic set of activities to integrate the core curriculum with their field experience. Students work through structured exercises to analyze their experience in the organization using concepts central to the curriculum; keep journals and accounts of weekly "critical incidents"; and attend 2-day "workshops" where faculty lead activities integrating classes they teach with the field experience. The program was organized into four classes for credit purposes (rather than a block of 12 credits as in 1988) to emphasize each academic component to students.

Identifying Learning Outcomes. The difficulty in identifying a potential set of learning outcomes from such diverse experiences undoubtedly accounts for the dearth of studies that attempt to assess the intellectual impact of internships. We knew that students could apply concepts and strategies when they were asked to do so, as they were repeatedly asked in classes. To gauge whether the internship increased the likelihood that students would both see the relevance of principles in their core coursework and use them when they were not cued, we devised an instrument to give them discretion in choosing the most useful information to share with a "friend" from another university who was entering a new job in a large organization:

Job Advice for a Friend

You are a star student in Vanderbilt's innovative UHD program; your best friend from back home has been content with a program at Somewhere Else University, and while s/he feels academically prepared to do the job, s/he is terrified about the world of work. You are, of course, an expert on people and organizations. Write him/her a one page letter of advice in which you discuss how s/he should approach the new job; what clues about working in a complex organization can you share?

To reduce the cueing factor the '1etter" task was buried among others that assessed affective and career related variables and critical thinking skills not obviously tied to the curriculum. If the internship was effective in helping students break down the compartmentalization of knowledge and experience, students should both see the relevance of the knowledge they possess by spontaneously drawing on it in advising their friend, and have a deeper understanding of its meaning as shown by the quality of that advice.

Scoring the Letters. The '1etters to a friend" were analyzed for ideas related to the core areas of the curriculum: people skills, organizational analysis, politics and policy, and a systematic inquiry orientation. In addition, notice was taken of explicit reference to formal learning.

As a result of analysis of a set of pilot letters, two other categories were added. The pilot students rarely mentioned the five categories related to program goals, but two other themes emerged that we have labeled "platitudes" and "empathetic responses." These categories were added to the scoring scheme. Brief descriptions of these variables are provided in Table 1; descriptions and examples are given when we present results.

Table 1
Variables Used To Score 'Letter To A Friend'

The pre and post '1etters" of Groups I and II were scored blind without knowing whether the letter was generated pre or post internship or from 88 or 89 interns. Two scorers analyzed the letters with an initial inter-rater reliability of .81; most disagreements were related to whether a passage was scored as medium or high in the attribute. These differences were resolved through conference. Group Ill was measured only after the internship to eliminate any possibility of a testing effect.

No mention of concepts in each of the seven categories was scored L (low); a mention without significant elaboration was scored M (medium); and some elaboration showing deeper understanding of the uses of the concepts was scored H (high). Occasionally the same material might contribute to more than one category. In a one page letter, the level of elaboration is of necessity limited, but the measure does allow us to assess the salience of the curriculum to the tasks that students recognize must be solved in successfully entering and succeeding in an organization.

Results

The 171988 fall interns produced both pre- and post-test letters much like those of the pilot groups. Even after the internship, their advice was rooted in platitudes with few mentioning the many principles about organizations and human relations studied in their classes. The distribution of responses is presented in Table 2. It displays the pre- and post-test raw scores for both groups as well as the post-test only results for Group ID.

The changes in the letters produced by the 30 1989 fall interns before and after their internship experiences were surprisingly dramatic. Students were significantly less likely to write platitude filled missives and significantly more likely to mention specific course materials, and give advice process, and suggest a systematic inquiry orientation. The raw score tabulations showing these shifts are presented in Table 2; and the change scores are graphed in Figure 1 with significance based on t tests for non-independent samples. All shifts but that in the empathy category, which was stable, were at a significance level of .005 or above. The distribution of scores for Group III was similar to that for Group II suggesting that the pre-test had not been a factor in the significant changes noted for Group II.

Platitudes. The initial letters for both groups, like those of the pilot test group, were filled with such cliched advice as "Keep a smile and try not to get frustrated."" … do what feels comfortable to you. Be yourself." This is all reasonable advice, but many of these letters tended to stop with these generalities. Some of this advise remains in the introductions of letters written after the internship, but it does not dominate the letters which are generally filled with advice related to one or more of the core content categories.

Table 2
Content Analysis Letters To A Friend

Empathetic Responses. Many students began their letters, both pre and post-internship, by acknowledging the point made in the stimulus questions, i.e., while the student is "academically prepared … s/he is terrified about the world of work." This may show their natural orientation as human and organizational development students to helping others, or may result from emphasis on acknowledging others' feelings in such courses as the required small group behavior class, or the elective in counseling skills. Students often attempt first to reassure their "friend" with such phrases as :

Figure 1
Change Scores Letter To A Friend N = 30

"Your feelings of fear are natural as you go into a new situation. Remember those you will work with were once where you will be - new/ not knowing anyone."

and

'It's only natural to feel this way, and once you get settled you will be fine because there are people to help you, plus you are qualified to do the work."

Specific Course Materials. A few students recommended specific books or other course materials or specifically reminded the "friend" that the courses taken in college would offer skills and information that would be useful (coded Hand M respectively). Group II interns were somewhat more likely to do this after the internship perhaps as a result of the explicit attempts to tie coursework to the experience during internship seminars, assignments, and workshops. Group ill patterns were similar to Group II.

"Your college as well as mine gives us information … to put to future use. The future is now."

" Of course, we have talked about applying the four frames to the organization - politicaL symbolic, human resource, and structural.

People Skills. The stimulus question specifically mentions that the student writing the letter is "an expert on people and organizations" as the result of their work in the program. We would expect mentions of people skills and, indeed, we find many more pre- test mentions in this area than in the other core curriculum areas. Much of this advice is "common sense" rather than technical information about small group behavior and reflects the salience of getting along with people to these students. Typical pre-test responses coded M included:

" … understand what makes your boss tick."

"respect your colleagues so that they will respect you; learn to communicate."

The importance of people skills became more salient by the time of the post-test letter; almost all Group II and III students discussed getting along with co-workers at least this limited level of elaboration on the post-test; a handful were more detailed and appeared to draw from their study and scored H:

" … active listen, so you will understand their (co-workers) points and how they work together. Try to make sure that everyone feels their job is important. The higher up you go, don't forget the smaller jobs make your job possible."

". . . remind you of important communication skills that Americans sometimes fail to use in an organization. Such topics as empathy, using open ended questions, and body language are some of the themes of effective interpersonal communication."

Organizational Understanding. Because students are writing about entering an organization to work, and because the primary focus of their intern experience is learning to be an effective member of an organization, we would expect the most dramatic growth in understanding to be in this area. Indeed this is the case, although this methodology is probably too crude to capture the full extent of development. About a third of the students mentioned understanding the organization on the pre-test letter but were very sketchy in their advice.

"Get to know the organization well. Get to know the different people in the organization and their roles."

" … should be aware of the structure of the company and how it all works as well as the hierarchy of the organization."

Over two thirds of the Group II and III students addressed this topic in the post-test letter and half of the total number of these respondents wrote with greater specificity.

" … define the organization: what are the obvious norms? what roles are seen? who are the important players? what is the goal of the organization? and define your own presence: what is your role? who are the people you need to be close too? how will you aid the company in achieving their goal? how will you achieve your goal and the company's goal simultaneously … "

"Be aware of the four frames - ways of studying what happens in an organization. These are its political, human resource, structural and symbolic functions. Be aware that even though the organization is structured in a specific hierarchy, certain individuals have … more power of authority over others … learn the codes of behavior, the important symbols etc. slide in gracefully … Find the "storyteller " in the organization and talk to him/her so you can find out what this organization is all about."

Politics and Policy. The curriculum focuses on public policy issues. In the internship seminars we apply political concepts to organizational politics and in this study we coded a focus on the organization's power structure, on influence, the informal hierarchy, and explicit mention of policy making or office politics as demonstrating sensitivity to the political aspects of organizational life. Even with this extremely broad definition of 'political understanding' virtually no response fit this category on the pre-test. There was a dramatic increase in political responses on the post-test for Groups II and III although these were rarely elaborated on. Typical responses included:

" … notice how policy decisions are made and give your ideas to those who have the most influence."

"figure out the 'real' power structure - who has power and who doesn't- who is behind decisions - who holds influence."

Systematic Inquiry. One of our curriculum goals is to encourage students to base their judgments and decisions on good information gathered in response to good questions. In the letters we looked to see if this inquiry orientation was transmitted as a valuable strategy to the 'friend' entering a new organization. Only a third of the students in either group suggested an observation or other data gathering strategy in the pre-test letter. Nearly two thirds of Groups II and III did so on the post-test letter and about a third did so with some elaboration.

"Ask for background information: organizational chart, annual reports, written job descriptions, policy manual -Get informed about where you work and what is expected of you. Look for the power structure - who's who and who's important in what area (who to avoid and who to shmooze with) … take part in office rituals like celebrating birthdays, or snacks at staff meetings or whatever … "

" … talk to people and try to find out details about how the organization works. Get people to tell you stories about the office; learn the hierarchy. It's important to figure out who holds the real power, who has the most respect, who is most valuable to the firm. See how others do things … many companies have unwritten rules that you don't want to break… "

Conckusion

Students in a semester long internship with extensive opportunities for guided analysis and reflection grew in their ability to see spontaneously the relevance of the curriculum to life in a complex organization. Having noted this relevance, they were also more likely to elaborate on the concepts they chose to apply. Thus, the intensive internship experience appears to have contributed to breaking down a barrier to transfer, i.e., the compartmentalization ( of knowledge and experience. Group I interns who had less systematic experience in applying the core concepts during the internship, showed relatively little transfer; their knowledge remained inert.

The importance of this shift in focus from pre- to post-test 1etter to a friend' for the second group and not the first is all the more impressive when one considers the great care that designers of this curriculum have taken to assure that transfer will occur. Students are routinely asked to apply course material in simulations, group projects and in analytic papers. All have had experience conducting their own research and analysis projects including the analysis of an organization. All the interns demonstrated their ability to use these ideas when asked explicitly to do so. That, before their internship, these students still fell back on platitudes when advising a 'friend' on how to successfully enter a complex organization is a testimony to how difficult the transition from classroom to 'real world' is, and perhaps how critical well structured field experiences might be for successful learning transfer. The lack of transfer in the first group even after extensive field work and opportunities to process these experiences with faculty and peers demonstrates how difficult it is to engineer internship experiences that enhance academic learning as well as affective or career development.

Needed Research

This study suggests some directions for fruitful explorations of the role of field study in enhancing the learning goals of liberal education. The sample is drawn from one program in a selective private institution and needs replication with varied populations and programs.

The biggest research need in this area however, is in the identification of appropriate cognitive outcomes and in their measurement. Exercises that cue the domain to be applied miss the need for the student to recognize the situation. Perhaps simulations or systems for observing students in their field settings could be devised to assess these outcomes. Both methodologies are expensive and difficult to manage.

If field placements are to gain legitimacy as an integral part of an effective liberal arts program, it is necessary to demonstrate that they do more than make students feel good and find jobs; more needs to be done to establish their role in enhancing learning. For as one student noted in his journal at the end of the internship:

'1'm ready for this internship to be over. It's not that I haven't learned or enjoyed anything. It's just that I am ready to get back on campus. I've got plenty of work to look forward to in my life." - Human Development Intern