AN EVALUATION MODEL FOR A COMPREHENSIVE COOPERATIVE EDUCATION PROGRAM

A little over three years ago three higher education institutions were invited to participate in a comprehensive project that was to expand significantly the size and scope of their respective cooperative education programs. The institutions selected for participation were Drake Universi­ty, Harry S. Truman Community College (Chicago) and the University of Detroit. With the assistance of the National Commission for Cooperative Education and with financial support from the United States Branch Office of Cooperative Education, each of these institutions began participation in an experiment intended to institutionalize cooperative education as an in­tegral part of its respective curricula. Since that time, the number of schools participating in comprehensive cooperative education projects has grown to 24.

What follows is a case study of one of these institution's attempts to in­troduce and then analyze the results of this program. The focus of the study is on an evaluation model that was used at Drake University.

Drake University is an independent comprehensive university with ap­proximately (j,500 students. There are nine schools and colleges, over 60 distinct degree-related programs and a full-time equivalent faculty of over 300. The curricular mission emphasizes education for the professions based on a solid liberal arts foundation. The institution emphasizes quality teaching, with important consideration given to faculty scholarship and service. Prior to rece1vmg the comprehensive grant, the University had established a limited cooperative education program involving many of the departments from several of the undergraduate colleges on campus. With the receipt of the grant, Drake took steps to expand on its beginnings to develop a comprehensive cooperative education curriculum that would be available to all students. These efforts resulted in a program that grew from 200 to over 700 student participants in three years.

During the 1983-84 academic year, the administration at the universi­ty must make a series of important decisions. Federal funding for cooperative education will end. Will the program continue? At what level? Will some area􀀠 receive greater support than others? What level of financial support will be made available if the program is to continue?

The answer to these questions rests in large part on the ability of the cooperative education staff to demonstrate the degree to which the cooperative education program contributes to Drake's institutional mission and to the quality of its academic excellence. Should the staff be unable to demonstrate the value of the program, the university could not be expected to carry the significant burden of its operational costs. It therefore became critical to develop evaluative methodologies and techniques that would be capable of identifying and measuring the successes and failures of the pro­gram, thus permitting the demonstration of the program's contributions to the university and its mission.

The purpose of this paper is to suggest a model for program evaluation that may be useful to cooperative education professionals in the field and to college and university officials as they seek to advance the implementation of cooperative education while remaining faithful to the curricular mission of the institution. To these ends, this model must identify and employ evaluation methods that are capable of pinpointing problem areas, pro­viding useful information for correcting program deficiencies and develop­ing assessment tools that will identify quantitative and qualitative in­dicators of contributions to the curricular mission.

At this point, a caveat seems in order. As was noted earlier, the evalua­tion model presented in this paper is to be understood in the context of a case study. Its generalizable qualities may be limited: we presented it here to demonstrate what has been done at Drake University and what might be done elsewhere. Program evaluation is a complex task that must be fitted to unique institutional and program characteristics. At a minimum, all other things being equal, the key problems of evaluating a "comprehensive" cooperative education program are associated with program size and com­plexity and thus may differ significantly from those of smaller, less comprehensive programs. In general, it may be noted that as programs grow in size and complexity,

• More students; are placed in jobs and more jobs have to be developed
• The variety of jobs will be greater
• More employers will participate
• More faculty are involved
• Greater staffing needs arise
• A more formalized administrative structure is needed
• Increased curriculum activity takes place
• A larger research effort and data base are needed by the pro­gram manager.

In addition to developing and organizing these elements, the challenge of institutionalizing cooperative education in a large college or university is grounded in the introduction of organizational change and faculty and cur­riculum development. If a co-op program is to succeed and continue, it must be integrated with the mission of the institution and must gain the commitment and involvement of a large majority of educational units. If an evaluation model is to be effective, these issues must be recognized, planned for, and taken into account as significant dimensions of the evaluation pro­cess, They in turn mark the uniqueness of the program being evaluated.

Evaluation Research: An Assessment Model for a Comprehensive Program

Traditional evaluation models in higher education have been developed to help improve existing educational programs by assisting local program planners in decision-making and in initiating subsequent changes. These traditional evaluative efforts have been based, in large part, on the knowledge that the programs being evaluated are generally valued as in­tegral parts of an on-going curricular design and that the audience for the evaluation is a local one. As a result, modifications to programs are seldom radical. Time has generally not been a significant constraining factor. Ac­countability for efficiency and effectiveness has been limited to internal concerns and considerations. The effort to introduce a comprehensive university-wide cooperative education program at Drake, via the award of a federal demonstration grant with a three-year time limit, meant that, traditional evaluation models, while instructive, were probably not ap­propriate to the situation.

While elements of a cooperative education program were in place and well-supported in some quarters, it remained a generally controversial venture. Clearly, the introduction and integration of such a program into the curricula of the several schools and colleges of the university would require substantial time and effort. The three-year duration of the demonstration grant limited the time available to develop and implement the program, much less demonstrate the need for and practicality of continuing the pro­gram at an expanded level after the funding was to end. That the project was funded by an agency external to the University meant not only that there was not to be a normal evolution of the program but also that the agency's intervention would entail an interest in the outcome of so sizeable an investment in attempted curricular change.

Moreover, there are distinct cooperative education program benefits that are based in sound educational values. For instance, through cooperative educational experiences, students may gain personal and career development outcomes (Wilson, 1974). As colleges and universities are con­fronted with budget cutbacks, valuing of educational programs, especially cooperative education programs, takes on special significance.

Clearly the evaluation of this program effort required the development of assessment tools that would provide more and different information than data supplied by more traditional evaluative techniques.

Evaluation research is receiving increased attention from managers and administrators of social programs. Evaluation research deals with valu­ing, in this case, the valuing of a program's worth and importance. Evalua­tion research can play an important part in program planning for a com­prehensive cooperative education program, especially since assessments common to most cooperative programs typically address the areas of process and outcomes. Very few assessments are designed to evaluate costs and benefits related to educational value.

In this model, three major program elements (see Figure I) are evaluated:

1. The objectives of the program, i.e., anticipated outcomes;
2. Processes or program functions, i.e., the policies, procedures and staff contributions;
3. The cost of the program in relation to the benefits achieved, i.e., expenditures as compared to educational contributions to the University.

There are two phases to program evaluation in this situation. Evalua­tion is an on-going process. This is particularly so in an evolving cooperative education program such as that at Drake University. Objectives must be continuously defined, refined and redefined as original goals and objectives are continually scrutinized. Consultants play a key role in this process, as does cooperation with the university's institutional research center, which can show the potential impact of goals on various areas of the university. The impact is wide.

The second phase of the evaluation relates the activities and their values to the University as a whole so that the new program can be shown to compete successfully with older more established values and programs. How can these two phases be developed?

Program Goals, Objectives and Outcomes

Criteria

In developing an evaluation model for a comprehensive cooperative education program, it is helpful to identify as many indices for use in the measurement process as possible. The development of precise, measurable program objectives is essential. The primary research goal of evaluation becomes the answer to the question, "To what extent are the objectives of the program being achieved?" As objectives are explored, it may be useful to delineate among goals, objectives, outcomes and criteria. The distinction rests with the level of generality of each kind of statement. Goals are logically mentioned first and are often phrased in general or global terms. Objectives tend to be narrower and usually short-range. Outcomes are the end result or products of program ac­tivities. Each, of course, is a legitimate target for evaluation and in whichever case, it is especially important that the outcomes are related to the objectives.

Another issue in the process of evaluation is the development of some means of measure and criteria selected to permit this measure. The ideal criteria should permit the researcher to answer all questions, "Has the ob­jective been attained, are we doing what we intended to do and are the results what we had hoped for?" (Anderson, 1975). The following example may enable a better understanding of the tying together of objectives, out­comes and criteria. In the case of a comprehensive cooperative education program, a major objective may be the projected level of student placement in the program. An outcome for measuring the success of this variable may he the number of students placed on the job. The criteria would be students \\Orking on the job. Then, too, the Federal Government has its criteria for granting federal funds to cooperative education programs. (See Appendix Ii. This approach enables the director of the program to evaluate more ef­fectively the particular as well as the overall success of the program.

Appropriate Objectives for a Cooperative Education Program

If one were to visit colleges and universities where the many cooperative education programs exist, one would see a great deal of diversi­ty in the institutions, the students served, the geographic location, and the community makeup. However, the diverse programs also exhibit great similarity in purpose. Some of these common purposes would include

1. Recruiting students to the program ( and employer),
2. Inducting or counseling students about the program (career preparation and job skills),
3. Placing students on jobs related to their major (job develop­ment and matching of student with employer),
4. Evaluating the student's progress and experience (another level is to evaluate the program impact on employers).

At Drake University, and in other comprehensive programs as welL three additional objectives take on significance when program evaluation is concerned:

5. Integrating cooperative education into the curriculum,
6. Conducting research and disseminating results, and
7. Institutionalizing cooperative education.

By integrating cooperative education into the curriculum We mean:

• The involvement of faculty in developing and implementing learning objectives as an integral part of each student's work experience and the awarding of academic credit
• The relation of those objectives and outcomes to course objec­tives in the student's college major,
• The development of courses and seminars for students prior to and concurrent with their work experience.

The sixth objective, "to conduct research and disseminate results", ad­dresses the concern of any demonstration model, namely, the quest for new knowledge and the need for it and the sharing of that knowledge with col­leagues.

And, finally, the ultimate objective of a demonstration grant is to in­fuse cooperative education into the mission of the institution and to innJ!w all colleges and units where it is decided appropriate.

We mentioned that evaluation begin with the setting of well-defined and specific objectives. In addition, it is clear that the measurement of the suc­cess of Drake's Comprehensive Cooperative Education Program, or any program for that matter, is dependent upon agreed-upon criteria that can be readily measured and evaluated in terms of program outcomes. Through the evaluation process, internal and external evaluators can more readily determine findings and make recommendations for alterations and im­provement. This process in turn lays the foundation for claims against in­stitutional resources when funding ceases. Perhaps Drucker best summar­ized the use of such objectives:
Objectives are not fate; they are directive. They are not commands; they are commitments. They do not determine the future; they are means to mobilize the resources and energies of the institution (business)' for the making of the future" (1974, p. 102).

Processes for Evaluation

In order for a program to evolve in a coherent and cohesive manner, it is necessary to have an underlying framework and strategy that binds together various procedures and policies governing the development. This strategy allows the program manager to relate resource decisions directly to developments and goals, which in turn promotes performance which can be measured.

A continual evaluation of the processes and functions of the program is critical to the successful development of a comprehensive cooperative education program. The goals of the program must be educational in in­tent, design and practice. Processes must be integral and functional and at the same time should be identified and monitored through written or verbal means.

The following are processes that were identified as necessary to the im­plementation and maintenance of Drake's Comprehensive Program:

• Administration: Administering various groups and constituen­cies (e.g., centralized vs decentralized management, planning, supervising, budgeting).
• Policy-making: Communicating, through written or verbal statements, how aspects of the program operate (e.g., pro­cedures for giving academic credit, student pay, number of multiple work experience'>).
• Recruitment of Students: Recruiting students on or off campus (e.g., letters to prospective high school seniors, posters, group meetings).
• Conseling: Orienting prospective students to the program (e.g., establish needs and interests, meetings on resume preparation, job interviews).
• Curriculum Development: Designing and implementing for­mal cooperative education curricula by faculty (e.g., seminars, co-op preparation classes, designated new courses).
• Faculty/Staff Development: Introducing faculty and staff to. cooperative education philosophy and procedures (e.g., con­ferences, in-service workshops, meetings).
• Job Development: Contacting employers about positions for students (e.g., personal visits or telephone call'>, employer in­stitutes and meetings).
• Student Placement and Supervision: Placing students on jobs related to major. Evaluating student performance (e.g., types of positions, length of student employment, increase of stu­dent responsibilities).
• Collection of Data: Collecting and retrieving information (e.g., student and employer records, program evaluation data).
• Communication and Dissemination: Promoting the program and sharing results with colleagues (e.g., newsletters, journal publications, research, conferences).
• Program Evaluation: Measuring program successes and weaknesses (e.g., internal, external, summative, and for­mative evaluations).

By identifying these processes and developing measures, ad­ministrators may more effectively determine the effectiveness and efficiency of their activities. Other institutions may have a somewhat different set of circumstances peculiar to their uniqueness; the point is that their identifica­tion and evaluation with reference to effectiveness and efficiency is impor­tant to the program.

Cooperative Education's Contribution to the University

The question of a program's educational contributions to the Universi­ty is an area that will demand considerable attention by top administration when allocating funds.

Evaluation of the Cooperative Education Program at Drake will take into account more than the actual growth of the program itself. Additional­ly, cooperative education can be evaluated in terms of the benefit it pro­vides to the University. The program's contributions to the institution are both quantitative and qualitative. For practical purposes those contribu­tions can be quantified and examined separately.

Financial Aid

At Drake, cooperative education is offered as an alternative to work­ study in the student's financial aid package. Those selecting co-op who are then placed in paying co-op jobs do not use allocated work-study moneys. For example, in the academic year 1981-82, the placement of thirty-five students in cooperative education resulted in $39,392 in freed work-study moneys. Sixty-six students accepted co-op instead of work-study for the academic year 1982-83. Placing all of them would return $81,706 to work study funds, to be used to help other students with financial need.

Recruitment

Sample surveys conducted during registration periods were under­taken to measure student interest concerning cooperative education and, particularly, to detect any change in interest during the period of the Demonstration Grant. The response size for these surveys ranged from 7.5 to 13.5 percent of all registering students; the rate of return was at least 70 percent for each of the surveys. For example 550 survey forms were distributed during the Fall 1982 registration. The percentage of total student'> responding, to "Co-op contributed to decision to attend Drake" in­creased from four percent to nine percent from Spring '81 to Fall '82. This includes 20 percent of all freshman respondents registering for the Spring '82 semester and 14 percent of all Fall '83 freshman respondents.

Retention

Currently, a data file is being constructed which will be used to com­pare the retention rate of cooperative education/internship students to the retention of the remainder of the undergraduate student body. The data file will extend from 1977 to 1982.

Cooperative education may have an underlying effect on student retention in that it increases students' satisfaction with their Drake educa­tion. A survey was distributed to all Summer '82 co-op students (N = 204). Thirty-five percent of the respondents indicated that they felt that their job experience helped them to better understand their course work to some ex­tent; 42 percent said it helped to a great extent. Sixty-three percent agreed that, to a great extent. their co-op jobs provided a rewarding learning ex­perience.

Generating Tuition

A positive impact by the Cooperative Education Program on recruit­ment, retention, and financial aid savings ultimately leads to tuition generation.

Those cooperative education students who participate in the program during the summer are a direct source of tuition revenues. During the Sum­mer '82 semester, at least 132 credit hours were taken by co-op students from the Business, Education, Fine Arts and Journalism Colleges. This translates into income through generated tuition, assuming that their enrollments were related to the internship and that they would not have at­tended otherwise.

Placement

Of the co-op students surveyed during Summer 1982, 68 percent agreed that co-op helps to make professional contacts for jobs. Impact on placement may be more directly assessed by following co-op students after graduation. For the 1981-82 academic year, the Career Development Center has identified co-op students in the Career Development Follow-up Study. Placement among the co-op group has been compared to that of the remainder of the August '81, December '81, and May '82 graduating population. An equal proportion (86 percent) of both groups were found to be employed, either full-time or part-time. The representativeness of this study is questionable regarding co-op respondents; however, with modifications in the follow-up survey instrument and technique, more em­,phasis may be placed on 1982-83 findings.

A survey which has recently been distributed to co-op employers for the 1981-82 academic year (N = 247) has also yielded information related to placement. The questionnaire inquires as to whether Drake students, specifically co-op students, have been hired by the participating organiza­tion. Thirty-seven percent of those co-op employers who responded (N = 155) indicated they had hired at least one Drake graduate within the past two years. Of those hired, 49 percent were reported to have been co-op students.

Outside Moneys Generated

The Cooperative Education Program, since receiving the Demonstra­tion Grant, has sought additional outside funding. To date, receipts include two Supplemental Grants for a total amount of $14,600. A Special Com­mittee for Outside Funding has been formed, to seek additional grants.

Community Service

Drake University was permitted to involve the Law and Pharmacy School interns as part of the total cooperative education program. These students are supervised, earn academic credit, and receive multiple ex­periences; however, a proportion of these co-op students work for local organizations and receive little or no pay. When the average number of hours (15) worked is multiplied by the minimum wage standard ($3.65), a sizable contribution per student is generated. The amount of foregone wages for services provided to outside organizations by Drake Cooperative Education students can be significant. Thus, cooperative education con­tributes a service function to the community. which is also a goal of the University.

Qualitative Contribution

Last, there are also qualitative contributions of cooperative education to the University. If students, faculty and employers are satisfied with the program, an important statement has been made. Follow-up surveys of each of these groups and their positive results can indicate to top ad­ministration that there is strong need for a program of this nature at the university (Korngold and Dube, 1982). More importantly, the combination of quantitative and qualitative data showing the impact of the curricular program on the University mission indicates that institutionalization is tak­ing place.

Summary and Conclusions

By presenting relevant data on program objectives, processes and cost benefits to external evaluators and internal audiences, a powerful case can be made for program accomplishments, and the value of the program can be demonstrated to top administration. Thus, an evaluation model, such as previously outlined, speaks to a critical issue in cooperative education, "What happens after Federal funding?" Cooperative education programs need not suffer from a lack of systematic planning, inappropriate identifica­tion of objectives before implementation or inadequate data collection and its use. An evaluation model such as this provides a sound base for decision making.

Much more examination of this topic needs to take place by other com­prehensive program administrators. However, an important step in this analysis has taken place by discussing the importance of objectives, pro­cesses, and "cost benefits" and their value for the educational mission of the supporting institution.