A critical analysis of interpretive research studies in cooperative education and internships

This critical analysis of research into cooperative education and internships seeks to complement a review of quantitative research in co-op by Bartkus (2007) recently published in the Journal. Research in co-op, like much educational research, has been subject to some criticism (Bartkus & Stull, 2004; Ricks & Mark, 1997) - in particular, for lacking a solid theoretical basis (Eames & Cates, 2004; Eames & Coll, 2006; Russell, 2006; Quay, 2003). Quantitative research in co-op very seldom explicitly identifies its theoretical underpinnings (Bartkus & Stull, 2004), but it generally seems to be based on a comparative/experimentalist approach (or quasi-experimentalist approach), with roots in empirical-positivism. However, in interpretive ¹ studies, authors more typically identify at least some sort of theoretical basis to their work (Quay, 2003) even if their work is not about theory testing as such (Creswell, 2003). Some authors argue that quantitative work is superior to interpretive work (see, e.g., discussion in Trochim, 2000) - others argue that they simply represent different approaches to the same basic educational issues, with different aims and methods of inquiry (Coll & Chapman, 2000; Guba & Lincoln, 1989, 1994; Ricks & Mark, 1997; Patton, 1990).

The analysis of qualitative research presented here is based on the idea of a meta-analysis used in quantitative research. A metaanalysis is related to, but differs from, a literature review in that a meta-analysis seeks to analyze a collection of studies about a set of related questions or hypotheses. Meta-analyses done on quantitative work apply statistical analyses across related studies in order to allow us to achieve more confidence in the findings. Typically the sample used in quantitative studies is much larger than that interpretive studies. However, Bartkus (2007) observes that many co-op quantitative studies also suffer from small sample sizes, which he argues reduce the applicability or generalizability of their findings. The intention in a meta-analysis is to in effect increase the sample size, making the findings from a meta-analysis more widely applicable than for a single quantitative study. This is the same rationale used in the present work, but the approach used is different in that it draws upon different methods for evaluation of research quality (i.e., non-statistical methods). Hence, as in most meta-analyses we have screened published work for both relevance and quality before inclusion. The details of how we did this are presented below. The research question that formed the basis of this critical analysis is: What is the nature of interpretive/qualitative co-op research with respect to output, diversity, quality, and research themes?

This analysis seeks to identify interpretive research inquiries into cooperative education and internships, published approximately over the last 30 years. All forms of published media were canvassed including printed journals, electronic journals and conference proceedings. The analysis examined the major journals dedicated to co-op; viz., the Journal of Cooperative Education & Internships (and its predecessor, the Journal of Cooperative Education), the Asia-Pacific Journal of Cooperative Education, the Journal of Vocational Education & Training; and the Journal of Experiential Education. It also sought to locate co-op-related articles published in other non-co-op journals, and books/monographs or handbooks on cooperative education. For the latter publications in particular, the focus was on interpretive research, rather than reviews of research or essays about co-op (except in a few cases to provide background commentary). Conference proceedings (e.g., WACE biennial conferences, AsiaPacific regional biennial conferences, CEIA and other North American conferences, along with national conferences from South African, Australia and New Zealand, also were included), also were examined, but not conference papers. The latter were deemed too hard to locate by the authors of this paper and co-op researchers generally, meaning analysis based on abstracts alone was unlikely to be meaningful.

MODALITY

The research approach utilized in this critical analysis consisted of identifying articles from the sources above, and subsequent analysis based on the following criteria:

1. The theoretical basis to the work (if identified explicitly in the work);
2. The research methods employed (interviews, observations, document analysis, etc.);
3. The educational context or situation of the inquiry;
4. Any measures taken to enhance the quality of the study, that is, validity and/or reliability, as conceptualized by the authors;
5. An analysis of the overall quality or trustworthiness of the inquiry, based on the framework provided by Guba and Lincoln (1988) (see below);
6. The key research findings based on a thematic analysis; and
7. A consideration of the implications of interpretive inquiries for the practice of co-op programs.

In terms of relevance to co-op, to warrant inclusion the paper had to deal with programs that were explicitly identified as co-op or internships, or for which there was evidence the program involved the integration of on-campus and off-campus learning or at least an intention that some integration would occur (e.g., if it failed that too is of interest). So papers about technical training were included if the research dealt with an aspect of workplace learning that formed part of a formal program of study. Papers were included if they were explicitly identified as interpretive or qualitative by their authors, or that in the judgment of the present authors they showed clear evidence of the use of methods such as interviews, or observations. A few papers that utilized quantitative methods were included. Such papers were case studies that used a simple quantitative method (such as an in-house survey) as part of a case study, and typically in conjunction with qualitative methods such as interviews. The distinguishing feature was the intention of the use of quantitative methods. If the use of numbers was for purely descriptive purposes the paper was included, if it was for comparative purposes (typically involving the use of tests for statistically-significant differences) as part of an experimental study, then it was excluded. Interestingly, one might expect quantitative studies to involve larger numbers of participants, but as Bartkus (2007) notes, many genuine quantitative studies involve a relatively small number of participants (i.e., less than 100), and almost inevitably employ convenience samples. In summary, a study was deemed interpretive if it was identified as such by its authors, if it employed qualitative methods (or mixed methods), was plainly a case study, or clearly showed no intention to generalize the findings.

THEORETICAL BASIS FOR THIS CRITICAL ANALYSIS

Background

Consistent with our proposition that quality research studies in co-op should be grounded in theory, we present a discussion of the theoretical basis to this critical analysis. This is deemed necessary so the reader can track the process, and understand the analytical tools used to inform this analysis.

At the heart of evaluation of interpretive work is an understanding of the theoretical basis to such work, and how this compares with, and is distinguished from, the theoretical basis of quantitative educational research. First we consider the basis to quantitative work which has as a key assumption the notion of determinism, where it is believed that by identifying and interrelating variables, the specific behavior within the system can be known (or determined). In simple terms this means that events have causes, that events are determined by other circumstances. Causality is used to define relationships among empirical variables on a cause-and-effect basis that can be explained or manipulated to produce conditionally predictable outcomes; viz., if X occurs, then Y will be the effect. To illustrate with a co-op example, if students do a co-op then we might investigate whether or not their starting salaries are higher, or they advance more rapidly within their employing organization.

A second key assumption is generality - related to both deductive and inductive methods of reasoning. This also is derived from a scientific world view, and through observation, researchers in the natural sciences attempt to generalize their findings about the inanimate world. In contrast, interpretive researchers generalize their findings only with great caution as we shall see below.

Theoretical Basis to Interpretive/Qualitative Research and Quality Criteria used in this Critical Analysis

The ideas developed from an empirical-positivist paradigm mentioned above are consistent with the methodological assumptions of traditional quantitative educational research (Guba & Lincoln, 1989). Quantitative researchers believe that there is a reality 'out there' to be studied, that is set apart from the beliefs of individuals, and that it can be captured and understood (Denzin & Lincoln, 2003). Interpretive research works from very different theoretical and (concomitant) methodological assumptions, and in terms of the measures needed to enhance the quality of such research (Guba & Lincoln, 1989; Quay, 2003).

The traditional means of judging the quality or rigor of an educational research inquiry is by reference to four criteria: internal validity, external validity, reliability, and objectivity (Altheide & Johnson, 1994; Guba & Lincoln, 1989; Lincoln & Guba, 1985; Merriam, 1988). The internal validity of an inquiry is the extent to which variations in outcomes can be ascribed to controlled variation in dependent variables - in other words, how well the research findings match reality. External validity has its focus on the ability to generalize the findings of the inquiry to a target population. Reliability, a precondition for internal and external validity, refers to an inquiry's consistency, predictability, dependability, stability and accuracy. In other words, the question asked is, how can we be sure that if this measurement were repeated we would get the same results? Objectivity addresses the desire for neutrality, that is, freedom from the biases, prejudices, and personal perspectives of the investigator; put simply, how much are the findings of the inquiry influenced or determined by the participants, particularly the investigator?

As noted above, these criteria for judging the quality of an inquiry are consistent with an empirical-positivist paradigm (Altheide & Johnson, 1994; Janesick, 1994; Lincoln & Guba, 1985). However, objections have been raised as to their suitability for judging the quality of interpretive inquiries (Altheide & Johnson, 1994). Hence, different criteria have been proposed for interpretive inquiries, to judge what Guba and Lincoln (1989) refer to as the trustworthiness of the inquiry. Guba and Lincoln (1989) propose that credibility replace internal validity, dependability replace reliability, confirmability replace objectivity, and transferability replace external validity.

Credibility can be judged from the degree match between individual's views of reality (e.g., views about the value of co-op), and it is enhanced by a number of factors: prolonged engagement, persistent observation, peer debriefing, negative case analysis, member checks, and progressive subjectivity. The purpose of prolonged engagement is to provide the opportunity to establish good rapport and trust with participants, allowing the researcher to overcome the effects of presented fronts (e.g., co-op students not wanting to appear negative about their co-op experiences because they know the interviewer). Persistent observation allows the researcher to identify the characteristics and elements in the inquiry that are most relevant to the issue under inquiry. Peer debriefing helps the researcher to understand his or her own position and values and their role in the inquiry, via reflection. Negative case analysis involves revising the working hypothesis in the light of hindsight; that is, looking to test an emerging conclusion, by seeking opposite conclusion (which if absent or 'negative', then is seen to provide support the original conclusion). Member checks - the process of negotiation with participants and stakeholders - provide participants with the opportunity to confirm individual data (e.g., getting interviewees to read transcriptions of their interviews so as to confirm they are an accurate picture of their views). Finally, progressive subjectivity - monitoring the researcher's own ideas - serves to remind the inquirer that in interpretive inquiry the researcher's views should not be afforded a higher status than that of the participants: what emerges in data should be a joint view (i.e., allow the participants' 'voices' to be heard as clearly as those of the researchers).

Dependability is concerned with the stability of data over time. It is perhaps on the issue of dependability that interpretive inquiries differ most from their empirical-positivist equivalent of reliability. Guba and Lincoln (1989) point out that methodological changes and shifts in construction are the very life-blood of interpretive inquiries, and do not impact adversely on dependability. For quantitative inquiries, alterations in research design are thought to expose inquiries to unreliability. In interpretive inquiries, these changes are seen as an integral part of the inquiry process and, rather than representing flawed methodology, indicate increasing maturation of the inquiry. What is critical, is that the changes and shifts in thinking be clearly identified and fully described, they should be "tracked and trackable" (Guba & Lincoln, 1989, p. 242).

Confirmability, like its empirical-positivist equivalent objectivity, seeks to ensure that the results of an inquiry have not been subject to influence by the investigator. The usual means of ensuring objectivity in quantitative work is via strict adherence to method that is determined in advance (Guba & Lincoln, 1989). However, in interpretive inquiries, the confirmability of an inquiry rests on the data themselves. Hence, the raw data and the process used to reduce them or code them are made available to the reader (Cronbach & Suppes, 1969). So we might describe how we did the interviews, provide the actual questions asked, and provide substantial portions of interview transcriptions to show how the interviews were actually conducted. This provides an audit trail which is intimately linked with the tracking process necessary to assure dependability. Such measures represent conventional ways interpretive researchers seek to address the complex issue of confirmability/objectivity. A further key issue is, however, that the researcher/s may not be aware of any personal bias. As a number of authors have pointed out, the researcher is the interpreter, and interpretation is thus inevitably influenced by the researcher's own prior assumptions, prior experiences and personal biases. Richards and Patton, for example, argue that interpretive researchers need to make a genuine attempt to identify their own biases in advance (Patton, 1990; Richards, 2005). So an interpretive researcher might make explicit his/her prior assumption. For example, that the co-op researcher has assumed honesty in interview responses, that the sample may be biased because only good students or students/others we have a good relationship were used in the sample. Such assumptions may in turn influence the coding of a thematic analysis. We may, for example, only seek out data which support our biased a prior assumptions (Costas, 1992), and this may be lost in any data coding we subsequently employ.

Finally, transferability is the interpretive equivalent to external validity or generalizibility. In quantitative inquiries, generalizability is strongly dependant on sample selection, particularly on the random selection of the sample (Merriam, 1988; Patton 1990; Wiersma, 1986). Ideally a target population is identified, and a selection of participants made via a random sampling procedure or some more efficient variation such as stratified random sampling. As noted earlier, Bartkus (2007) notes this is seldom achieved in reality in co-op research. In interpretive work, sampling is intentionally purposive or convenience in nature (Patton, 1990). Guba and Lincoln (1989) and Geertz (1973) suggest that the nature of interpretive inquiries is such that the researcher describes the context of the inquiry and provides detailed descriptions of methodology and interpretation; it is then up to the reader to decide if the findings are relevant or pertinent to their own situation (Merriam, 1988). So, for example, we would describe our co-op program in detail, talk about the institutional setting and students and employer, and so on and the reader can then compare our situation with his or her and decide if there are differences and decide if they matter.

We have covered these ideas in a fair amount of detail here, because they formed the basis of a key aspect of this critical analysis: namely, the analyses of the studies with respect to quality. In order to conduct our critical analysis we felt a need to screen or evaluate the quality of interpretive work on sound criteria, criteria that are made evident to the reader.

FINDINGS: NATURE AND SCOPE OF INTERPRETIVE CO-OP RESEARCH

The inclusion criteria were negotiated between the authors in advance, and the papers were categorized by the authors separately and compared; any disagreement was negotiated. The modality detailed above resulted in the identification of 119 articles from the Journal of Cooperative Education/Journal of Cooperative Education & Internships, of which 35 were deemed to fit the initial selection criteria. For the Journal of Vocational Education & Training, some 83 articles were identified of which 61 fitted the selection criteria. For the Asia-Pacific Journal of Cooperative Education 31 were reduced to 18, and for the Journal of Experiential Education 17 were reduced to 7. There were 24 articles identified from conference proceedings, of which 20 were deemed to meet the criteria. In total then we used 141 articles in this critical analysis.

Theoretical Basis or World views Used in Interpretive Co-op Research, and Research Methods Employed

There are two aspects to the 'theoretical basis' to co-op interpretive research. First, is the notion of a paradigm such as empiricalpositivism or interpretivism to which the researcher/s subscribe. Second is the notion of a specific theory used to inform the methodology, such as ethnography or grounded theory. One might argue that each methodology has implicit assumptions about paradigms and so on (e.g., grounded theory and ethnography have their origins in an interpretive paradigm), but we were interested to see if authors explicitly identified the paradigm they adhered to, and in turn explicitly linked this to the methodology, or vice versa. It is worthwhile to note here that authors seldom developed or created specific theory or links from their findings to extant theories. Hence, what we are talking of here are theories that authors identified prior to engaging in research. One might then argue these are not so much theories, as the personal world views or epistemological assumptions of the authors that they used to inform their research. That is, the authors were not engaged in theory creation.

As might be expected given our section criteria, the theoretical basis/worldview of the studies, where identified, reflected the nature of interpretive research. Authors sometimes directly specified a paradigm (i.e., interpretive/qualitative) or a particular theoretical basis to their work (e.g., sociocultural theories of learning) which had its roots in interpretivism. However, in most cases the paradigm or theoretical basis was simply alluded to, rather than explicitly stated. In about two thirds of the articles reviewed, the authors did not explicitly specify any paradigm or theoretical basis to their work.

Far more common was to specify the research approach or methodology, but many authors failed to link this to any paradigm (e.g., interpretive) or theoretical basis such as an epistemology (e.g., constructivism, or sociocultural theories of learning). To illustrate, in terms of research approach or methodology, a reasonable proportion of studies said they were case study. Mostly such case studies were identified by the authors as narrative or naturalistic in nature. However, the latter is a genuine theoretical perspective, the former is more to do with how the case study is presented or described. In a similar way, some authors talked of their work involving 'grounded theory', which is more of a research approach or data analysis approach than a theoretical basis (but which does carry a set of epistemological assumptions as discussed above). Other signs of confusion between approach and paradigm/theory were descriptors such as a cultural-historical, ethnographic and phenomenographic approach. These are legitimate research approaches, but authors typically failed to explicitly link them to an interpretive paradigm as might have been desired. More recent research found in the analysis drew upon modern theories of learning such as sociocultural theories or situated cognition/learning, which were grounded in interpretivism. A few took a critical theory or related perspective - some of which identified paradigms (e.g., post-modern), and others research approaches (e.g., feminist or critical ethnography).

Research methods employed were, not surprisingly, dominated by qualitative methods of enquiry, in particular interviews, and specifically semi-structured interviews. Surveys also were quite common, but as noted above these were descriptive in nature. Documentary analysis featured on occasion - this included analysis of reflective journals, student reports on their placements and policy documents. In general there was a logical link between the specific methods used (e.g., interviews, observations), and the research approach and paradigm/theoretical basis (e.g., interpretivism). Many studies employed some triangulation² (e.g., mixed methods triangulation), but a more detailed critique presented below showed quality assurance measures were not always dealt with appropriately. The number of participants in a given study was usually modest (ca. 20), but a few employed a fairly large number of participants (e.g., one study involved interviewing 478 participants).

Research Quality of Interpretive Co-op Research

The research quality of each paper was evaluated according to the criteria above, and these data are summarized in Table 1.

It may seem an odd thing to quantitatively analyze the quality of interpretive research in an analysis such as this! However, we employed a fairly broad measure with each paper rated 1, 2 or 3 for each of the criteria in turn. Here, 3 means we felt it clearly met the criteria specified by Guba and Lincoln (1989), and 1 meant it failed on at least half the counts, with 2 being in the middle. Thus the purpose here was to provide a 'broad brush' look at the quality of these highly diverse enquiries. Each author of the present work classified the studies independently, and comparison was made. Any disagreement was resolved by negotiation. There was little disagreement observed, perhaps because the classification scheme used was discussed in detail in advance, and fairly broad in nature.

The summary data in Table 1 paint a fairly positive picture of interpretive co-op research, probably indicative of the rigor of reviewing. The best performance is in the area of credibility. If we examine the descriptors of the criteria, this would mostly be satisfied by a sound description of the methods of inquiry employed (something editors and reviews probably place emphasis on). Weakest are the areas of dependability and confirmability, which is broadly consistent with the main criticisms of interpretive work by its quantitative critics (viz., it is neither replicable nor objective). Interestingly, transferability is handled reasonably well, and this is consistent with the good ratings for credibility – probably for the same reason; viz., such descriptions attract the attention of reviewers (i.e., an audit trail and detailed description of research methods employed in the inquiry).

We considered ranking the papers with a view to giving more weight to those deemed more trustworthy. On balance we decided against this, because we felt each paper has survived this 'cull', and thereby had something to contribute. Additionally, it was not obvious how we might rank papers. For example, it would be hard to decide whether to give more credence to a paper that rated more highly for confirmability versus transferability. But overall the fact that the papers retained all scored relatively highly (i.e., the lowest mean was 2.3 out of a maximum of 3), and this suggests to us they are meritorious enough to warrant inclusion.

Diversity and Research Themes for Interpretive Co-op Research

We identified considerable diversity in these studies in terms of program structure (i.e., internships, co-op, training), disciplines of the programs of study (e.g., teaching, hospitality, sport, science & engineering, management studies), and geographical location (USA, Canada, Australia, UK, Thailand, New Zealand, Germany, Sweden, South Africa, etc.). The research outputs are, however, dominated by a few countries: the USA, UK and Australia and New Zealand. This is probably at least in part due to the journals canvassed; for example, the Journal of Cooperative Education & Internship.s contains mostly USA-based studies, the Journal of Vocational Education & Training, mostly UK-base studies, and so on.

FINDINGS: THEMATIC ANALYSIS OF INTERPRETIVE CO-OP RESEARCH

Given the immense amount of work canvassed we could have grouped findings in almost any way, or indeed used grounded theory to allow themes to emerge naturally from the data, or to code studies in some pre-determined way. The papers that met the criteria for inclusion were subsequently analyzed thematically as they related to the research question for this critical analysis; viz., What is the nature of interpretive/qualitative research with respect to output, diversity, quality and research themes. The following key research themes were subsequently identified: Co-op students' personal growth; co-op's contribution to broader educational and societal needs; issues in co-op practice and management; and employer and other co-op stakeholder views. Each of these is now discussed in turn. We first describe how papers match such themes - but also consider how they vary or expand the themes. This is because the idiosyncratic nature of interpretive research is an integral part of such work, therefore findings do not necessarily have to be replicated to be of equal interest.

Theme 1: Co-op Students' Personal Growth

A number of researchers report that co-op, particularly the workplace aspect of co-op, aids student personal growth. Such research reported on student perceptions, not evidence that, for example, co-op actually did help career growth. This included a number of aspects of personal growth, such as skill acquisition (see, e.g., Cullen, 2005, 2007; Göhringer, 2002; Pickles, 1993), direct enhanced career benefits or career prospects (Auret, 2003; Busby, 2004; Cheung & Wong, 2006; Hutchinson et al., 2001; Jordan, 2003; Spielhofer, 2001), and the development of specific technical or 'hard' skills (Beard, Coll & Harris, 2001; Coll, Eames & Halsey, 1997; Martin & Leberman, 2005; Menon, 2002). Perceived benefits reported also included more holistic views of what is learnt (Busby, 2003; Williams, 1991), that is, more than just skill acquisition or training. For example, it is reported that students feel that they develop in terms of self-efficacy (Coll & Lay, 2001; Coll, Zegwaard & Lay, 2001), self-confidence and self-esteem (Fletcher, 1989, 1990; Tierney & Slack, 2005; Wellington, 1992), intellectual development (Keen, 2001), capacity for reflection on their learning (Cates & Langford, 1999; Green & Holloway, 1996; Hayward, DiMarco, Kranz & Evans, 2001; King, 2004; Quezada & Christopherson, 2005; Shibley, Shivbley & Knell, 2006; Wong & Coll, 2001), and development of 'soft' skills generally (Auret, 2003; Eames, 1999; Holt, Mackay & Smith, 2004; Lount & Hargie, 1998; Smith-Eggeman & Scott, 1994). Students are reported to feel they gain a sense of what it means to be a practicing professional in a particular discipline or area of study such as teaching (Ahola-Sidaway & McKinnon, 1999; Dyson, 2005; LaMaster, 2001), engineering (Allen & Peach, 2007; Apostolides, 1995; Eames, 2003a, 2003b; Hutchinson et al., 2001), sport studies (Martin & Leberman, 2005), the hospitality industry (Girad, 1999; Spowart, 2006; Walo, 2001), and music and creative industries (Draper & Hitchcock, 2006; Southcott, 2004; Sterling, 2007). Students also are reported to become more reflective and lifelong learners (Bathmaker, 2001; Hillier, 1999; O-Shea & Watson, 2007; Wong & Coll, 2001).

Theme 2: Contribution to Broader Educational and Societal Needs

Coll and Eames (2004) argue that co-op and education generally need to provide a more holistic contribution to society and societal needs than just developing work skills desired by employers (see also Hall, Hardman, Smith & Taverner, 1999; Sutton & Stephenson, 2005). A number of authors have examined how, and in what ways, this might work (Cutt & Loken, 1995).

Groenewald (2003) argues that co-op allows for talent growth, mostly because of the holistic nature of the co-op experience (particularly the workplace experience). Higher order learning goals like this are, according to Collier and McManus (2005), facilitated by learning partnerships. It seems well-constructed learning partnerships can aid the transfer of learning to the workplace, as can effective mentoring (Huddleston, 1998; Jones, Wu & Hargrove, 2002; MacLaren & Marshall, 1998; Van Gyn & Ricks, 1997; Smith, Mackay, Challis & Holt, 2005), a supportive learning environment (Somerville, 2006), and relationship building (Smith, 2002). Conversely, the lack of mentoring (Cornford & Gunn, 1998), a heavy reliance on the instructor (Smith, 2003), or a very informal approach (Barber, 2003) to learning are seen as inhibiting of learning. Informal or tacit learning, it is reported, needs strong workplace or off-campus support structures (Dalton & Smith, 2004; Viskovic, 2005), or to be well structured if it is to be of value in terms of advancing learning (Brooker & Butler, 1997). Learning models for experiential learning (Diambra, Cole-Zakrzweski & Booher, 2004; Peterson & Nelson, 1986; Velde, Cooper, Harrington & Mailler, 1999), the setting of learning objectives (Weaver, 1993), and pedagogies that make use of experiential and social forms of learning in an environment that helps students connect to the future (Hager, 2004; Lumby, 2007), also are reported to aid learning. Zanibbi et al. (2006) argue that there are now theoretical models that serve to validate the notion of exemplary practice in co-op, at least in terms of successful entry into the workforce.

Mitchell and Poutiatine (2001) observe co-op helps in the development of leadership – provided appropriate curricula are in place. Co-op can play a role in equity with, for example, Keeling et al. (1994) stating co-op helps identify career issues for people of color (e.g., realistic career goals), and Thurtle, Hammond and Jennings (1998) saying it helps in the case of gender minority situations. Cooke and Brown (2006) argue that co-op can help understand, and potentially address, complex societal issues such as the integration of new migrants into the workforce.

Analysis of workplace discourse points to significant culture-based issues related to power relationships (James & Coleman, 1998), and lack of communication because of workplace cultural differences (Hodgson, 1997). That is, co-op experiences can help students better understand the intricacies of the particular workplace culture (Cooke & Brown, 2006; Eames, 2003a, 2003b; James, 1997; Sutton & Tse, 1997). Likewise, McCain (1986) reports that co-op provides a future for the disabled, because it can exemplify how any individual can contribute to the workplace and society as a whole, and Hudson (1996) states co-op helps low-achieving students by helping them overcome the notion that they were society's 'expected failures'. Christianson (1985) explains how co-op helped her cope in dealing with the terminally ill, Ayling (2007) about developing a moral business person via co-op experiences, and Hall and Raffo (2004) about how it helps the transfer between workplace and school settings for disengaged and disaffected students. An interesting study by Atherton (1999) looked at co-op's influence on student 'resistance to learning'. Apparently, the 'crisis' nature of many co-op learning experiences or 'learning moments', helps students to see the need to be open to learning in different ways.

Theme 3: Issues in Co-op Practice and Management

A number of issues or problems are reported with respect to co-op placements and in the management of co-op programs. For instance, Fleming and Eames (2005) argue that students need to spend a minimum amount of time in the workplace to gain much in terms of learning, and that the longer they spend, or if they do multiple placements, the more they learn. Co-op learning is facilitated by the use of sociocultural 'tools' (Lave & Wenger, 1991), such as artifacts (Walker & Stehlik, 2007), projects or project work (Bloomfield, Tyalor & Maxwell, 2004; Davies & Farquharson, 2004; Svensson & Snis, 2004), appropriate use of industry-specific language (Eames, 2003a, 2003b; Hagerty, 1980-81), or audio-visual teaching aids such as interactive instructional television (Stuart, 1999), via virtual learning (Bird, 2001; McSporran & King, 2002; Murray, 2006), or the extensive use of telecommunications (Martz, Repka, Kramer & Reale, 1999). The link between on-campus and off-campus, or workplace learning, has been explored by a number of authors. It seems that whilst authentic learning does occur in the workplace (see Eames, 2003a, 2003b; Harris, Willis, Simons & Collins, 2001), there is a significant theorypractice gap for many co-op programs (Allen & Peach, 2007; Dye, 1999; Stevenson & Yashin-Shaw, 2004). In other words, there is a lack of integration of theory and practice, in the management of programs (Butler & Brooker, 1998), in integration of learning (Hodkinson & Hodkinson, 1995; Hudson, 1996; Warmington, 2002), or failure to take into account how student learning occurs both on and off-campus (Beckett, 2001; Eames, 2003a, 2003b; Moodie, 2002), with learning often reported as being transmissive or highly instrumental in nature (Holloway, 1992).

Perennial issues such as assessment of workplace learning and evaluation of co-op programs surfaced as might be expected (Green, 1999; Harkin, 1998; Tillema, 2003; Tizard, 1995). According to Velde (1997), there is a need for the development of a model for the evaluation of co-op's competencies based on a more holistic and multi-faceted conception of competence. The use of portfolios may be the most useful way of looking at this issue (Hodges & Ayling, 2007; Jones, 2002). Because of the diversity of workplace environments, skills to be assessed and general complexity of assessing a dual-sector experience is problematic (Jones, 1992; Velde, 1997). The involvement of some student self-assessment is considered as being of merit (Tillema, 2003), since it produces a more reflective learner (Hodges & Ayling, 2007).

Theme 4: Employer and Other Stakeholder Views

A number of studies investigated employers' or other stakeholders' views of co-op and its practice. Similar findings to those reported for quantitative work are reported (Braunstein & Loken, 2004; Wellington, 1992). Ferkins (2002) reports a number of employer recommendations for co-op practice. According to employers, students need to network, act professionally, and be career-focused while on placement (professionalism being something students also see as important, see Talbot, 2000). Likewise, employers think higher education co-op providers should better prepare students, be more flexible about placement timing, and maintain more effective communication (Powell, 2001). Interestingly, despite much rhetoric to the contrary, research suggests that employers still value technical skills highly (Gerber & Velde, 1997).

As seen in numerous quantitative studies, employers report gaining benefits from co-op interaction and similar findings are reported for employers generally (Chapman, 1997), and also for small and medium enterprises in particular (Case & Hoy, 1981). Cullen (2005), for example, says benefits include screening for future employees and enhanced link with universities (see also Hartley & Smith, 2000). However, Spielhofer and Sims (2004) comment that the employer too has a role to play in developing students' skill base, and others note that modern apprentice programs suffer from a variety of things - mobility of workers which may arise from lack of employer support for learning (Gorard, 2003), or a lack of commitment to helping learning - mostly for financial reasons (Gorard, 2003; Unwin & Wellington, 1995).

Other stakeholder views also have been researched. Hartley (1988) provides a fascinating study of some 'movers and shakers' of the American co-op scene (namely, people who won awards for their contributions to co-op). Interestingly, it seems participation in co-op as a career for such people was in fact highly beneficial (see Lazarus, 1992 for a description of co-op benefits for experienced senior level administrative staff, and Beattie, 2003 likewise for line mangers), contrasting with other claims that co-op adversely affects faculty career development (Coll & Eames, 2000). There is little interpretive research about higher educational institutions' perspective on co-op. However it seems that they accrue similar benefits to those reported in quantitative studies (Weisz & Chapman, 2004), such as in student recruitment and more interaction with industry (Martin, 1999).

CONCLUSIONS AND IMPLICATIONS

The findings from this critical analysis provide a picture of interpretive-based co-op research. It seems interpretive research is a significant part of the co-op research landscape, with numerous such studies published in the literature. These are highly diverse in nature and scope, spanning a wide range of disciplines of study in a variety of geographical locations. They use a variety of research methods, but most employ interviews as the main data collection method. It seems the methods used are appropriate for the topics investigated, which tend to be complex issues such as assessment or student perceptions of highly diverse co-op experiences. Such methods are probably essential given the diversity of co-op programs, the complexity of the learning contexts or situations (i.e., the dual context, meaning that both workplace and on-campus learning occurs), and type of issues investigated (e.g., leadership, equity and migrants' integration into the workforce).

Quality in these studies varies in much the same way it does for quantitative co-op work (see Bartkus, 2007 and comments therein). Bartkus notes that reviewing of research by journals is a demanding task and the fact that quality varies in both quantitative and interpretive work does, to some extent at least, throw the reviewing process into question. In their defense, journal editors often struggle to find reviewers, and struggle even more to match a reviewer's expertise to a given study. The diversity of topics and studies indentified in this work shows just how problematic this issue is in the case of interpretive work. As an illustration, there are probably not that many reviewers capable of providing critique about the appropriateness (or otherwise) of sociocultural learning theories as a theoretical basis for a given study. However, the fact that a large proportion of the studies failed to clearly identify any theoretical basis gives us a way forward. One recommendation then is for journal editors to require authors to present papers based on a template that might consist of: rationale for the study; paradigm and theoretical basis for the work; methodological approach; methods of inquiry; research findings; and, implications of the research. We are not advocating here that interpretive researchers seek to 'test' theory in the way that quantitative researchers do; indeed in many cases that theory development is the purpose of interpretive research (see, e.g., Fletcher, 1990). Rather we are saying, authors should specify the paradigm within which they are working, talk about epistemology or what other theoretical assumptions they are working from, and seek coherency across such dimensions. Let us illustrate with an example from some of our own work. We wished to investigate students' self-efficacy towards practical science (Coll et al., 2001). We felt this was an issue that needed intensive research tools such as interviews and that we wanted to develop an in depth understanding of self-efficacy in a specific educational context. Hence, we worked from an interpretive paradigm, and used sociocultural theories of learning to inform our work (because we felt students were in effect seeking to become part of a community of practice). We also developed a definition or understanding of self-efficacy towards practical science from theory of self-efficacy (i.e., in this case Bandura's work see Bandura, 1986), and specifically as it applied to co-op, by drawing on Fletcher's work (see Fletcher, 1990). Interestingly, instructions for authors for most journals almost always specify that authors should specify theoretical underpinnings and assumptions and so on. However, it seems this structure is in fact seldom adhered to; at least based on our analysis. Our proposal here then, is that editors should pre-screen papers and not send them out for review unless they adhere to journal specifications, rather than leaving it to chance that their reviewers will pick up on lack of adherence to manuscript guidelines.

To address the quality issue also we need to require authors to make more explicit some aspects of their research design. The critical analysis reported in the present work identified dependability and confirmability as areas for improvement. As noted earlier, these are the two main issues subject to criticism by critics of interpretive research. Our view is that what may be missing is some detail that convinces readers the study is trustworthy. Hence, we need more detail about research design, and specifically that which addresses issues of credibility and confirmability. Shulman (1988) argues that any study, no matter how well resourced or how sophisticated the design, has some limitations. He argues that what matters is to clearly identify the limitations, make it clear what impact any limitations may have on the study and most of all, not over-interpret the findings of the study. In other words, don't make claims that are not supported directly by your research findings! We concur, and our recommendation is for editors and reviewers to again drive this process.

It would appear interpretive-based co-op studies are comparable in quality to quantitative work (albeit evaluated using different tools), and as a consequence we suggest they have much to offer the co-op research community. However, although the themes investigated are similar for both types of enquiry, the nature of the research is quite different. As we might imagine, the quantitative studies have been to do with issues of prevalence or involved comparisons (typically of coops and non-co-ops). In contrast interpretive co-op research has been mostly concerned with student perceptions of their placement experiences; their 'lived experience' which is presented in more detail than possible in qualitative work. In the present work we have sought to address our research question, which is largely concerned with the nature and quality of interpretive co-op research. We have commented on the quality of research, the types of issues addressed via such an approach and the diversity of research themes. This analysis indicates that similar issues are addressed as per quantitative research. In hindsight this is not that surprising; co-op researchers are likely interested in similar issues whatever their paradigm, or methodological stances. As such we have not sought to delve greatly into the detail of what such studies have to say about specific co-op issues and compare them directly with like quantitative work. We did identify which issues were studied and attempted to give an overview of the findings. Our analysis, because of space limitations perhaps did not do justice to the detail and depth of insight provided by these interpretive studies. We illustrate the differences between quantitative and interpretive work with two examples. We recently surveyed science and technology employers, co-ops, recent co-op graduates and faculty about their views as to the most desirable graduate competencies in new graduates (Coll & Zegwaard, 2006). From our work we developed an understanding of the views for each cohort, and were able to compare differences statistically. We were not, however, able to understand why these groups held different views. Contrast this with another interpretive study about co-op experiences of full fee paying international students (Coll & Paku, 2007). The richness of the participants' experiences in this latter study were accessed by the nature and depth of this interpretive work; we were able to ascertain reasons for travelling overseas, the experiential influences in both learning contexts (i.e., on- and off-campus), to understand issues about learning in the workplace and their perceptions of a scientist or engineer.

Our analysis of the interpretive literature suggest much of it is of good quality, and that judicious guidance by editors and reviewers may serve to further improve the quality of interpretive research. In this work we do not seek to influence researchers' choice of inquiry or paradigm. We suggest that each approach has merit and as Patton (1990) observes, the key issue is to match research approach to the research issue or question, and to conduct the research guided by quality indicators reported in the literature.