Posted by: Ellie Kutz | October 19, 2010

Realistic Expectations: What Faculty Users Need to Know about Plagiarism Software

Realistic Expectations: What Faculty Users Need to Know about Anti-Plagiarism Software

This is going to be great. It should save me hours because it will trawl through thousands of pages of material on the databases and find spots where text has been lifted. I hope it will be a quick and relatively thorough means of finding out where students plagiarise and highlight those spots for me. It takes me so long at the moment and even though I can spend a couple of hours checking suspect papers, I’m never really convinced that I’ve got everyone. This could be the answer (Australian faculty member quoted in Sutherland-Smith and Carr).

Wherever plagiarism software has been introduced, faculty expectations have been very high that it will allow them, quickly and easily, to monitor accurately all of their students’ submissions.  Yet studies of such software show particular strengths and limitations that faculty need to know about in order to use it effectively.

1.The most commonly used software (Turnitin and SafeAssign) does not identify all cases of plagiarism.

Faculty and librarians have been running their own informal experiments with these sites, constructing heavily plagiarized essays and submitting them for review.  Results seem to vary, depending on the source materials used, but typically, some instances of copying are missed.  Catherine Pellegrino, a librarian at St. Mary’s College, reports on one such experiment in her academic blog (http://www.spurioustuples.net/?p=384), in which  she found that SafeAssign identified 10 out of 15 copied passages (using Google allowed her to find 8 of them).  SafeAssign did better at identifying paraphrased passages, while Google did better with content from JStor and Project Muse. And the SafeAssign identified web sources, even when the content had originally been drawn from licensed databases).  Similar experiments suggest that SafeAssign is particularly helpful for identifying the web content that students are most likely to draw on, but it isn’t foolproof.

2. Much of what is identified as a match to materials on the web or in institutional databases of student work has other explanations besides intentional plagiarism.

One of the most carefully researched studies was carried out at the University of Texas, (Gillis et al). The researchers submitted 356 freshman essays (typical 10 page research papers with 8-10 sources) to both Turnitin and SafeAssign , following up with a careful analysis of a random sample of 40 essays.  Both tools showed that the full sample of 356 essays fell into the “green zone”—suggesting “low risk” of plagiarism (with less than 24% of the text showing a match with other sources). Both flagged material from web sources and student sources (the institutional database of student work) but not from print materials. There were some differences in what the two tools showed (in the percentage of instances flagged, the percentage of instances that were citation errors, and the number of sources flagged). SafeAssign showed all matches it found with other text, whether correctly quoted and cited or not.  SafeAssign flagged fewer matches to the student data-base, and where it did so, papers were sometimes flagged based on students citing the same source—for example, a student discussion of three different topics related to birth control was flagged because all three cited a link to the website of Planned Parenthood.

3. The use of such software in identifying actual plagiarism requires careful instructor interpretation of the results.

For their closer analysis, after eliminating all correctly cited text (which is flagged by both programs), Gillis et al began by asking “ what the marked text rhetorically represents,”  looking carefully at the context in which it appeared.  They found that 70% of the text marked by Turnitin and 83% by SafeAassign fell into four categories: the use of a  topic term (such as global warming), a topic phrase (e.g. “global warming is a serious problem”);  a commonly used phrase (e.g.  “there are many risks associated with global warming,” “researchers have found that ___”), and jargon (words typically attached to discussions of a specific topic such as “global surface temperature, “climate model,” “global dimming”).  The remaining matches were actual citation errors—which could have been either accidental or intentional.

In the end, Gillis et al did not recommend the adoption of either tool at the University of Texas.  They were concerned that students, in their attempts to avoid “plagiarism” by using the common terms and phrases of  a field’s discourse, would be likely to turn to quick fixes like the thesaurus and end up avoiding the “expert, insider prose,” they should have been acquiring, writing, instead, to the software.  Their conclusion?  ” The applications’ approach to writing is inconsistent with WAC [Writing Across the Curriculum] pedagogy. That is, in lieu of good pedagogy, the applications often penalize students for doing exactly what we want them to do: learn the basic language structures used by people who are writing about a common topic in a given discipline.”

4. The software can nevertheless be useful as a learning tool for students, if they are given the opportunity to learn about its possibilities and limitations in an appropriate teaching context.

In such a context, students could, for example,

  • Be given the opportunity to submit their essays as drafts.
  • Be taught  what the individual reports show, including the ways in which the appropriate use of common terms and phrases may be flagged.
  • Be  given the opportunity to change their work for resubmission (where they have actually paraphrased inadequately or missed a citation)
  • Be  asked to provide a narrative account of what in their work should stand as it is.

The faculty interviewed by Sutherland-Smith and Carr in their study at South Coast University in Australia reached similar conclusions. While they were disappointed to find that the software they were using (Turnitin) would not prove to be a foolproof and simple tool for addressing all plagiarism, they came to believe that “it could perform a useful function in heightening awareness of plagiarism as an issue of academic integrity, where subject-specific exercises were developed and students were invited to submit their own work.”

Pellegrino, Catherine. “SafeAssign vs. Google for Plagiarism Detection.” Spurious Tuples: Flapping the Unflappable Since 1996. March 30, 2010 <http://www.spurioustuples.net/?p=384>.

Gillis, Kathleen, Susan Lang, Monica Norris and Laura Palmer.  “Electronic Plagiarism Checkers:  Barriers to Developing an Academic Voice.”  The WAC Journal 20, November 2009 <http://wac.colostate.edu/journal/vol20/gillis.pdf >.

Sutherland-Smith, Wendy and Rodney Carr. “Turnitin.com:  Teachers’ Perspectives of Anti-Plagiarism Software in Raising Issues of Educational Integrity.”  Journal of University Teaching and Learning Practice 2 (3), 2005 <http://ro.uow.edu.au/jutlp/vol2/iss3/10 >

For an excellent discussion of the effective use of SafeAssign in a larger pedagogical context, visit the writing center website at Virginia Commonwealth University at http://www.vcu.edu/uc/writingcenter/safeassign/index.html .

Another helpful site offering best practices for using SafeAssign, information about potential sources of misunderstanding in its use, and a detailed guide to how to read reports is  University of Missouri’s Learning Center at https://courses.missouri.edu/faculty/blackboard/safeassign.php#what.


Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories

%d bloggers like this: