choosing the right tool for the job; how maastricht university is selecting its new plagiarism...

Plagiarism Detection: new tool for

Maastricht University?

Maarten van Wesel

Antoinette Vesseur

Jeroen ten Haaf Full paper in the procedings on http://www.s-ict.org

Faculty name

Current situation

• Safe Assign (since 2003)• 25.000 papers (2009)• (unannounced) downtime

Faculty name

Pilot

• Performance• Usability• Manageability

Faculty name

Two Questions

• Is there a suitable product to use next to Safe Assign?

• Is there a suitable product to replace Safe Assign with?

Faculty name

Why results differ

• Difference in Algorithm• Difference in Sources

Faculty name

Assumptions

• No false positives• Higher percentage = Better

performance

Faculty name

Three candidates

Compatible with Blackboard

• Ephorus• Turn-it-In• Urkund

Faculty name

Numbers

• 3,341 papers• Problems

– 1,779 papers accepted by all

Faculty name

Two Questions

1. Is there a suitable product to use next to Safe Assign

– Difference between SA score and Candidate Score

2. Which product is most suitable to replace Safe Assign with

– Papers per Threat level ( <15%, 15-40%, 40%>)

Faculty name

Answering Question 1

UrkundvsSafe Assign

Faculty name

Answering Question 1

EphorusvsSafe Assign

Faculty name

Answering Question 1

TurnitinvsSafe Assign

Faculty name

Answering Question 1

Mean SD

Urkund – SA 6.6 25.3

Ephorus – SA 3.5 21.8

Turn-it-in – SA 24.7 21.7

Faculty name

Answering Question 1

• Paired sample T-test– Turn-it-In – SA vs Ephorus – SA;

t(1796) = 50.2, p = .000

– Turn-it-In – SA vs Urkund – SA;t(1796) = 39.3, p = .000

Faculty name

Answering Question 2

Faculty name

Answering Question 2

Faculty name

Answering Question 2

Faculty name

Answering Question 2

• Sign Test– Turn-it-In – Urkund;

z = -34.9, p = .000

– Urkund – Ephorus;z = -5.8, p = .000

– Turn-it-In – Ephorusz = -32.0, p = .000

Faculty name

Conclusion

• Is there a suitable product to use next to Safe Assign– Yes there is, Turn-it-In

• Which product is most suitable to replace Safe Assign with– Turn-it-In

Faculty name

Questions?