Evaluating Automated Sentiment Analysis of LibQUAL+

Peter Corrigan

<Sentiment Analysis - Goals>

• Maximize the value from LibQUAL+ Comments e.g. track the efficacy of initiatives to address issues at granularity

• Develop a simple to execute, repeatable process, with visualizations of results

• Get this project done quickly.

<Caveats>

• Not a clinical trial of Sentiment Analysis

• There’s no substitute for someone reading the comments

• Purists may question Methodology

• Proceed with caution

<Sentiment Analysis = Opinion Mining>

<The API>

• AlchemyAPI

• Hybrid: Linguistics/(Grammar) + Statistics/(idiom, slang, misspelling)

• Large-scale deep-learning technology trained on massive data from a variety of domains.

<LibQUAL+ * 5>

• Year No. of Comments

• 2010 1383

• 2012 705

• 2013 927

• 2014 1133

• 2015 791

<LibQUAL+ Comment>

textbox11: 714 of 1323id: 6015515SubmitDate: 4/13/2015 8:52:39 AMUserGroup: Postgraduate: Doctoral Research degreeDiscipline: Arts, Humanities and Celtic Studies Age: 31 - 45Sex: FemaleIsComplete: TruetextResponse: I've always found library staff really pleasant and helpful. Remote access to library resources is excellent, though I could benefit from some guidance on how to search more effectively.

<Document Level Results>

I've always found library staff really pleasant and helpful. Remote access to library resources is excellent, though I could benefit from some guidance on how to search more effectively.<?xml version="1.0" encoding="UTF-8"?><results> <status>OK</status><totalTransactions>1</totalTransactions> <language>english</language> <docSentiment> <score>0.901543</score> <type>positive</type> </docSentiment></results>

<Keyword Level Results>

I've always found library staff really pleasant and helpful. Remote access to library resources is excellent, though I could benefit from some guidance on how to search more effectively.

<Keyword>

I've always found library staff really pleasant and helpful. Remote access to library resources is excellent, though I could benefit from some guidance on how to search more effectively.

<Comment Table>

<Comment Table>

<Keyword Table>

Best comments ever!

The most important thing that makes the library a brilliant place to study is the kind and courteous staff who go out of their way to help you.0.989229

Best comments ever!

Overall I find the library services to be excellent and I would like to say that the staff are superb, very friendly and always willing to above and beyond their duties to help students. 0.988592

Best comments ever!

The Hardiman library is an invaluable resource for my research - an excellent range of services, modern, user-friendly facilities and very friendly and helpful staff.0.995445

Correlated strongly with Adequacy Gap

r = 0.9827The p-value is 0.002724. The result is significant at p < 0.05

WiFi

Noise

Staff

Plugs and Sockets

<Keyword Term Frequency>

Hapex terms - significant

Kolmogorov-Smirnov test

Project Costs

• €0.00

• API free for academic use < 30,000 calls per day

Source Code:

• https://github.com/pcorrigan