recomment: towards critiquing-based recommendation with speech interaction

S C I E N C E P A S S I O N T E C H N O L O G Y

www.tugraz.at

Critiquing-basedRecommendationwith Speech InteractionPeter Grasch ([email protected])Alexander Felfernig ([email protected])Florian Reinfrank ([email protected]),Institute for Software Technology

October 15, 2013

www.tugraz.at

Speech Interaction in Recommender Systems

(Written) natural language input has shown promisein (Shimazu 2001; arnestal, 2004)

Constraint satisfaction using spoken languagepresented by Thompson et al in 2004

Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 20132

www.tugraz.at

Critiquing-based Recommender Systems

Pioneering work as early as 1984: M. Williams’RABBIT [Williams, 1984]

Seminal work by Burke et al: FindMe[Burke et al., 1997]

Continued, active research, especially in the areas ofadvanced critiques[McCarthy et al., 2004, Zhang and Pu, 2006] anduser modeling[Reilly et al., 2005a, McCarthy et al., 2010]


www.tugraz.at

Traditional Critiquing-based Recommender

Figure : QwikShop [Reilly et al., 2005b]


www.tugraz.at

ReComment: Concept

Preference Model

Recommendation

RecommendationStrategy

Preference Elicitation


www.tugraz.at

ReComment: Rationale

A speech-based natural language interface can allowmore expressive feedback, thus reducing sessionlength.


www.tugraz.at

ReComment: Recommendation Strategy

Incremental unit critiquing-based system[Burke, 2000, Reilly et al., 2005a]

Prior probability based on sales rank

No initial search, relaxed similarity constraint

Custom utility function


www.tugraz.at

ReComment: Recommendation Strategy

P ′ ← {p ∈ P|p satisfies last given critique};maxUtility ← −∞; bestOffer ← rold ;for p ∈ P ′ do

thisUtility ←∞ ;for c ∈ C do

thisUtility ← thisUtility + (1− c.ageMaxAge ) ∗ c.utility(p) ;

endif thisUtility > maxUtility then

maxUtility ← thisUtility ; bestOffer ← p ;end

endreturn bestOfferPeter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 20139

www.tugraz.at

ReComment: Utility Function

Control rate of change: Implicit goalsdistance = distance(a.value, p[a.id ].value) ∗ r .direction;perfectDist = metaModifier ∗ 0.5;if critiqueViolated then

return −abs(distance − perfectDist);else

if distance < perfectDist thenreturn

√distance

perfectDist ;

elsereturn max(perfectDist − distance + 1, 0.0001);

endend

Algorithm 1: Schematic utility calculation.Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 201310

www.tugraz.at


Control rate of change: Implicit goals

Figure : Utility function of the critique x > 50.Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 201311

www.tugraz.at



Figure : Utility function of subsequent critiques.Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 201312

www.tugraz.at



Figure : Utility function of subsequent critiques.Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 201313

www.tugraz.at

ReComment: Speech Processing

Speech recognition solution based on CMU SPHINXand Simon [sph, 2013, sim, 2013]

Adapted to recommender situation

Keyword parser


www.tugraz.at

Experiment

Comparison with traditional interface

80 participants

Measuring:

Interaction cyclesPerceived recommendation qualityUsability (adapted SUS)


www.tugraz.at

Experiment: Traditional User Interface

Figure : ReComment: Mouse-based user interface.Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 201316

www.tugraz.at

Experiment: Speech-based User Interface

Figure : ReComment: Speech-based user interface.Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 201317

www.tugraz.at

Experiment: Speech-based User Interface

SentenceI am looking for a camera with 12 megapixel anda weight of around 200 gram.This camera with the same properties justsmaller.An even smaller camera.Optical zoom of 14 times would be better.More optical zoom.[...]

Table : Sample user interaction session.


www.tugraz.at

Results: Feedback Strategies

Category CountDiscarded 49 (12.8%)Unit critique 329 (85.7%)Compound critique (2 attributes) 3 (0.8%)Compound critique (3 attributes) 2 (0.5%)Compound critique (5 attributes) 1 (0.3%)

Table : Types of used commands.

74 sentences (20 %) referred to explicit values.12 sentences (3 %) used modifiers.Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 201319

www.tugraz.at

Results: Speech Processing

03

10

24

0

5

10

15

20

25

1 2 3 4

"Rec

omm

ent u

nder

stan

dsm

y vo

ice

inpu

t"

Figure : Participants’ perception of the speech-recognitionaccuracy ([1, 4], higher is better).


www.tugraz.at

Results: Usability

Figure : Usability evaluation (adapted SUS scores).Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 201321

www.tugraz.at

Results: Recommendation Quality

0

5

10

15

20

25

Mouse−based interface Speech−based interface

Score

1

2

3

4

Figure : User score of last recommended item ([1, 4],higher is better).


www.tugraz.at

Results: Recommender Efficiency

34.5

47.7

79.0250

10

20

30

40

50

Mouse interface(mean)

Mouse interface(median)

Speech interface(mean)

Speech interface(median)

Ses

sion

leng

th (

cycl

es)

Figure : Session length (lower is better).


www.tugraz.at

Conclusion

Spoken language recommender systemsare worth exploring!


www.tugraz.at

Future Work

Explore more natural user interfaces

Advanced sentiment analysis

Use of prosodic features, timing information, etc. toinfer certainty, frustration, etc.

Compare different recommender systems (e.g.,constraint based approaches)


www.tugraz.at

Thank you for your attention.

Q & A


www.tugraz.at

Sources I

[sim, 2013] (2013).

About Simon — Simon.

http://simon.kde.org.

[sph, 2013] (2013).

CMU Sphinx - Speech Recognition Toolkit.

http://cmusphinx.sf.net.


www.tugraz.at

Sources II

[Burke, 2000] Burke, R. (2000).

Knowledge-based recommender systems.

In Encyclopedia of Library and InformationSystems. Marcel Dekker.

[Burke et al., 1997] Burke, R. D., Hammond, K. J.,and Yound, B. (1997).

The findme approach to assisted browsing.

IEEE Expert, 12(4):32–40.


www.tugraz.at

Sources III

[McCarthy et al., 2004] McCarthy, K., Reilly, J.,McGinty, L., and Smyth, B. (2004).

On the dynamic generation of compound critiquesin conversational recommender systems.

In Adaptive Hypermedia and Adaptive Web-BasedSystems, pages 176–184. Springer.


www.tugraz.at

Sources IV

[McCarthy et al., 2010] McCarthy, K., Salem, Y., andSmyth, B. (2010).

Experience-based critiquing: reusing critiquingexperiences to improve conversationalrecommendation.

In Case-Based Reasoning. Research andDevelopment, pages 480–494. Springer.


www.tugraz.at

Sources V

[Reilly et al., 2005a] Reilly, J., McCarthy, K., McGinty,L., and Smyth, B. (2005a).

Incremental critiquing.

Knowledge-Based Systems, 18(4):143–151.


www.tugraz.at

Sources VI

[Reilly et al., 2005b] Reilly, J., Smyth, B., McGinty, L.,and McCarthy, K. (2005b).

Critiquing with confidence.

In Case-Based Reasoning Research andDevelopment, pages 436–450. Springer.


www.tugraz.at

Sources VII

[Williams, 1984] Williams, M. D. (1984).

What makes rabbit run?

International Journal of Man-Machine Studies,21(4):333–352.


www.tugraz.at

Sources VIII

[Zhang and Pu, 2006] Zhang, J. and Pu, P. (2006).

A comparative study of compound critiquegeneration in conversational recommendersystems.

In Adaptive Hypermedia and Adaptive Web-BasedSystems, pages 234–243. Springer.


recomment: towards critiquing-based recommendation with speech interaction

Technology

software technology

sus15peter grasch

utility p end

p p p satises

mousebased user interface

recommendation strategy

thisutility maxutility

maxage c