The Challenges of Learning MachineLearning for Non-Technical Users orNovice Programmers

Robert CincaUCL Interaction CentreUniversity College LondonLondon, UKrobert.cinca.14@ucl.ac.uk

Mirco MusolesiUniversity College LondonThe Alan Turing InstituteLondon, UKm.musolesi@ucl.ac.uk

Stefan KreitmayerUCL Interaction CentreUniversity College LondonLondon, UKs.kreitmayer@ucl.ac.uk

Yvonne RogersUCL Interaction CentreUniversity College LondonLondon, UKy.rogers@ucl.ac.uk

Enrico CostanzaUCL Interaction CentreUniversity College LondonLondon, UKe.costanza@ucl.ac.uk

Permission to make digital or hard copies of part or all of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for third-party components of this work must be honored.For all other uses, contact the owner/author(s).

Copyright held by the owner/author(s).CHI, ’19 Glasgow, UKACM 0-12345-67-8/90/01.http://dx.doi.org/10.1145/2858036.2858119

AbstractRecent breakthroughs in machine learning and AI have ledto increasing interest in data science among the generalpublic. This trend is evident in the abundance of new onlinecourses, interactive websites like Kaggle and face-to-facecode camps marketed to people with little or no program-ming experience. However, there appears to be a mismatchbetween how machine learning is commonly taught and thespecific needs of this emerging user group. Our paper aimsto explore this mismatch and highlight some of the chal-lenges for explainable AI and interaction design. Our anal-ysis draws on evidence from a study with 144 beginnersat machine learning. Relevant bottlenecks were identifiedin four main areas, namely: machine learning, maths, pro-gramming and operating systems. Our preliminary findingssuggest that educational systems should be designed toselectively "blackbox" and "whitebox" these areas in waysthat are customisable and transparent to the learner.

Author KeywordsLearning machine learning; explainable AI; data science;blackbox; whitebox

ACM Classification KeywordsH.5.m [Information interfaces and presentation (e.g., HCI)]:Miscellaneous

IntroductionLarge-scale data analysis promises to change many ar-eas, ranging from driverless cars in the automobile industryto smart thermostats within the home. Much of this dataprocessing involves training machine learning models ofever-increasing complexity. Through those breakthroughs,we have observed an increase in the number of novice pro-grammers that are taking up machine learning (ML), withthe goal of applying it to their own fields. This emerginggroup often uses online tutorials and practical examples(among other resources) to teach themselves machinelearning, with mixed levels of success.

In response, graphical user interfaces (GUIs) have emergedto help these non-technical audiences with their ML anal-ysis. Weka, one of the most popular GUI-based ML tools,provides a platform to quickly train and visualise modelswithout writing code [8]. Similarly, a wide range of librariesand APIs exist to help programmers use ML while hidingthe underlying maths. Conversely, systems like Matlab andOctave can support a mathematical approach to learningML, while shielding the novice from the details of computa-tional implementation. Each of these approaches promotesclarity in some aspects while making others opaque for thesake of users making quick progress.

Black and White BoxesThe distinction between blackbox and whitebox systems iscommon in computing as well as education. In the contextof human learning with computer models and simulations,Jonassen and Strobel [4] add a third category which theycall the glass-box. Table 1 provides an overview of thesedifferent types of system.

Figure 1: A simple user interfaceallowed participants to comment oneach machine learning tutorial.

Table 1: Types of systems according to Jonassen and Strobel [4]

System Type Implication

Blackbox The learner cannot see the modelWhitebox The learner can see the modelGlassbox The learner can change the model

AimsThis paper aims to contribute to the workshop in the follow-ing ways:

1. to describe the range of pedagogical needs and ex-pectations among ML beginners, using empirical evi-dence from a recent user study

2. to contrast these needs with the current tool land-scape and educational offerings for ML

3. to inform the design of explainable-ML systems forbeginners

MethodThis research draws evidence from a study where pairs ofself-motivated adult learners followed a 10-part video tu-torial series [3] on ML. Seventy-two pairs were recruitedthrough meetup.com and word of mouth. Each pair wasgiven access to the material through the custom web inter-face shown in Figure 1, allowing them to leave commentsduring and after each tutorial. Our qualitative analysis fo-cuses on these comments as well as preliminary and finalinterviews with individual participants. We first performedopen-ended coding on the data. Themes were then iden-tified regarding individuals’ motivations to study ML, theirexpectations of the learning experience and the challengesthey encountered on their journey.

FindingsWe identified several themes from the study. These areas follows: Backgrounds and motivations vary; Beginnersenjoy real world problems; Beginners want to understandhow models behave; Visualisations can engage and sup-port understanding; Interests vary regarding coding details;Operation system details can frustrate.

Backgrounds and motivations varyPeople approach machine learning from a diverse range ofbackgrounds and with different motivations. For example,one participant said that she worked in the field of naturallanguage processing and wanted to communicate moreeffectively with the engineering team. Another, a physicist,said her aim was "to broaden [her] data analysis skills".

Beginners enjoy real-world problemsBeginners expressed positive reactions when the tutorialinvolved analysing real data sets: "We found this one tobe probably the most useful because [the instructor] talksabout real data sets".

Beginners want to understand how models behaveSeveral comments revolved around understanding howmodels behave. For example, one participant struggled withthe random splitting of data: "The two classifiers give differ-ent results when you run them every time, one outperformsthe other at times". Another participant was unsure how toimprove the model: "testing the classifier gives about 90%;how do we improve that?". This is in line with previous find-ings that users are interested in model behaviour [6].

Visualisations can engage and support understandingMany participants mentioned that they liked the visualisa-tions and that the graphs helped them better understandthe underlying model. These findings corroborate prior re-search in the field of Explainable AI [1, 5, 6, 7].

Interests vary regarding coding detailsWith regard to the coding implementation, levels of inter-est varied widely among the diverse, largely non-technicalgroup of beginners. Some participants expressed a desirefor full visibility: "I’m a bit annoyed by the fact that they useScikit Learn as a black box and never give definitions prop-erly". Others questioned the need for coding details: "notsure when we would practically write a classifier".

Operation system details can frustrateMany participants expressed frustration about issues suchas "getting Anaconda installed on MacOS (had issues withit altering the PATH)" or "I wasn’t able to install Docker inmy machine". Such issues were generally perceived asdistracting from learning about ML.

DiscussionWhile our findings are preliminary, they suggest a new per-spective on the current landscape of tools for learning andusing ML. Our discussion centres around: Current learningML methods and whiteboxing attempts; Blackboxing the de-velopment environment; and How to choose the right toolsand pathways when ’re-painting the boxes’.

Several ways of teaching ML to non-technical experts haveemerged. One such way, through the use of GUIs (such asWeka), actually blackboxes the ML by hiding away all theimplementation details. Not all user want this. As one par-ticipant said: "weka being GUI driven, would have beengreat to be able to do this all via CLI or via python pro-grams". Although some users prefer that when coding fortheir own projects, at least for the initial learning process,the study showed a preference for a more detailed break-down of the models.

Another way of teaching ML would be through an interac-tive visual of the model. TensorFlow’s Playground [2] aims

to do just that, whilst bringing a transparency to AI mod-els. Although the program aims to visualise the insides ofa neural network, it is just a demo, and therefore not a tool.In addition, it can still be considered a blackbox as it doesnot explain how the weights are decided. However, com-pared to Weka, it is a step in the right direction because itvisualises connections between the weights.

Tools like Google Colab were appreciated by novices, asthey prevented users from getting stuck, by blackboxing theoperating system and development environment.

ConclusionAs machine learning techniques are rapidly adopted by awide range of professional fields, new challenges emergefor educating a diverse audience of non-technical users andnovice programmers. Our analysis of user comments andinterviews revealed a better understanding of the needsand expectations among this important user group. Whilethere are some apparent commonalities in users’ needs(beginners want transparency in the ML concepts, whilstnot having to worry about the development infrastructure),interest in computational and mathematical detail variedgreatly across participants. We argue that by selectivelyblack-boxing and white-boxing these aspects, educationalsystems can embrace the audience’s diversity of back-grounds and goals when it comes to learning machinelearning.

AcknowledgementsThis work was supported by the UK Engineering and Physi-cal Sciences Research Council (EPSRC) grant EP/R513143/1for the University College London Interaction Centre (UCLIC).Our study was approved by the UCLIC Ethics Committee(UCLIC/1718/003).

REFERENCES1. Ashraf Abdul, Jo Vermeulen, Danding Wang, Brian Y

Lim, and Mohan Kankanhalli. 2018. Trends andtrajectories for explainable, accountable and intelligiblesystems: An hci research agenda. In Proceedings ofthe 2018 CHI Conference on Human Factors inComputing Systems. ACM, 582.

2. Google Developers. 2015. TensorFlow: Large-ScaleMachine Learning on Heterogeneous Systems. (2015).http://tensorflow.org/ Software available fromtensorflow.org.

3. Google Developers. 2017. Machine Learning Recipes.https://www.youtube.com/playlist?list=PLOU2XLYxmsIIuiBfYad6rFYQU_jL2ryal. (2017).[Online; accessed 19-July-2018].

4. D Jonassen and J Strobel. 2006. Modeling formeaningful learning. Engaged learning with emergingtechnologies (2006), 1–27.

5. Vassilis Kostakos and Mirco Musolesi. 2017. Avoidingpitfalls when using machine learning in HCI studies.interactions 24, 4 (2017), 34–37.

6. Brian Y Lim and Anind K Dey. 2009. Assessingdemand for intelligibility in context-aware applications.In Proceedings of the 11th international conference onUbiquitous computing. ACM, 195–204.

7. Brian Y Lim and Anind K Dey. 2010. Toolkit to supportintelligibility in context-aware applications. InProceedings of the 12th ACM international conferenceon Ubiquitous computing. ACM, 13–22.

8. Ian H Witten, Eibe Frank, Mark A Hall, andChristopher J Pal. 2016. Data Mining: Practicalmachine learning tools and techniques. MorganKaufmann.

Author BackgroundI am a 1st year PhD student at the UCL Interaction Centre(part of University College London), with a background inComputer Science. My research interest is in explainable

AI. My motivation for participating in the Human-CenteredStudy of Data Science Work Practices Workshop stemsfrom my interest in building tools or methods to supporthuman activities in data science work.