making public spaces safer dr b. nasa, prof j. binner dr m. a. ferrario, dr w. simm, prof j....
TRANSCRIPT
![Page 1: Making Public Spaces Safer Dr B. Nasa, Prof J. Binner Dr M. A. Ferrario, Dr W. Simm, Prof J. Whittle, Dr B. Lam; Sheffield Management School Lancaster](https://reader036.vdocuments.us/reader036/viewer/2022070307/551afe01550346f70d8b5462/html5/thumbnails/1.jpg)
Making Public Spaces Safer
Dr B. Nasa, Prof J. BinnerDr M. A. Ferrario, Dr W. Simm, Prof J. Whittle, Dr B. Lam;
Sheffield Management SchoolLancaster University/ Brunel University
Towards Policing Perception Maps automatically extracting quantitative analysis of
qualitative survey data
![Page 2: Making Public Spaces Safer Dr B. Nasa, Prof J. Binner Dr M. A. Ferrario, Dr W. Simm, Prof J. Whittle, Dr B. Lam; Sheffield Management School Lancaster](https://reader036.vdocuments.us/reader036/viewer/2022070307/551afe01550346f70d8b5462/html5/thumbnails/2.jpg)
![Page 3: Making Public Spaces Safer Dr B. Nasa, Prof J. Binner Dr M. A. Ferrario, Dr W. Simm, Prof J. Whittle, Dr B. Lam; Sheffield Management School Lancaster](https://reader036.vdocuments.us/reader036/viewer/2022070307/551afe01550346f70d8b5462/html5/thumbnails/3.jpg)
Background: Derry/NI
HOW TO build trust in the police after 30+ years of Conflict (The Troubles)?
SELF POLICING & FREE DERRY: 1960s on going clashes between Protestants and Catholics. By the end of 1971, barricades prevented access to Free Derry (Bog Side Area) . No police forces allowed.
BLOODY SUNDAY: 30th January 1972, 27 civilians were shot by the British Army Parachute Regiment during a Civil Rights march. 13 died on the scene.
![Page 4: Making Public Spaces Safer Dr B. Nasa, Prof J. Binner Dr M. A. Ferrario, Dr W. Simm, Prof J. Whittle, Dr B. Lam; Sheffield Management School Lancaster](https://reader036.vdocuments.us/reader036/viewer/2022070307/551afe01550346f70d8b5462/html5/thumbnails/4.jpg)
Derry District Policing Partnership
Derry DPP is one of the 26 DPPs in NI
• Responsibilities:• Monitoring local police performance• Giving a voice to community views on policing• Gaining the public's cooperation to prevent crime.
DDPP Household survey 2009 Confidence and Satisfaction
![Page 5: Making Public Spaces Safer Dr B. Nasa, Prof J. Binner Dr M. A. Ferrario, Dr W. Simm, Prof J. Whittle, Dr B. Lam; Sheffield Management School Lancaster](https://reader036.vdocuments.us/reader036/viewer/2022070307/551afe01550346f70d8b5462/html5/thumbnails/5.jpg)
DDPP Household Satisfaction & Confidence Survey’09
• All-household survey to inform next year local policing plan (2010/11)
• First-ever DPP blanket consultation by Neighbourhood
• 46,000 households: 465 responses received to the survey (1% response rate)
![Page 6: Making Public Spaces Safer Dr B. Nasa, Prof J. Binner Dr M. A. Ferrario, Dr W. Simm, Prof J. Whittle, Dr B. Lam; Sheffield Management School Lancaster](https://reader036.vdocuments.us/reader036/viewer/2022070307/551afe01550346f70d8b5462/html5/thumbnails/6.jpg)
![Page 7: Making Public Spaces Safer Dr B. Nasa, Prof J. Binner Dr M. A. Ferrario, Dr W. Simm, Prof J. Whittle, Dr B. Lam; Sheffield Management School Lancaster](https://reader036.vdocuments.us/reader036/viewer/2022070307/551afe01550346f70d8b5462/html5/thumbnails/7.jpg)
Verbatim vs Tick-Box
![Page 8: Making Public Spaces Safer Dr B. Nasa, Prof J. Binner Dr M. A. Ferrario, Dr W. Simm, Prof J. Whittle, Dr B. Lam; Sheffield Management School Lancaster](https://reader036.vdocuments.us/reader036/viewer/2022070307/551afe01550346f70d8b5462/html5/thumbnails/8.jpg)
• The survey contained questions with multiple choice answers and the opportunity for the respondent to expand upon their selection in free text form.
• Traditionally, this free text is difficult and time consuming to analyse by theme and sentiment.
• Automatically extracting ‘quantitative summaries’ could help identify themes which are not part of the multiple choice options and would allow comparison across results from different districts.
Making Sense of Qualitative Data
![Page 9: Making Public Spaces Safer Dr B. Nasa, Prof J. Binner Dr M. A. Ferrario, Dr W. Simm, Prof J. Whittle, Dr B. Lam; Sheffield Management School Lancaster](https://reader036.vdocuments.us/reader036/viewer/2022070307/551afe01550346f70d8b5462/html5/thumbnails/9.jpg)
Factual vs Perception Mapping
Map of actual crime statistics
Perceptions maps
![Page 10: Making Public Spaces Safer Dr B. Nasa, Prof J. Binner Dr M. A. Ferrario, Dr W. Simm, Prof J. Whittle, Dr B. Lam; Sheffield Management School Lancaster](https://reader036.vdocuments.us/reader036/viewer/2022070307/551afe01550346f70d8b5462/html5/thumbnails/10.jpg)
Challenge: How do people feel...
• 36% of respondents** answered ‘don’t know’ to the satisfaction question => How can the police ‘win’ the undecided?
• Understanding ‘Why’people are undecidedmay help…
SATISFACTION (Derry City Council Area)
Yes35%
No27%
Don't Know36%
No Answer2%
Yes
No
Don't Know
No Answer
** diagram and figures from DDPP survey
![Page 11: Making Public Spaces Safer Dr B. Nasa, Prof J. Binner Dr M. A. Ferrario, Dr W. Simm, Prof J. Whittle, Dr B. Lam; Sheffield Management School Lancaster](https://reader036.vdocuments.us/reader036/viewer/2022070307/551afe01550346f70d8b5462/html5/thumbnails/11.jpg)
Data Set: Original Source
1
2
3
4
56
DDPP IDLOCALECONFIDENCE LEVELCONFIDENCE VERBATIMSATISFACTION LEVELSATISFACTION VERBATIM
123456
![Page 12: Making Public Spaces Safer Dr B. Nasa, Prof J. Binner Dr M. A. Ferrario, Dr W. Simm, Prof J. Whittle, Dr B. Lam; Sheffield Management School Lancaster](https://reader036.vdocuments.us/reader036/viewer/2022070307/551afe01550346f70d8b5462/html5/thumbnails/12.jpg)
Extracted – Full Data Set
First Step: 449 Records in totalSecond Step: Sentence Level Split + Clean Data+ Remove Blank Verbatim = 379 Sentence level comments (Satisfaction)= 485 sentence level comment (Confidence)
![Page 13: Making Public Spaces Safer Dr B. Nasa, Prof J. Binner Dr M. A. Ferrario, Dr W. Simm, Prof J. Whittle, Dr B. Lam; Sheffield Management School Lancaster](https://reader036.vdocuments.us/reader036/viewer/2022070307/551afe01550346f70d8b5462/html5/thumbnails/13.jpg)
Sample Data Set – 95% CL; CI 5%
• 200 comments (Satisfaction)• 221 comments (Confidence)• Manual analysis (three researchers)
• Theme, Actionability, Sentiment (TAS)
• Automated Theme Extraction – Accuracy Testing• VYV• Bayes Classifier
![Page 14: Making Public Spaces Safer Dr B. Nasa, Prof J. Binner Dr M. A. Ferrario, Dr W. Simm, Prof J. Whittle, Dr B. Lam; Sheffield Management School Lancaster](https://reader036.vdocuments.us/reader036/viewer/2022070307/551afe01550346f70d8b5462/html5/thumbnails/14.jpg)
TOP THEMES (Satisfaction & Confidence)Approach:1. Initial Identification of themes (Cross-checked with DDPP)2. Independent Theme Review (by three researchers)3. Up to 4 themes per comments4. Calibration and Compilation of final theme List
Emerging Results:
Satisfaction Frequency % Confidence Frequency %General ImpressionGeneral Impression 6161 17.617.6 Response Time and QualityResponse Time and Quality 7070 18.418.4
Presence and Visibility 48 13.9 Police Behaviour/Political Policing 64 16.8
Police Behaviour/Political Policing 46 13.3 Presence and Visibility 61 16.0
Offence and Crime Level 40 11.6 General Impression 38 10.0
Response Time and Quality 32 9.2 Resources incl. Admin 31 8.1
Other 29 8.4 Offence and Crime Level 23 6.0
Resources incl. Admin 21 6.1 People behaviour and Attitude 21 5.5
People behaviour and Attitude 17 4.9 Other 17 4.5
Community Policing 14 4.0 Community Policing 15 3.9
Specific Events/Location 14 4.0 Specific Events/Location 15 3.9
Complexity of Situation 10 2.9 Complexity of Situation 13 3.4
Authority/Law enforcement 10 2.9 Perceived Safety 7 1.8
Perceived Safety 4 1.2 Authority/Law enforcement 6 1.6
Total – themes 346 100 Total 381 100%
![Page 15: Making Public Spaces Safer Dr B. Nasa, Prof J. Binner Dr M. A. Ferrario, Dr W. Simm, Prof J. Whittle, Dr B. Lam; Sheffield Management School Lancaster](https://reader036.vdocuments.us/reader036/viewer/2022070307/551afe01550346f70d8b5462/html5/thumbnails/15.jpg)
THEMES BY LOCATIONSecond Step: Match Themes to NeighbourhoodsNeighbourhood analysis of Satisfaction with Policing in Derry Themes (aggregated) Foyle City
CentreCity North Foyle City
WestWaterside
RuralWaterside
UrbanTotal by ThemesFreq. %
General Impression 8 17 8 17 11 61 17.6
Presence and Visibility 2 13 9 16 8 48 13.9
Police Behaviour/political policing 4 13 7 13 9 46 13.3
Offences and Crime Level 3 14 8 11 4 40 11.6
Response Time and Quality 1 2 6 11 12 32 9.2
Other 5 3 4 10 7 29 8.4
Resources Incl. Admin 2 3 5 4 7 21 6.1
People Behaviour 3 7 2 2 3 17 4.9
Community Policing - 5 3 3 3 14 4.0
Specific Events/Location - 4 2 5 3 14 4.0
Complexity of Situation 3 2 - 4 1 10 2.9
Authority/Law Enforcement - 5 1 1 3 10 2.9
Perceived Safety - - - 4 - 4 1.2
NeighbourhoodFreq. 31 88 55 101 71 346 100.0
% 9.0 25.4 15.9 29.2 20.5 100.0
![Page 16: Making Public Spaces Safer Dr B. Nasa, Prof J. Binner Dr M. A. Ferrario, Dr W. Simm, Prof J. Whittle, Dr B. Lam; Sheffield Management School Lancaster](https://reader036.vdocuments.us/reader036/viewer/2022070307/551afe01550346f70d8b5462/html5/thumbnails/16.jpg)
3-WAY ANALYSIS: themes v. sentiments v. location
![Page 17: Making Public Spaces Safer Dr B. Nasa, Prof J. Binner Dr M. A. Ferrario, Dr W. Simm, Prof J. Whittle, Dr B. Lam; Sheffield Management School Lancaster](https://reader036.vdocuments.us/reader036/viewer/2022070307/551afe01550346f70d8b5462/html5/thumbnails/17.jpg)
SATISFACTION v. SENTIMENTS
![Page 18: Making Public Spaces Safer Dr B. Nasa, Prof J. Binner Dr M. A. Ferrario, Dr W. Simm, Prof J. Whittle, Dr B. Lam; Sheffield Management School Lancaster](https://reader036.vdocuments.us/reader036/viewer/2022070307/551afe01550346f70d8b5462/html5/thumbnails/18.jpg)
Is stated level of satisfaction reflected in verbatim
Satisfaction Level
Sentiment derived from verbatim
Negative Neutral Positive Total
No 109 14 0 123
32.15% 4.13% 0% 36.28%
Don’t Know 46 40 6 9213.57% 11.8% 1.77% 27.14%
Yes 14 32 78 1244.13% 9.44% 23.01% 36.58%
Total 169 86 84 33949.85% 25.37% 24.78% 100%
Null: SL is independent of VSChi square statistics : χ2 = 211.04 (0.000 )Reject null, so SL is associated with VS
![Page 19: Making Public Spaces Safer Dr B. Nasa, Prof J. Binner Dr M. A. Ferrario, Dr W. Simm, Prof J. Whittle, Dr B. Lam; Sheffield Management School Lancaster](https://reader036.vdocuments.us/reader036/viewer/2022070307/551afe01550346f70d8b5462/html5/thumbnails/19.jpg)
ExampleVERBATIM ANSWERS**
Do you think the PSNI are doing a good job in the Foyle area? Don’t know – PSNI take too long to respond to a reported incident.
Don’t Know - They certainly don’t appear to be able to curb the problems in my area of Culmore.
Don’t Know -Because they let the young joy riders straight out again also they know all the drug dealers in the one area still they turn a blind eye.
![Page 20: Making Public Spaces Safer Dr B. Nasa, Prof J. Binner Dr M. A. Ferrario, Dr W. Simm, Prof J. Whittle, Dr B. Lam; Sheffield Management School Lancaster](https://reader036.vdocuments.us/reader036/viewer/2022070307/551afe01550346f70d8b5462/html5/thumbnails/20.jpg)
Automated Theme Extraction
VYV-Bayes Classifier
![Page 21: Making Public Spaces Safer Dr B. Nasa, Prof J. Binner Dr M. A. Ferrario, Dr W. Simm, Prof J. Whittle, Dr B. Lam; Sheffield Management School Lancaster](https://reader036.vdocuments.us/reader036/viewer/2022070307/551afe01550346f70d8b5462/html5/thumbnails/21.jpg)
Accuracy of Automated VYV Theme Extraction Approach:
1. VYV automatically assigned (two) themes
2. Accuracy assessed on a 3-point Likert scale
3. Independent assessment by 2 reviewers, followed by a ‘calibration exercise’ to address differences
4. 69-72% of the themes were acceptable or accurate for both sets
5. Reviewers agreed on both themes 93-94% of the time
![Page 22: Making Public Spaces Safer Dr B. Nasa, Prof J. Binner Dr M. A. Ferrario, Dr W. Simm, Prof J. Whittle, Dr B. Lam; Sheffield Management School Lancaster](https://reader036.vdocuments.us/reader036/viewer/2022070307/551afe01550346f70d8b5462/html5/thumbnails/22.jpg)
• Noun picked from POS = “Staff” sem tag I3.1
• Secondary sem tag S2
<w pos="NN" lemma="staff" sem="I3.1/S2">staff</w>
“Very helpful and friendly staff”
![Page 23: Making Public Spaces Safer Dr B. Nasa, Prof J. Binner Dr M. A. Ferrario, Dr W. Simm, Prof J. Whittle, Dr B. Lam; Sheffield Management School Lancaster](https://reader036.vdocuments.us/reader036/viewer/2022070307/551afe01550346f70d8b5462/html5/thumbnails/23.jpg)
VYV tags vs Manual (Satisfaction)
VYV identified a total of 90 unique themesManual tagging identified 17 themesSome Interesting Parallels and Challenges
Unique Themes freq % rank (top 17)Places 34 9.7% 1Work and employment: Generally 14 4.0% 2Evaluation: Good 12 3.4% 3Speech acts 11 3.1% 4Belonging to a group 11 3.1% 5Difficult 11 3.1% 6People 10 2.8% 7Time: Period 10 2.8% 8Grammatical bin 9 2.6% 9Time: New and young 9 2.6% 10Location and direction 9 2.6% 11Helping 9 2.6% 12General actions / making 9 2.6% 13Existing 9 2.6% 14Vehicles and transport on land 9 2.6% 15Crime 8 2.3% 16Getting and possession 7 2.0% 17
![Page 24: Making Public Spaces Safer Dr B. Nasa, Prof J. Binner Dr M. A. Ferrario, Dr W. Simm, Prof J. Whittle, Dr B. Lam; Sheffield Management School Lancaster](https://reader036.vdocuments.us/reader036/viewer/2022070307/551afe01550346f70d8b5462/html5/thumbnails/24.jpg)
Theme Assignment Alternative
Classifier methods can be used to classify texts:
1. The classifier is trained using manual data2. A number of exclusive classifications are defined 3. As subset of data is selected and manually classified4. This subset is used to train the classifier on the
features (in this case the words) that make up each classification
5. New comment are classified by the classifier based on features
![Page 25: Making Public Spaces Safer Dr B. Nasa, Prof J. Binner Dr M. A. Ferrario, Dr W. Simm, Prof J. Whittle, Dr B. Lam; Sheffield Management School Lancaster](https://reader036.vdocuments.us/reader036/viewer/2022070307/551afe01550346f70d8b5462/html5/thumbnails/25.jpg)
Bayes Classifier applied to DDPP Data
• Naive Bayes Classifier was applied to “Confidence” data:• Training subset used previous 83 manually tagged
comments• classifier accuracy increases with number of training data• 20 further comments were extracted at random and
classified by the Bayes classifier to test accuracy.
• Preliminary Results:• In a manual review by 2 researchers (method described
previously) the classifier returned an “Acceptable” or “Accurate” classification for 60% of the 20 comments.
![Page 26: Making Public Spaces Safer Dr B. Nasa, Prof J. Binner Dr M. A. Ferrario, Dr W. Simm, Prof J. Whittle, Dr B. Lam; Sheffield Management School Lancaster](https://reader036.vdocuments.us/reader036/viewer/2022070307/551afe01550346f70d8b5462/html5/thumbnails/26.jpg)
Towards Perception Maps
![Page 27: Making Public Spaces Safer Dr B. Nasa, Prof J. Binner Dr M. A. Ferrario, Dr W. Simm, Prof J. Whittle, Dr B. Lam; Sheffield Management School Lancaster](https://reader036.vdocuments.us/reader036/viewer/2022070307/551afe01550346f70d8b5462/html5/thumbnails/27.jpg)
Interactive Map Display
• Plot the information derived from automatic analysis of survey verbatim on an interactive map
• Allows interaction with, and evaluation of the data by stakeholders and possibly public
Safer Streets
![Page 28: Making Public Spaces Safer Dr B. Nasa, Prof J. Binner Dr M. A. Ferrario, Dr W. Simm, Prof J. Whittle, Dr B. Lam; Sheffield Management School Lancaster](https://reader036.vdocuments.us/reader036/viewer/2022070307/551afe01550346f70d8b5462/html5/thumbnails/28.jpg)
Implementation
• Design Storyboard by Information Architecture student Zoe Zhao (Brunel University)
• Development Progress using web technologies• Google maps• Web 2.0 technology including Ajax and JQuery
![Page 29: Making Public Spaces Safer Dr B. Nasa, Prof J. Binner Dr M. A. Ferrario, Dr W. Simm, Prof J. Whittle, Dr B. Lam; Sheffield Management School Lancaster](https://reader036.vdocuments.us/reader036/viewer/2022070307/551afe01550346f70d8b5462/html5/thumbnails/29.jpg)
1. Frontpage, which contains:
i. Map of the whole area
ii. Overall impression – e.g. Themes, Number of People Reporting, etc
StoryBoard A
![Page 30: Making Public Spaces Safer Dr B. Nasa, Prof J. Binner Dr M. A. Ferrario, Dr W. Simm, Prof J. Whittle, Dr B. Lam; Sheffield Management School Lancaster](https://reader036.vdocuments.us/reader036/viewer/2022070307/551afe01550346f70d8b5462/html5/thumbnails/30.jpg)
StoryBoard B
2. Click on the map to select the area that you want more information
3. The selected area will be highlighted
4. The information of the specific area replaces the overall impression
![Page 31: Making Public Spaces Safer Dr B. Nasa, Prof J. Binner Dr M. A. Ferrario, Dr W. Simm, Prof J. Whittle, Dr B. Lam; Sheffield Management School Lancaster](https://reader036.vdocuments.us/reader036/viewer/2022070307/551afe01550346f70d8b5462/html5/thumbnails/31.jpg)
StoryBoard C
5. Select the information that you want to explore further
6. Place the cursor on the theme for actual figures
7. You also can rate your satisfaction level in this category
8. The sentiment charts will be updated automatically
![Page 32: Making Public Spaces Safer Dr B. Nasa, Prof J. Binner Dr M. A. Ferrario, Dr W. Simm, Prof J. Whittle, Dr B. Lam; Sheffield Management School Lancaster](https://reader036.vdocuments.us/reader036/viewer/2022070307/551afe01550346f70d8b5462/html5/thumbnails/32.jpg)
StoryBoard D
9. Click on the themes for actual comments
10. You will see both positive and negative comments under this theme
11. You can enter your comments or respond to other people’s comments
12. You will see your comment appears on the list straightaway
![Page 33: Making Public Spaces Safer Dr B. Nasa, Prof J. Binner Dr M. A. Ferrario, Dr W. Simm, Prof J. Whittle, Dr B. Lam; Sheffield Management School Lancaster](https://reader036.vdocuments.us/reader036/viewer/2022070307/551afe01550346f70d8b5462/html5/thumbnails/33.jpg)
DDPP Survey Data 2009
Challenges:
• Database design• Google map plots• Ajax for realtime updates• Jquery interface animation
![Page 34: Making Public Spaces Safer Dr B. Nasa, Prof J. Binner Dr M. A. Ferrario, Dr W. Simm, Prof J. Whittle, Dr B. Lam; Sheffield Management School Lancaster](https://reader036.vdocuments.us/reader036/viewer/2022070307/551afe01550346f70d8b5462/html5/thumbnails/34.jpg)
Conclusion and Next Step
• Empirical Analysis• Test specific hypothesis such as
• whether there is more crime/offences where police are sectarian
• Is response related with police behaviour
• Automatic TAS• Train bayes classifier using larger dataset with fewer
categories
• Interactive Map• Put the plans into action
![Page 35: Making Public Spaces Safer Dr B. Nasa, Prof J. Binner Dr M. A. Ferrario, Dr W. Simm, Prof J. Whittle, Dr B. Lam; Sheffield Management School Lancaster](https://reader036.vdocuments.us/reader036/viewer/2022070307/551afe01550346f70d8b5462/html5/thumbnails/35.jpg)
Partners & Collaborators
www.voiceyourview.com