why do models fail? problems, problems …
DESCRIPTION
Why do models fail? Problems, problems …. John Rees BGS Head of Policy and Science Co-ordination Andrew Hughes BGS Groundwater Modeller and NMPI Co-organiser. Ideal vs reality. Generic issues as seen from a government institute - the British Geological Survey. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Why do models fail? Problems, problems …](https://reader036.vdocuments.us/reader036/viewer/2022062423/568148f0550346895db60d90/html5/thumbnails/1.jpg)
Kingsley Dunham CentreKeyworthNottingham NG12 5GGTel 0115 936 3100
© NERC All rights reserved
Why do models fail?Problems, problems …
John ReesBGS Head of Policy and Science Co-ordination
Andrew HughesBGS Groundwater Modeller
and NMPI Co-organiser
![Page 2: Why do models fail? Problems, problems …](https://reader036.vdocuments.us/reader036/viewer/2022062423/568148f0550346895db60d90/html5/thumbnails/2.jpg)
© NERC All rights reserved
Ideal vs reality
![Page 3: Why do models fail? Problems, problems …](https://reader036.vdocuments.us/reader036/viewer/2022062423/568148f0550346895db60d90/html5/thumbnails/3.jpg)
© NERC All rights reserved
Generic issues as seen from a government
institute - the British Geological Survey
![Page 4: Why do models fail? Problems, problems …](https://reader036.vdocuments.us/reader036/viewer/2022062423/568148f0550346895db60d90/html5/thumbnails/4.jpg)
© NERC All rights reserved
BGS and numerical modelling
Groundwater - Model Developer and user Coastal change - Poacher and gamekeeperContamination - often the ‘honest broker’
![Page 5: Why do models fail? Problems, problems …](https://reader036.vdocuments.us/reader036/viewer/2022062423/568148f0550346895db60d90/html5/thumbnails/5.jpg)
© NERC All rights reserved
Typical reasons for failure
• Lack of consistency of approach
• Turnover and education of staff in client organisations
• Language between user and modellers, and between modellers
• Appropriate and sufficient data
• Trust between parties
• Unrealistic expectations
• Honesty about the limitations of models
• Cost of taking-up new developments
![Page 6: Why do models fail? Problems, problems …](https://reader036.vdocuments.us/reader036/viewer/2022062423/568148f0550346895db60d90/html5/thumbnails/6.jpg)
© NERC All rights reserved
\Policy makers
\Resource Managers
\Modellers
![Page 7: Why do models fail? Problems, problems …](https://reader036.vdocuments.us/reader036/viewer/2022062423/568148f0550346895db60d90/html5/thumbnails/7.jpg)
© NERC All rights reserved
Key organisations
• Universities
• Consultants
• Government Institutes
• Regulators
• Utilities
• Government Ministries
• Pan-governmental
![Page 8: Why do models fail? Problems, problems …](https://reader036.vdocuments.us/reader036/viewer/2022062423/568148f0550346895db60d90/html5/thumbnails/8.jpg)
© NERC All rights reserved
Regulator issues
• Limited number of staff that have modelling skills
• Accessibility of models is often poor – they are often not easy to run
• The maintenance and updating of models has a high cost (the leaky roof syndrome)
• Inadequate data to support development or understanding
• Expectations are often not met
• Stakeholders are not adequately consulted
![Page 9: Why do models fail? Problems, problems …](https://reader036.vdocuments.us/reader036/viewer/2022062423/568148f0550346895db60d90/html5/thumbnails/9.jpg)
© NERC All rights reserved
Resource manager issues
• Common uncertainty about the specification and scale of model required
• Lack of clarity about the needs of regulators
• Find that models have too much uncertainty for detailed use
• Inadequate timely stakeholder involvement
• Find that models are not as flexible as managers would like
![Page 10: Why do models fail? Problems, problems …](https://reader036.vdocuments.us/reader036/viewer/2022062423/568148f0550346895db60d90/html5/thumbnails/10.jpg)
© NERC All rights reserved
Model developer issues
• Models are driven by policy and not the other way round e.g. Habitats Directive
• Few examples of models being trusted by all parties
• Stakeholders are recognised as important, but involvement is very variable
• Personalities are important in defining how much stakeholders interact
• Managing expectations is very important
• Those consultants who deliver to spec and on time may not be the best to drive modelling forwards
![Page 11: Why do models fail? Problems, problems …](https://reader036.vdocuments.us/reader036/viewer/2022062423/568148f0550346895db60d90/html5/thumbnails/11.jpg)
© NERC All rights reserved
Broader implications
• Lack of efficiency associated with development encourages adoption or tweaking of older ‘industry standard’ models instead of adoption of newer models.
• Drive to conservatism encourages usage of ‘tried-and-tested’ consultants who focus on delivery, rather than more innovative scientists who will introduce new concepts and stretching the modelling.
• Acceptance of limitations (e.g. empirical constants, black-boxes) instead of driving better modelling.
![Page 12: Why do models fail? Problems, problems …](https://reader036.vdocuments.us/reader036/viewer/2022062423/568148f0550346895db60d90/html5/thumbnails/12.jpg)
© NERC All rights reserved
Towards solutions• Need common understanding before attempting to
formulate solutions
- Climate change models are not taken up by some national governments that do not accept reality!
• Language – differs markedly between disciplines
• Definition of problem type, organisations involved and geographic extent
• Problem is potentially huge, so need boundaries
• Guidelines are needed
![Page 13: Why do models fail? Problems, problems …](https://reader036.vdocuments.us/reader036/viewer/2022062423/568148f0550346895db60d90/html5/thumbnails/13.jpg)
© NERC All rights reserved
Summary
![Page 14: Why do models fail? Problems, problems …](https://reader036.vdocuments.us/reader036/viewer/2022062423/568148f0550346895db60d90/html5/thumbnails/14.jpg)
© NERC All rights reserved
Questions ?