crc press - c.ymcdn.comc.ymcdn.com/sites/ press is an imprint of taylor & francis group, an...

CRC PressTaylor & Francis Group6000 Broken Sound Parkway NW, Suite 300Boca Raton, FL 33487-2742

© 2015 by Taylor & Francis Group, LLCCRC Press is an imprint of Taylor & Francis Group, an Informa business

No claim to original U.S. Government works

Printed on acid-free paperVersion Date: 20140730

International Standard Book Number-13: 978-1-4398-2619-5 (Hardback)

This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot assume responsibility for the validity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may rectify in any future reprint.

Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmit-ted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers.

For permission to photocopy or use material electronically from this work, please access www.copyright.com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and registration for a variety of users. For organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged.

Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe.

Library of Congress Cataloging-in-Publication Data

Vosk, Ted, author.Forensic metrology : scientific measurement and inference for lawyers, judges and

criminalists / Ted Vosk, Ashley F. Emery.pages cm. -- (International forensic science and investigation)

Includes bibliographical references and index.ISBN 978-1-4398-2619-5 (hardback : acid-free paper) 1. Forensic sciences. 2. Metrology. 3. Evidence (Law) I. Emery, A. F. (Ashley Francis), 1934- II. Title.

HV8073.5.V67 2015363.25--dc23 2014024204

Visit the Taylor & Francis Web site athttp://www.taylorandfrancis.com

and the CRC Press Web site athttp://www.crcpress.com

This book is dedicated to our wives

For Kris

My love, my light, and my world. . .

To Linda

for her love, patience, and unwavering support

Acknowledgments

I would not have been able to write this text without those who fed my passion forphysics, astronomy, and mathematics almost two decades ago as an undergraduateat Eastern Michigan University, including Professors Norbert Vance, Jim Sheerin,Waden Shen and, in particular, Professors Natthi Sharma and Mary Yorke whoselove and patience changed my life.

Those legal and forensic professionals and organizations who have providedopportunities for me to educate practitioners around the world about forensicmetrology through my writings and lectures, as well as those who have done soalongside me, also have my thanks. These include A.R.W. Forrest, Rod Kennedy,Thomas Bohan, William “Bubba” Head, Edward Fitzgerald, Henry Swofford, LaurenMcLane, Steve Oberman, Pat Barone, Jay Siegel, Doug Cowan, Jon Fox, the Ameri-can Academy of Forensic Sciences, U.S. Drug Enforcement Administration’s South-west Lab, American Physical Society, Supreme Court of the State of Virginia, LawOffice of the Cook County Public Defenders, National College for DUI Defense,National Association of Criminal Defense Lawyers, Washington Association ofCriminal Defense Lawyers and criminal defense, DUI, and bar organizations fromstates around the United States.

Nor would I be writing this text if not for the many lawyers, forensic scientists,and organizations here in Washington and around the country who have contributedto the development of forensic metrology in the courtroom. These include AndyRobertson, Quentin Batjer, Howard Stein, Rod Gullberg, Edward Imwinkelried,Sandra Rodriguez-Cruz, Mike Nichols, Justin McShane, Chris Boscia, Rod Frechette,David Kaye, Jonathon Rands, Eric Gaston, Peter Johnson, Linda Callahan, Scott“Scooter” Robbins, Joe St. Louis, Bob Keefer, Liz Anna Padula, Dr. Jennifer Souders,Dr. Andreas Stolz, Judges David Steiner, Mark Chow and Darrell Phillipson, JasonSklerov, George Bianchi, Sven Radhe, who assisted with my research for Chapter3, Dr. Jerry Messman, Janine Arvizu, the Washington Foundation for Criminal Jus-tice, which funded much of the litigation I’ve undertaken using forensic metrology tobring about reforms, and, in particular, my sidekick Kevin Trombold who is alwayswilling to go tilting after windmills with me.

Without Gil Sapir I never would have had the opportunity to write this book andwithout Max Houck nobody would have taken notice. Nor would it have been a realityhad our editor, Becky Masterman, not continued to believe in us over the almostfour years it took to get started and the subsequent 10 months it took to write. Mycoauthor and friend, Ashley Emery, is responsible for this book reaching completion.He spurred me on when I was ready to quit. Thank you, Ash.

My mom, Susan, and my little brother, Rob, I’m sorry that I was never strongenough to protect you. But you are part of every fight I make and every word I writeto make the world a little better place to live in.

xxxvii

xxxviii Acknowledgments

And, as always, it was my wife, Kris, who made me believe. Your love continuesto lift me and make all things possible.

T. Vosk

I would like to acknowledge my former colleagues Charles Kippenhan (who intro-duced me to E.T. Jaynes’ ideas) and Dean McFeron whose support and interest wasa great encouragement when I first became seriously involved in Bayesian inference.Unfortunately, both died before this book was completed. Also to Galen Shorack(Department of Statistics, University of Washington) and Joseph Garbini (Depart-ment of Mechanical Engineering, University of Washington) with whom I had manyfascinating discussions about statistics, random processes, and experiment design foroptimal data analysis. To my former students, particularly Dawn Bardot and Elisa-betta Valenti who journeyed with me through some very complicated data analysesand validation/verification problems.

Above all I need to especially acknowledge Norman McCormick (professor emer-itus, mechanical engineering) who encouraged me, guided me through my journeyfrom plain Tex to LaTex, and as a published author gave me invaluable counsel andconstant encouragement through the birth of this book.

A. Emery

Authors

Ted Vosk. The product of a broken home, Ted was kicked out of his house as ateenager. Although managing to graduate from high school on time, he lived on thestreets, homeless, for the better part of the next four years. It was during this periodthat Ted began to teach himself physics and mathematics from books obtained at thepublic library. After getting into trouble with the military and running afoul of thelaw, he decided to change his situation. He gained admittance to Eastern MichiganUniversity where he was named a national Goldwater Scholar before graduating withhonors in theoretical physics and mathematics.

Suffering from severe ulcerative-colitis, Ted finished his last semester at EasternMichigan from a hospital bed. Days after graduating, he underwent a 16-hour surgeryto remove his colon. Despite this trauma, Ted entered the PhD program in physics atCornell University the following fall before moving on to Harvard Law School wherehe obtained his JD.

Since law school, Ted has been employed as a prosecutor, public defender, andthe acting managing director of a National Science Foundation Science and Tech-nology Center. On the side, he helped form Celestial North, a nonprofit organizationdedicated to teaching astronomy to the public and in schools. As vice president ofCelestial North, he played an integral role in its winning the Out of This World Awardfor Excellence in Astronomy Outreach given by Astronomy Magazine. He is currentlya legal/science writer, criminal defense attorney, and legal/forensic consultant.

Over the past decade, Ted has been a driving force behind the reform of forensicpractices in Washington State and the laws governing the use of the evidence theyproduce. His work in and out of the courtroom continues to help shape law in juris-dictions around the country. For this work, he has been awarded the President’s Awardfrom the Washington Association of Criminal Defense Lawyers and the Certificateof Distinction from the Washington Foundation for Criminal Justice. A Fellow ofthe American Academy of Forensic Sciences and member of Mensa, he has written,broadcast, presented, and taught around the country on topics ranging from the ori-gins of the universe to the doctrine of constitutional separation of powers. He has beenpublished in legal and scientific media, including the Journal of Forensic Sciences,and his work has been cited in others, including Law Reviews.

During the past several years, Ted waged the fight for reform while suffering fromdebilitating Crohn’s disease. This delayed the beginning of this text by almost 4 years.In the Summer of 2012 he underwent life saving surgery to remove a major sectionof what remained of his digestive system. With help from his wife and friends aroundthe country, though, he rose once again. Within six months he ran two half marathonson consecutive weekends to help find a cure for the diseases that have afflicted himfor two decades so that others wouldn’t have to suffer as he has. Only in the wake of

xxxix

xl Authors

this, about 10 months before this text was written, was he able to sit down and beginwriting.

Ted lives in Washington State with his wife, Kris, whose love saved him frommore destructive paths. Although his life has been one of overcoming obstacles, ithas always been Kris who gave him the strength to do so. Whether chasing downactive volcanoes, swimming with wild dolphins, or simply sharing a sunset in themountains, they live their lives on their own terms . . . together.

Ashley F. Emery is a professor of mechanical engineering at the University of Wash-ington and an adjunct professor of architecture and of industrial and systems energy.He has been an associate dean of the College of Engineering, chair of the Departmentof Mechanical Engineering, and director for the Thermal Transport Program of theNational Science Foundation. His areas of research interest are heat transfer, fluiddynamics, architectural and building energy, thermal stresses, fracture, design andinterpretation of experiments, and Bayesian inference. He has published more than200 technical papers in refereed journals. He is a fellow of the American Society ofMechanical Engineers and the American Society of Heating, Refrigerating and Air-Conditioning Engineers. He is a recipient of the American Society of MechanicalEngineers Heat Transfer Memorial Award and the 75th Anniversary Heat TransferAward.

Prologue

This text grew out of a 120-page outline I wrote from a hospital bed to accom-pany a presentation I delivered at William “Bubba” Head’s National Forensic Bloodand Urine Testing Seminar in May of 2009. The presentation was titled, ForensicMetrology—The Key to the Kingdom, and is the first I’m aware of to introduce foren-sic metrology as an independent forensic discipline to the legal community. At thetime I wrote the outline, a Google search of the phrase Forensic Metrology turnedup a single reference in the literature. It was a generic paragraph in a chapter aboutmeasurement standards which read:

Forensic metrology is the application of measurements and hence measurement stan-dards to the solution and prevention of crime. It is practiced within the laboratories of lawenforcement agencies throughout the world. Worldwide activities in forensic metrologyare coordinated by Interpol (International police; the international agency that coordi-nates the police activities of the member nations). Within the U.S., the federal Bureau ofInvestigation (FBI), an agency of the Department of Justice, is the focal point for mostU.S. forensic metrology activities.1

Given the paucity of literature addressing metrology as a forensic discipline andthe need for legal professionals to become acquainted with it as such, I made theoutline available as a self-published PDF text titled, Forensic Metrology: A Primerfor Lawyers and Judges.2

Interest in the subject grew quickly in the legal community. Legal professionalswere not the only ones who wanted to learn more, though. Before long, I was gettingcalls from forensic scientists as well wanting to know about this “new” discipline.Less than a year later, in February 2010, I presented, Metrology: A Knowledge Basefor Communication and Understanding, at the 62nd annual meeting of the AmericanAcademy of Forensic Sciences as part of a Workshop that forensic scientist MaxHouck, Dr. Ashley Emery, and I put together.3 By this time, the Primer had beenrenamed Forensic Metrology: A Primer for Lawyers, Judges and Forensic Scientists.∗Max passed the Primer on to his publisher and recommended that it be turned intoa book. Within a short period of time, CRC Press contacted me with an offer to turnthe Primer into this textbook.

The text actually started taking shape almost a decade before this, though, inSeattle, Washington in early 2001. That is when I began work as a public defender.The firm I was working for was challenging the admissibility of breath alcohol results.The issue involved was the measurement of temperature of “simulator” solutions usedto calibrate and check the accuracy of breath test machines. With degrees in physicsand mathematics, I was well equipped for the challenges presented by the use offorensic science so my boss, Howard Stein, asked me to take a look at it. It soonbecame apparent that there was more wrong than anyone had realized.

∗ The latest version of the Primer is included on the CD that accompanies this text.

xli

xlii Prologue

The State Toxicology Lab claimed that the accuracy of the temperatures reportedfor the solutions were given by the “margin of error” of the thermometers used tomeasure them. The problem with this is that the claim ignored other, potentially moresignificant, sources of error involved in the measurement. As an attorney, though, myrole in the courtroom is as an advocate, not a witness. If I was going to be able to provethis, I needed an expert who could investigate and testify about the issues involved toa judge.

I made up a list of potential experts in the measurement of temperature to inter-view. The first name on my list forgot about our meeting and wasn’t there when Iarrived. This turned out to be one of those fortuitous turns of fate that so often leadsto something special. I say this because the next name on the list was an individualwith whom I would become close friends and collaborate with for over a decade.

Dr. Ashley Emery, or Ash as I came to call him, is a professor of mechanicalengineering at the University of Washington. His research focus in thermodynamicsand heat transfer made him a perfect candidate for what I needed. Given his manyaccomplishments, however, which included being part of a group that consulted forNASA concerning the heat shield used on the Space Shuttle, I didn’t think he wouldbe interested. To my surprise, after I finished explaining the issue, Ash jumped rightin. To him, this wasn’t about a courtroom battle. Rather, it was a matter of goodscience and of being able to apply the knowledge he had built over a lifetime to thesolution of a new problem.

The next couple of months involved a lot of hard work. Ash conducted a study onthe thermometers used and found that the uncertainty of the temperatures reportedwas significantly greater than claimed. I visited the State Lab in question and discov-ered that the thermometers themselves were not being used in a manner consistentwith their validation rendering any values reported unreliable. After a day-longhearing wherein these issues were addressed, the Court suppressed the breath testresults.4

The victory wasn’t about “just trying to get another guilty person off” as is sooften lamented by critics, though. It was about preventing the government from usingflawed science to deprive citizens of their liberty. Every one of us is innocent untilproven guilty. That is one of the safeguards against tyranny provided by our Consti-tution. When the government tells a judge or jury that science supports claims thatit does not, it is tantamount to committing a fraud against our system of Justice. Itdoesn’t matter whether the deception is purposeful or not because the result is thesame: a Citizen’s liberty is imperiled by a falsehood. This is what Ashley and I havefought against for over a decade.

Bad government science doesn’t necessarily arise from bad government scientists.Nor is the desire to ensure that science is used correctly to discover truth in the court-room confined to defense attorneys. Forensic scientists, prosecutors, and judges havesought the same goals Ash and I have and worked with us to achieve them.

In 2004, forensic scientist, and then head of the Washington State Breath TestProgram, Rod Gullberg helped Scott Wonder and I to keep the government fromadministratively suspending a woman’s driver’s license.5 She had submitted to abreath test that yielded duplicate results both in excess of the legal limit. ThroughRod, we showed that the uncertainty associated with the results proved that there was

Prologue xliii

actually a 56.75% probability that her true breath alcohol concentration was less thanthe legal limit.

The bad government science in this case was not that done by forensic scientists.To the contrary, it was one of the State’s top forensic scientists who, without chargingthis woman a single penny, used metrology to establish that it was more likely thannot that she had not violated the law. Rather, it was what government officials didwith otherwise good science that rendered it bad. Ignoring what science said aboutthe conclusions these results supported, the Washington Department of Licensingsuspended this Citizen’s license anyway. It was only on appeal that a court reversedthe suspension. Without her license, this woman would have lost her job. And withoutRod’s help, she probably would have lost her license.

Much of the work we’ve done over the years has involved forensic breath and bloodalcohol testing. The determination of a person’s breath or blood alcohol concentrationin these ways are examples of forensic measurements. This type of forensic evidenceis quite common because the crime of DUI is defined by the results of these measure-ments. Although neither Ash nor I were forensic scientists, or even very familiar withforensic breath and blood alcohol testing in the beginning, we were able to subjectthem to analysis because we were both well versed in the science of metrology.

Metrology is the science of measurement. Its principles apply to all measurementsmade anywhere and for any purpose and provide the basic framework necessary toperform, analyze, and arrive at sound conclusions based on measured results. Mea-surement uncertainty and the use of validated methods, which were relied upon inthe cases above, are fundamental elements of metrology. They are by no means theonly ones though. Another is measurement traceability, which ensures that measuredresults represent what they are purported to. On the heels of our first victory, Ash andI used traceability to help attorneys Howard Stein and Scott “Scooter” Robbins put anend to more bad government science and get the first published decision to explicitlyrecognize metrology in a forensic context.6 And there are many other metrologicaltools that can be relied upon in the courtroom and lab alike to ensure that the misuseof science doesn’t undermine the discovery of truth in our system of justice.

While breath and blood alcohol tests are common forensic measurements, they areby no means the only ones. Determining the weight of seized drugs using a scale; thespeed of a motor vehicle using a radar; the angle at which a bullet entered a wall usinga protractor; and even the distance between a drug transaction and a school using ameasuring wheel; these are just a few of the many types of forensic measurementsthat are performed. And the same underlying metrological principles that allowed Ashand I to analyze and determine the truth about breath and blood alcohol measurementsapply to each of these and every other forensic measurement as well.

This leads to an astonishing conclusion. Since the science of metrology under-lies all measurements, its principles provide a basic framework for critical evaluationof all measurements, regardless of the field they arise out of. Given a familiaritywith metrology, scientists and police officers can better perform and communicatethe results of the forensic measurements they make; lawyers can better understand,present and cross-examine the results of forensic measurements intended to be usedas evidence; judges will be better able to subject testimony or evidence based onforensic measurements to the appropriate gatekeeping analysis; and each of these

xliv Prologue

participants will be better prepared to play their role in ensuring that the misuse ofscience doesn’t undermine the search for truth in the courtroom.

This was the idea I had in mind when, in the Summer of 2007, a forensic scien-tist within the Washington State Toxicology Lab was discovered committing perjuryabout measurements she claimed to have made. Upon further investigation, though,a team consisting of Kevin Trombold, Andy Robertson, Quentin Bajter, Ash, andmyself, with assistance from others around the State, discovered that the Lab’s prob-lems went far deeper than perjury. The labs process for creating simulator solutionsfor the calibration and checking of breath test machines was in a state of disarray.Failures to validate procedures, follow approved protocols, adhere to scientificallyaccepted consensus standards, properly calibrate or maintain equipment, and even tosimply check the correctness of results and calculations were endemic.

In a private memo to Washington’s Governor, the State Toxicologist explained thatthe measurement procedures in question “had been in place for over twenty years andhad gone unchallenged, leading to complacency.” What allowed us to find what othershad missed over the years was, again, metrology. Viewed through the appropriatemetrological framework, it became clear that what complacency had led to was thesystemic failure of the Lab to adhere to fundamental scientific requirements for theacquisition of reliable measurement results. After a seven-day hearing that includedtestimony from nine experts, declarations from five others, as well as 161 exhibits, apanel of three judges issued a 30-page ruling suppressing all breath test results untilthe Lab fixed the problem’s identified.7

Under the leadership of newly hired state toxicologist Dr. Fiona Couper andquality assurance manager Jason Sklerov, the Lab subsequently used the same metro-logical framework to fix its problems that we had used to discover them. It didso by implementing fundamental metrological principles and practices and obtain-ing accreditation under ISO 17025, the international standard that embodies them.Because of this, the Washington State Toxicology Lab has one of the best Breath TestCalibration programs in the United States. The same metrological principles that canbe such effective tools in the hands of legal professionals can be even more powerfulwhen employed by competent forensic scientists.

In the wake of these proceedings, I was contacted by lawyers from around thecountry. I explained how we had used metrology to discover the Lab’s problems andeven shared the 150-page brief we submitted in the Washington proceedings. One ofthose lawyers was Bryan Brown who subsequently used many of the same metrolog-ical principles to expose problems in breath tests being administered in WashingtonDC. What we had done using metrology in Washington State could be done just aswell by others elsewhere. Unfortunately, most in the legal community had still neverheard of metrology and were unaware of what a powerful tool for the discovery oftruth it was.

It was during this period that Bubba invited me to teach an audience of crim-inal defense lawyers about metrology at his seminar. Shortly after this I attendedthe weeklong meeting of the American Academy of Forensic Sciences in Denver,Colorado. Near the end of the week, the National Academy of Sciences released areport on the state of forensic science in America.8 It was very critical of the prac-tices engaged in by many of the forensic sciences. As you will see as you make your

Prologue xlv

way through this text, the very issues identified by the report are those that metrol-ogy addresses. Method validation, adherence to appropriate practices as evidencedby consensus standards, the determination and reporting of measurement uncertainty,and others. What we had done in Washington State with respect to forensic measure-ment was to not only beat the Academy to the punch in the discovery of these issues,but also to the identification of the appropriate framework for their solution.

But how could that knowledge be shared with as wide an audience as possible?Judges and lawyers needed something that set forth the framework of metrology in amanner that could be easily understood and relied upon. That’s when the idea of thePrimer hit me. A brief hospitalization gave me the time to put together the 120-pagePrimer that would eventually introduce lawyers and judges around the Country to thesubject of forensic metrology.

Examples of lawyers who were introduced to metrology through the Primer andpresentations made based on it include: Mike Nichols from Michigan who wassuccessful in getting courts there to require the determination and reporting of uncer-tainty of forensic blood alcohol results; Justin McShane from Pennsylvania whoemployed its principles in educating courts there on the importance of the range ofcalibration; and Joe St. Louis from Arizona who also used the briefing from the Wash-ington State proceedings to help identify and expose similar problems in one of theArizona’s toxicology labs. And each of these individuals has already begun to passon what they’ve learned. And this is just the tip of the iceberg. Not only can lawyerslearn forensic metrology, but the success of these individuals proves that they canemploy its principles as good, if not better, than we originally did in Washington.

In the forensics community, metrology is being relied upon to address many ofthe issues identified by the National Academy of Sciences. Its principles are help-ing to improve how forensic measurements are developed, performed, and reported.Accreditation and adherence to international scientific standards are restoring con-fidence that forensic measurements comply with the same rigorous methodologyfollowed in other sciences. And it is providing a common language for all thoseengaged in making, or relying upon, forensic measurements to communicate aboutthem regardless of application.

Max Houck, co-chair of the AAFS workshop where the Primer was introduced tothe forensic community, has not only done much to contribute to the growth of foren-sic metrology as a discipline, but he relies upon it in practice. As the first directorof Washington DC’s Department of Forensic Sciences, he not only sought accredi-tation to ISO 17025 standards for the Lab, but achieved it in the almost unheard oftime frame of 8 months. Accreditation to ISO 17025 provides objective evidence tothe public that measured results reported by the Lab and relied upon by the criminaljustice system for the determination of factual truth can be trusted.

Present at the AAFS workshop Max, Ashley, and I put together was Dr. SandraRodriguez-Cruz. Dr. Rodriguez-Cruz is a senior forensic chemist with the U.S. DrugEnforcement Administration and the Secretariat of SWGDRUG, the Scientific Work-ing Group for the Analysis of Seized Drugs. She has been a driving force behindthe adoption and recommendation of rigorous metrological practices in the standardspublished by SWGDRUG. In addition to this, not only does she employ and teach

xlvi Prologue

metrology within the confines of the DEA, but she also spreads awareness by pre-senting on it at forensic conferences. None of this is done to the simple end thatthe government will “win” when it enters the arena. Rather it is to ensure that thosecharged with the task of discovering truth in the courtroom have the best evidenceavailable to do so.

Ashley and I teamed up once again in 2009, first with Attorney Eric Gaston andthen later separately with Kevin Trombold and Andy Robertson, to wage a new bat-tle over the use of forensic measurements in the courtroom. The second skirmishinvolved a five-day hearing that included testimony from Ash and the government’stop three experts as well as 93 exhibits. After the smoke cleared, the panel of threejudges presiding over the hearing issued a 30-page order declaring that breath testresults would henceforth be inadmissible unless they were accompanied by theiruncertainty.

The rulings from these cases garnered nationwide attention.9 Lawyers, judges,forensic scientists, and scholars from around the country began discussing and writingabout the importance of providing a measured result’s uncertainty when the result willbe relied upon as evidence at trial. Thomas Bohan, former president of the AmericanAcademy of Forensic Sciences, declared it to be “a landmark decision, engendering ahuge advance toward rationality in our justice system and a victory for both forensicscience and the pursuit of truth.”10 Law professor Edward Imwinkelried followed thisup by explaining that reporting the uncertainty of forensic measurements:

. . . promotes honesty in the courtroom. It is axiomatic that measurements are inher-ently uncertain. As the Washington cases emphasize, it is misleading to present thetrier of fact with only a single point value. There is a grave risk that without the ben-efit of qualifying testimony, the trier will mistakenly treat the point value as exact andascribe undue weight to the evidence. The antidote—the necessary qualification—is aquantitative measure of the margin of error or uncertainty.11

The battle was subsequently taken up by defense attorneys in several states includ-ing Michigan, Virginia, New Mexico, Arizona, California, and even in the FederalCourts, and continues to spread as of the time of this writing in January 2014.∗

It’s not just defense counsel who have joined this quest, though. In a 2013 paperpublished in the Santa Clara Law Review, my friend, prosecutor Chris Boscia, pro-vided the rational for why all those advocating on behalf of the state should be fightingfor the same thing. In fact, after a trial court denied a defense motion to require thereporting of uncertainty and traceability with blood test results, Chris worked withthe lab to make sure that this was done for all future results despite the court’s rul-ing. And now he’s working to make this a mandatory regulatory requirement. Why?Because he wants to ensure that the science presented by the state in court is “the bestscience regardless of what the law requires.”12

The truth about any scientific measurement is that it can never reveal what aquantity’s true value is. The power of metrology lies in the fact that it provides the

∗ The battle has been taken up in Michigan by Mike Nichols (and expert Andreas Stolz), Virginia byBob Keefer, New Mexico and the Federal Courts by Rod Frechette (and expert Janine Arvizu) and inCalifornia by Peter Johnson.

Prologue xlvii

framework by which we can determine what conclusions about that value are sup-ported by measured results. It tells us how to develop and perform measurementsso that high-quality information can be obtained. It helps us to understand what ourresults mean and represent. And finally, it provides the rules that guide our inferencesfrom measured results to the conclusions they support. Whether you are a prosecu-tor or defense attorney, judge or forensic scientist, or even a law enforcement officerwho performs measurements as part of investigations in the field, forensic metrol-ogy provides a powerful tool for determining the truth when forensic measurementsare relied upon. Forensic science, legal practice, and justice itself are improved by afamiliarity with the principles of forensic metrology.

The focus of this text is on the metrological structure required to reach soundconclusions based on measured results and the inferences those results support.Although metrological requirements for the design and performance of measure-ment are addressed in this context, the text does not set forth in detail proceduresfor doing so.

Section I provides an introduction to forensic metrology for both lawyers and sci-entists. The focus is on the development of principles and concepts. The scientificunderpinnings of each subject are presented followed by an examination of each inlegal and forensic contexts. By presenting the material in this manner, it will allowthe lawyer, judge, or forensic scientist to immediately see its application to the workthey perform.

Although there is some math, particularly in Chapters 6 and 7, it is not necessaryto work through it to understand the materials. For the forensic scientist, it providessome necessary foundation for employing metrology in the lab. For the legal profes-sional, it shows the type of analysis you should expect from a competent forensiclab and will prepare you for what you should see when metrologically sound resultsare provided in discovery or presented in court. The accompanying CD includes thelatest version of the Forensic Metrology Primer as well as motions, court decisions,and expert reports for legal practitioners.

Section II of the text provides a more advanced and mathematically rigorous cover-age of the principles and methods of inference in metrology. Statistical, Bayesian andlogical inference are presented and their relative strengths and weaknesses explored.On a practical level, this is intended for those who wish to engage in or challengemeasurement-based inference. As such, although it’s primary target is the scientist,legal professionals who feel comfortable with its material will find it very useful aswell. On a more fundamental level, it will be enjoyed by those who wish to under-stand the types of conclusions each school of inference can support and how their usecan facilitate the search for factual truth in the courtroom.

Citations in Sections I and II of the book follow different conventions. Citations inSection I of the book are formatted to make it more accessible to legal practitioners.Section II uses journal citation format which will be familiar to researchers.

As I write this, a new decision out of Michigan suppressing blood alcohol testresults for failure to establish their traceability or accurately determine their uncer-tainty has just been handed down by a trial court. And here in Washington, we havejust begun to introduce the criminal justice system to the concept of a measurand andthe important role it plays in forensic measurements.

xlviii Prologue

From the time of our first case together in 2001, the quest Ash and I have beenon is one to stop the government from using flawed science to deprive citizens oftheir liberty. And beginning with the Primer, our goal has been to teach others theprinciples of metrology, enabling them to join the fight to improve the quality ofjustice and science when forensic measurements are relied upon in the courtroom.The list of those who have contributed is long and there have been both victoriesand defeats, but every fight and every individual who has helped wage it has broughtabout improvement. We hope that this text will set spark to tinder and arm you tojoin the fight . . . because neither science nor justice can be any better than the peoplededicated to their perfection.

ENDNOTES

1. DeWayne Sharp, Measurement standards, in Measurement, Instrumentation, and Sensors Handbook5-4, 1999.

2. Ted Vosk, Forensic metrology: A primer for lawyers and judges, first published for National ForensicBlood and Urine Testing Seminar, San Diego, CA, 120pp., May, 2009.

3. Co-Chairs: Ted Vosk and Max Houck, Attorneys and scientists in the courtroom: Bridging the gap,Workshop for the American Academy of Forensic Sciences 62nd Annual Scientific (Feb. 22, 2010),in Proceedings of the American Academy of Forensic Sciences, Feb. 2010, at 15.

4. City of Bellevue v. Tinoco, No. BC 126146 (King Co. Dist. Ct. WA 09/11/2001).5. Herrmann v. Dept. of Licensing, No. 04-2-18602-1 SEA (King Co. Sup. Ct. WA 02/04/2005).6. City of Seattle v. Clark-Munoz, 93 P.3d 141 (Wash. 2004).7. State v. Ahmach, No. C00627921 (King Co. Dist. Ct. – 1/30/08).8. Nat’l Research Council, Nat’l Academy of Sciences, Strengthening Forensic Science in the United

States: A Path Forward, 2009.9. State v. Fausto, No. C076949 (King Co. Dist. Ct. WA – 09/20/2010); State v. Weimer, No. 7036A-

09D, (Snohomish Co. Dist. Ct. WA – 3/23/10).10. Ted Vosk, Trial by Numbers: Uncertainty in the Quest for Truth and Justice, the nacdl champion,

Nov. 2010, at 48, 54.11. Edward Imwinkelried, Forensic Metrology: The New Honesty about the Uncertainty of Measure-

ments in Scientific Analysis 32 (UC Davis Legal Studies Research Paper Series, Research Paper No.317 Dec., 2012), available at http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2186247.

12. Christopher Boscia, Strengthening Forensic Alcohol Analysis in California DUI Cases: A Prosecu-tor’s Perspective 53 Santa Clara l. rev. 733, 763, 2013.

Section I

An Introduction to ForensicMetrology for Lawyers, Judges,and Forensic Scientists

1 Science, Metrology,and the Law

1.1 SCIENCE!

Science has facilitated some of humankind’s greatest achievements. Through it, wehave been able to formulate simple mathematical laws that describe the orderlyUniverse we inhabit; to peer back to the moments following its creation and to traceits evolution over billions of years; to explain the creation of our home planet some4.5 billion years ago; and document the appearance and evolution of life that even-tually led to us. On a more practical level, science has freed humankind from its fearof the night through the creation of lights to guide us through the darkness; from theconstraints of geography as automobiles and airplanes transport us across continentsand over oceans; and finally even from the confines of our small planet itself as wereach out to travel to and explore other worlds. Science has allowed us to harness thepower of the atom and the gene, for both creative and destructive ends. And throughthe technologies made possible by science, life today is one of ease compared tothat of our brutish ancestors. Few would deny that our species relies upon science toanswer questions of fact, both profound and practical, every day.

1.1.1 SCIENCE AND THE LAW

Given the great power of science to help answer factual questions, one should notbe surprised at its widespread employment in the investigation and trial of crimes.And, in fact, it has been observed that “[c]omplete, competent, and impartial forensic-science investigations can be that ‘touchstone of truth’ in a judicial process that worksto see that the guilty are punished and the innocent are exonerated” [122].1 Historyteaches us, however, that courts, as do societies, often reject valid science in favor ofwidely held misconceptions.

Perhaps, the most infamous example of a court rejecting valid science in favorof such a misconception is the 1630 trial of Galileo “where the scientist was con-victed of heresy for asserting that the earth revolves around the sun.”2 Even commonsense dictated that Galileo must be wrong for if the Earth were actually movingabout the sun in this manner everyone would feel it doing so. It wasn’t until 1983,350 years after he had been sentenced to house arrest for the remainder of his life, thatthe Church reexamined Galileo’s evidence and admitted that it had made a mistakethereby exonerating him posthumously. And in what may be one of the first reportedinstances of expert testimony at trial, in 1665 an expert gave his “scientific” opinionthat the accused were witches and, by practicing their witchcraft at the devil’s bid-ding, had bewitched several children. Thereafter, the accused were found guilty and

3

4 Forensic Metrology

hanged [25].3 The issue of the validity of the process for determining whether onewas a witch was never even raised.∗

Science has made great strides since the seventeenth century. Over the past decade,however, forensic science and its use in the courtroom have come under increasing fireby scientists, scholars, and legal professionals. The worst of the criticism may havecome from a 2009 report published by the National Research Council of the NationalAcademy of Sciences titled Strengthening Forensic Science in the United States: APath Forward. One of the findings of the report was that “[t]he law’s greatest dilemmain its heavy reliance on forensic evidence [] concerns the question of whether—and towhat extent—there is science in any given ‘forensic science’ discipline” [28].4 Giventhe significant role forensic evidence and testimony often plays in the courtroom, theweaknesses identified threaten to undermine the integrity of our system of justice asa whole. Thus, it is critically important for today’s forensic scientists to understand,be able to carry out and communicate good science.

By itself, though, the forensic community cannot ensure that only good science isrelied upon in the courtroom.

The adversarial process relating to the admission and exclusion of scientific evidenceis not suited to the task of finding ‘scientific truth.’ The judicial system is encumberedby, among other things, judges and lawyers who generally lack the scientific expertisenecessary to comprehend and evaluate forensic evidence in an informed manner. . . 5

As a result, oftentimes the law itself either inhibits, or, at the very least, failsto require, good scientific practices. For example, “established case law in manyjurisdictions supports minimal analytical quality control and documentation” [70].6

If the law seeks outcomes consistent with scientific reality, it must require that sci-entific evidence “conform to the standards and criteria to which scientists themselvesadhere” [10].7 “In this age of science we must build legal foundations that are soundin science as well as in law” [17].8 Although the forensic community can inform thisprocess, they are not the ones with the power to shape those foundations. That powerlies in the hands of legal professionals, the very lawyers and judges who rely uponand encounter such evidence on a daily basis and the academics who write about it.No longer can legal professionals fall back on the excuse that they lack the scien-tific background or experience to comprehend and evaluate forensic evidence. If thegoal is to ensure just outcomes when scientific evidence is relied upon, then the legalprofession must shoulder a significant burden as well.

1.1.2 A FOUNDATION FOR SCIENCE IN THE COURTROOM

Where do we start, then, if we wish to give both forensic science and the legal doc-trines that govern its use a sound scientific foundation? A good way to start wouldbe how scientific investigation itself typically starts: by setting forth, in as complete

∗ Reed v. State, 391 A.2d 364, 370 n.7 (Md. 1978).

Science, Metrology, and the Law 5

a manner as possible, what it is we wish to investigate. We can do this by asking aseemingly simple question that still causes confusion and tumult: What is science?∗

1.2 WHAT IS SCIENCE?

Exactly what constitutes “science” has long been a matter of debate. The word “sci-ence” comes from the Latin scientia, meaning knowledge. This is natural enough asscience is characterized by the goal of acquiring knowledge. There are many activi-ties where the goal is the acquisition of knowledge, however, that would not be seenas science. For example, both philosophy and religion are frequently relied upon as asource of, or tool for obtaining, knowledge, but neither constitutes science. So, what isit that separates these disciplines from that which would be characterized as science?

1.2.1 KNOWLEDGE OF THE PHYSICAL UNIVERSE

There are several elements that, when taken together, are commonly considered todistinguish science from other disciplines. First, the overall goal of science is to obtainknowledge of a specific and strictly limited subject: the material, or physical, universewhich is made up of corporeal processes and entities. All of science is limited tophenomena of this sort.

1.2.1.1 Descriptive versus Explanatory

There is a subtlety here, though, that must be explored. What do we mean by knowl-edge of the physical universe? Is the knowledge sought explanatory, meant to helpus understand why certain phenomena occur? Or is it enough for it to simply bedescriptive, successfully revealing nature’s observable features or modeling how theybehave without telling us more?

According to Einstein “[s]cience can be created only by those who are thoroughlyimbued with the aspiration toward truth and understanding...” [49,85].9 The knowl-edge sought is to understand the Universe intimately and fundamentally. To know notsimply the path that the Sun will take across the sky, but to understand what dictatesits motion. Discussing the type of knowledge he sought, Einstein once explained:

I am not interested in this or that phenomenon, in the spectrum of this or that element.I want to know God’s thoughts, the rest are details [132].10

This is the quest for many scientists. But it is a quest that, from the outset, thewisest knows may be illusory. The reason is that it is based upon the dual assumptionsthat not only does the behavior of the physical Universe obey strict, fundamental anduniversal rules, but that we are capable of “seeing” and understanding them. Thefirst assumption seems obvious today, but a priori there is no scientific principlethat compels it to be so. Why should the Universe be composed of orderly laws thatdetermine what shall take place within it? Will those rules evolve or decay over time,

∗ The discussion that follows is not meant to be exhaustive but simply suggestive of some of the majorthemes.


or at least as long as something such as time exists? Is it possible that the order we seearound us is the result of a chance configuration of the state of the Universe and thatother states may manifest wherein such order is absent? At the core of this quest liesa belief, akin to faith although not lacking in empirical support, that, fundamentally,the Universe is of a particular character.

The second assumption seems far more precarious. To be sure, we interact withand sense the world around us. But how much are we really equipped to “see” andunderstand? Remember, we are simply another animal, inhabiting a wet rock, floatingthrough space around what seems to be a rather typical star, in the outskirts of a smallgalaxy that is barely a speck, in what appears to be, despite the existence of hundredsof billions of galaxies, a mostly vast and empty Universe. Against such a backdrop,almost any claim other than ignorance seems hubristic. That is, of course, until weremember our many great scientific achievements, which include those mentioned inthe first section of this chapter. With these in mind, it certainly seems that we are ableto “see” and understand the physical world about us. Still, what does this say aboutthe depth of our understanding? Consider quantum mechanics.

1.2.1.2 Example: Quantum Considerations

Developed in the early part of the twentieth century to address phenomena that clas-sical physics could not explain, quantum mechanics is one of human kind’s mostsuccessful scientific achievements. It governs the behavior of particles such as atomsand electrons; is the foundation for chemistry, describing the rules that hold moleculestogether; underlies our modern theories of electricity, magnetism, and light and howthe nuclei of atoms are held together and how they decay; and even plays a role inour cosmological theories. Because of quantum mechanics we have transistors whichmake personal computers possible, lasers which are necessary for DVDs and Blu-raytechnology, CCDs used in digital cameras, and a host of other technologies that werely upon every day.

Despite its great power, though, are we actually to believe the picture of realitythat quantum mechanics paints? Is our universe really made up of tiny entities thatexist as a superposition of particle and wave? Many of whose properties do not evenexist until observed? That pop into and out of existence throughout space? Accordingto Nobel Laureate Richard Feynman, we can “safely say that nobody understandsquantum mechanics” [55].11

1.2.1.3 Knowledge as Description and Model

By this Feynman does not mean that we do not understand the rules of quantummechanics and how to apply them. To the contrary, we understand these things verywell. He simply means that we do not understand the physical reality that createsthem. The relevance of this is that it shows us that no matter how powerful our scien-tific knowledge is, it may simply be descriptive in nature, successfully revealing thetypes of observations that can be made, or modeling how an aspect of the Universebehaves, without revealing the what or why of that which lays beneath. Thus, perhapssometimes science does reveal to us something about the actual why, what or how of


physical reality, but there is no requirement for it to do so. As quantum theorist JohnVon Neumann explained:

The sciences do not try to explain, they hardly even try to interpret, they mainly makemodels. By a model is meant a mathematical construct which, with the addition of cer-tain verbal interpretations, describes observed phenomena. The justification of sucha mathematical construct is solely and precisely that it is expected to work—that is,correctly to describe phenomena from a reasonably wide area [152].12

This understanding is fundamentally important if we wish to understand whatknowledge can actually be gained through science. Scientific knowledge is a descrip-tion or model of our experience of the physical world. While it may also reveal thephysical reality underlying it, that requires something more, belief.

1.2.1.4 Example: The Ptolemaic Model of the Universe

Consider the Ptolemaic model of the Universe relied upon by the ancient Greeks. Thisdescription was based on careful observation and measurement of the motion of theplanets. Described by Ptolemy in the Almagest, in this model the Cosmos consists ofa series of concentric spheres, referred to as deferents, with the Earth stationary atits center. The largest sphere contained the stars while the eight smallest accountedfor the known planets, Sun and Moon. Now, the Greeks believed in the principle thateach of these extraterrestrial bodies must be confined to uniform circular motion asthey traveled in their orbits. To produce the motion of these objects in the sky, eachof the deferents rotated about their centers at different rates.

In this form, however, the model did not match all of the Greeks’ observations.One of the most troubling incongruities was the phenomenon of retrograde motion.As the Earth passes by other planets in its annual trek around the Sun, those planetsappear to briefly circle backward against the background stars before continuing ontheir journey. Of course this is simply a visual effect, but it seemed real enough tothe Greeks. To account for these observations, Ptolemy and his contemporaries madesome adjustments to the initially simple model.

First, each of the planets was removed from their deferent and attached to small,perfectly circular tracks called epicycles. Each epicycle would then be attached to itsdeferent and, as the deferent rotated, so to would the epicycle about its own center.Now, each deferent was offset so that the Earth was no longer at its center but at apoint called the eccentric. Although the epicycle orbited the Earth as the deferentrotated, it did not do so about the center of the deferent. Instead, its orbit centered ona point referred to as the equant which was located directly across from the eccentricwith the deferent’s center bisecting them (see Figure 1.1).

This configuration helped the Greeks better account for the more complicatedmotions of the planets that were actually observed. And, since the epicycles wereperfectly circular, it allowed the Greeks to maintain the principle of uniform cir-cular motion in a somewhat modified form. Over time, whenever the model failedto account for astronomical observations, more epicycles would be added so as tomaintain the model’s correspondence to the planets’ positions in the evening sky.


EpicycleDeferent

Equant

Center

Earth/eccentric

FIGURE 1.1 Ptolemaic model.

The Ptolemaic model, based upon careful celestial observation, was a greatachievement. Some versions continued to provide good approximations of planetarylocations even centuries later. But today we know that deferents and epicycles, usefulas they may have been, do not actually underlie the motions of the planets. This is aprime example of how our scientific knowledge may describe what we experience ofthe Universe while not actually revealing the physical reality underlying it.

It would seem, then, that the quest for scientific knowledge must be comprisedof equal parts curiosity and skepticism. The curiosity to want to understand how theUniverse works but the skepticism to question whatever would be forwarded as theanswer.

1.2.2 EMPIRICISM

Another element distinguishing science from other pursuits is that the evidence reliedupon to build our description/model of the physical world must be empirical in nature.That is, such evidence is limited to what can be obtained through observation, mea-surement, and experiment. This is a well-accepted and uncontroversial statement. It isnot unfettered reason upon which we base scientific knowledge, but what we collectthrough our senses, or their extension, from the outside world.

No matter what we believe the physical world to be, “[s]cience is based on the prin-ciple that. . . observation is the ultimate and final judge of the truth” [56].13 Regardlessof how brilliant, logical, or beautiful an explanation, it must be discarded if it is con-tradicted by our observations. This is one of the primary creeds of science. Moreover,as an empirical endeavor, science is not beholden to recognized authority or even whatwe find desirable. Nature adheres to its own natural laws regardless of how they affectus. Systematic observation, measurement, and/or experimentation are the genesis ofscientific understanding.


1.2.2.1 Information versus Fact

But what is it that our observations, measurements, or experiments provide? Theobvious answer is that what we obtain are facts about the physical world. This state-ment can be somewhat misleading, though. The term “fact” seems to imply that ourempirical activity yields little nuggets of truth that speak for themselves. That eachsuch “fact” has a singular fixed meaning that is a direct reflection of the phenomenaexplored. And that the outcome of our observation, measurement or experiment, pro-vides an unambiguous reflection of the phenomena of interest. This view is overlysimplistic, however. Every observation, every measurement, and every experimenttakes place within a physical context that includes the procedures, instruments, andconditions attendant to our empirical endeavor. And every aspect of that context mayinteract and/or alter the outcome in a manner that is unknown. Thus, it is difficultto maintain that what is obtained during these empirical activities are unambiguousnuggets of truth regarding the phenomenon of interest.

1.2.2.2 Example: Blood Alcohol Measurements

Consider the testing of blood to determine its alcohol concentration. These tests aretypically performed utilizing gas chromatography to determine what an individual’salcohol concentration was at the time the blood is drawn. The presumption is that theconcentration of alcohol in the sample of blood will be the same when it is tested aswhen it was drawn.

Microbial contamination of a blood sample can occur during a draw, however, andsome microbiota can produce additional alcohol in the blood sample via fermenta-tion. Moreover, although a gas chromatograph is capable of providing accurate resultsconcerning the concentration of alcohol in a sample of blood, it does not test for thepresence of alcohol-producing microbiota. As a result, we may have no informationconcerning whether the alcohol present in the sample at the time it was tested couldhave been produced by microbiota after it had been drawn. Accordingly, we can-not necessarily conclude that the alcohol concentration at the time of the test was thesame as when the blood was drawn. The result of the blood test is not an unambiguousreflection of the phenomenon sought to be investigated.

1.2.2.3 Incomplete Information

There are no unambiguous nuggets of truth here providing us with the answers to ourquestions. What we obtain from observation, measurement, or experiment is simplya specific type of experiential information, nothing more. We can never know exactlywhat that information represents. To be sure we may have certain beliefs as to whatlies behind it, but there is no way to physically prove it. This information includes notjust the result obtained, but the universe of empirical information we have relevant tothe question asked as a whole. And whatever knowledge we build will be subject tothe limitations of the information it is based on. Whether it permits the question askedto be adequately addressed is dependent upon the amount, quality, and relevance ofthe information obtained.


The example of the blood test demonstrates this nicely. There, we had informa-tion concerning the result of the test but none concerning the presence of microbes.Thus, to extrapolate from a test result to the concentration of alcohol in the sampleat the time it was drawn may be misleading. On the other hand, there might be otherinformation that could help address that question such as whether and how muchpreservative was in the tube used to collect the blood, whether the sample was refrig-erated after collection and what precautions were taken with respect to the blood drawitself. But what can be learned from the blood test is dependent upon the universe ofinformation we have concerning the collection and testing of the blood sample.

In the same way that our understanding of physical reality may be of a more super-ficial (descriptive) kind, the information we obtain may concern only the surfacefeatures of the phenomena of interest and lack significant content concerning whatlies deeper as well. This may result because our procedures are not intended to obtaincertain types of information, because our instruments are only capable of exploring aparticular aspect of the phenomena of interest, because certain conditions cannot becontrolled for or any number of other possibilities. Thus, while our information mayreflect the core of the phenomena of interest, because it is never perfect or complete,we can never know whether it actually does. The claim that it does requires somethingmore, belief.

1.2.3 RECAP

Our description of science so far may be surprising to some. Despite our aspirations,any claim to understanding the actual why, what, or how of physical reality or basingthis knowledge on nuggets of factual truth requires something more usually attributedto other endeavors: our belief that these things are true. Instead, our scientific claimsare of a more limited nature. Our scientific knowledge is a description/model of ourexperience of the physical world, and that knowledge is based upon incomplete andimperfect information obtained by empirical means. Either one or both of these mayreveal the true nature of the phenomena giving rise to them, but this is something thatcannot be known, only believed or disbelieved.

1.2.4 HALLMARKS OF SCIENCE

Still this does not exhaust the elements that characterize what science is. Some ofthese attach to the conclusions and ideas science gives rise to, while others concernthe activities engaged in to generate them. Consider the following carefully as this isan area where debates commonly break out.

1.2.4.1 Falsifiability and Testability

Some have claimed that the hallmark of science is the falsifiability, refutability, ortestability of its claims [124].14 From this perspective, no matter how a claim or resultis arrived at, unless it is susceptible to being refuted, it does not constitute science.But what do we mean by falsifiable, refutable, or testable? Can any experiment or


series of experiments ever completely settle a scientific question? Do we ever reallyhave complete information or knowledge about any circumstance?

Nobel Laureate Richard Feynman once wrote that “[i]t is scientific only to saywhat is more likely and what is less likely, and not to be proving all the time thepossible and impossible” [55].15 In other words, science cannot tell us what is oris not, only what is more or less likely based on the information we have. The rea-son is that our knowledge of the physical world is never perfect or complete. Hence,no matter how confident we are in a given scientific proposition, there will alwaysremain at least some of the unknown from which a contrary explanation could arise.Accordingly, whether we are aware of it or not, attendant to even the most successfulscientific claims is a degree of doubt and to even the most woeful failures a spec ofhope. Doubt may be minimized, but it can never be eliminated. Thus, strictly speak-ing, falsification and refutability are matters of degree, even though often treated assettled fact.

1.2.4.2 Puzzle Solving

Another proclaimed hallmark of science is as a puzzle-solving activity that takes placeagainst a background of accepted beliefs and rules (accepted scientific laws and prin-ciples), and for which there are shared criteria for determining when a puzzle hasbeen solved [101].16 Instead of trying to disprove things all the time, the goal is toaffirmatively add to our knowledge by successfully solving the questions we choose.In this context, the background of accepted beliefs and rules (accepted scientific lawsand principles) and the shared criteria for determining when a puzzle has been solvedare crucial.

1.2.4.3 Example: Puzzle Solving in Forensic Toxicology

Consider the bench toxicologist whose job is to test blood for the presence of cer-tain predetermined substances. The toxicologist is presented with a simple puzzle:to determine the concentration of a particular substance in the sample of blood. Thepuzzle is simple because the steps in solving it are not only likely to be well under-stood but to be explicitly set forth in the laboratory’s standard operating procedures(SOP). The SOP governing the test will be similar to a detailed checklist or recipesetting forth the steps to be taken.

(1) Prepare the samples as prescribed by laboratory SOP; (2) Prepare testing instrument(e.g., a gas chromatograph) as prescribed by laboratory SOP; (3) Load samples into testinstrument as prescribed by laboratory SOP; (4) Make sure all settings are as prescribedby laboratory SOP; (4) Press the start button; (5) Wait for the results to be produced;(6) Check and interpret results as prescribed by laboratory SOP; (7) Report results asprescribed by laboratory SOP; (8) If a problem occurs, address it as prescribed by lab-oratory SOP; (9) If problem not solved by means prescribed by laboratory SOP, reportit and discontinue testing.

Does the following of such a checklist constitute science? Although care must beexercised so that we can be confident in the accuracy of any result, little indepen-dent thought seems to be required. Could not the same checklist be followed by a


reasonably intelligent individual, trained simply to perform the required steps in therequired manner, with little if any scientific understanding at all? It seems little differ-ent in nature from a lifeguard using available technology to test the level of chemicalsin the pool he is watching over? And this author has yet to hear a lifeguard be accusedof performing science while on the clock!

But, perhaps our focus is too narrow. It is not the individual performance of eachtest standing alone that constitutes science. Instead, maybe it is that activity consid-ered as a component of an overarching whole that includes the scientific principlesrelied upon coupled with the development of the testing instrument, procedures andstandard interpretations as the first act that constitutes science. This brings to the forethe reliance upon accepted beliefs and rules (accepted scientific laws and principles)and shared criteria for determining when a puzzle has been solved.

1.2.4.4 Predicting Novel Phenomena

Yet another characteristic often considered a hallmark of science is the ability topredict novel phenomena that can be confirmed by observation or experimentation[104].17 Every theory (at least so far) will have its limitations or areas where, it mayappear to fail or be refuted. Not every theory, however, will be able to forecast novelphenomena that can be confirmed. Where a theory can predict such phenomena it isbelieved to encompass something more fundamental about the universe. It is not sim-ply a post hoc portrait of what has already been glimpsed but a tool revealing what isyet to be learned.

The prediction of something new that is confirmed through observation or experi-mentation offers particularly strong support for a scientific claim. Although scientistsand mathematicians are quite skilled at building models to account for what is alreadyknown, it is quite another to forecast what is yet to be discovered. When a theory cando this, it seems more than the simple addition of epicycles to account for each newobservation. Instead, it is a hint that we may have gone beyond simple model buildingand touched on something fundamental with respect to the physical universe.

1.2.4.5 Example: Prediction of a New Planet

Newton’s law of gravity describes the attractive force between two bodies arisingfrom their masses. Published in 1687, it can be simply expressed as

Fg = G · m1 · m2

d2 (1.1)

This tells us that the gravitational force between any two bodies with mass is pro-portional to the product of those masses and inversely proportional to the squareof the distance between them. This simple mathematical model not only accountedfor the motions of falling objects here on Earth, but, amazingly, the motions of theknown planets as they orbited the Sun. In March of 1781, however, the planet Uranuswas discovered. Newton’s Law was utilized to determine the planet’s orbit but sub-sequent observations were not in agreement with its predictions. Some argued that


this contradiction was a refutation of the Law’s claimed universality. And it couldhave been.

Using the same model, however, scientists showed that discrepancies between pre-diction and observation would go away if there were another planet beyond Uranusalso exerting a gravitational pull on it. Scientists went to work trying to calculatewhere the hypothetical planet must be. Relying upon these predictions, in Septemberof 1846, two astronomers aimed their telescope at the designated location in the skyand discovered the planet Neptune. Hence, what appeared to be at least a partial refu-tation of Newton’s Law of Gravity, at first, turned out to be the key to predicting theexistence of an unknown planet. A similar process later led to the discovery of Plutoas well.

1.2.4.6 The Scientific Method

Regardless of whether one deems falsifiability and testability, puzzle solving orforecasting the ultimate hallmark of scientific activity, there is nothing inherentlyincompatible about these different activities. In their less dogmatic forms, each seemsto capture some aspect of the scientific enterprise. Like the three blind men who, upontouching different parts of an elephant, concluded that they were confronted with verydifferent entities, these descriptions of science seem to convey distinct characteristicsof a greater whole. Which aspect one focuses on may simply be a function of what thequestion we wish to address is. Moreover, each would seem to fit comfortably intoan edifice that almost every high school student has heard of: the scientific method.

Isaac Newton described the scientific method as follows:

Scientific method refers to the body of techniques for investigating phenomena, acquir-ing new knowledge, or correcting and integrating previous knowledge. It is based ongathering observable, empirical and measurable evidence subject to specific principlesof reasoning [118].18

This definition touches upon many of the elements already discussed. With minorvariations, the scientific method is typically taught as containing the following steps:

• Start out with some background description and information about thephysical world based on prior observation and experience.

• Formulate a question about an aspect of the physical world.• Develop a hypothesis predicting an answer to the question.• Design an experiment to test the hypothesis.• Conduct the experiment to test the hypothesis.• Draw a conclusion about the hypothesis based upon the result.• Share methods and results so that others can examine and/or duplicate your

experiment.

The first and last steps of this process are sometimes left out. The first step simplyrecognizes part of the overall context (discussed above) within which our observation,measurement, or experiment takes place and constitutes part of our Universe of infor-mation concerning it. The last step is critical for several reasons, not the least of which


is establishing the independent repeatability of the results obtained. If others repeatthe experiment and obtain the same result, then this can be reported as the discoveryof a common/shared/objective regularity in the phenomena of interest. Such observedregularities form the basis of our models of how the physical Universe behaves.

Despite its oft-quoted precepts, however, this is not a prescription that scientistsare compelled to follow. These steps are really just a heuristic describing generalizedpatterns of activity seen when the work scientists engage in is observed. Scientists arenot bound by them. As explained by Nobel Laureate Percy Bridgman, to the scientist:

. . . the essence of the situation is that he is not consciously following any prescribedcourse of action, but feels complete freedom to utilize any method or device whateverwhich in the particular situation before him seems likely to yield the correct answer. Inhis attack on his specific problem he suffers no inhibitions of precedent or authority,but is completely free to adopt any course that his ingenuity is capable of suggesting tohim [19].19

Even with the freedom Bridgman describes, however, the norms are what has beendescribed. And it is by engaging in the activities giving rise to these norms that has ledto the great successes enjoyed by science. Although deviations from the norm maybe called for and even improve the scientific enterprise on occasion, the soundnessof new methods and/or approaches must be established before they are relied uponas being scientific.

1.2.4.7 Defining Terms, Concepts, and Phenomena

Often overlooked in discussions about science, but nonetheless critical to any scien-tific enterprise, is the complete and explicit definition of any term or concept utilizedas well as any phenomena studied. All professions have their own technical terminol-ogy that must be understood in order to engage in them. The law is a prime example.The ninth edition of Black’s Law Dictionary has over 45,000 terms defined, includ-ing words or phrases such as malum prohibitum, adverse possession, consideration,privity, and the like. These are tools of the lawyer’s trade just as is the technicalterminology relied upon by scientists in their distinct fields.

What is required in the scientific context goes beyond this, however. Remem-ber that whatever phenomena it is that we are studying does not come out andidentify itself for us. Before any meaningful conclusions can be drawn about a phe-nomenon of interest, we must rigorously define exactly what we consider to constitutethat phenomenon. This definition may be provisional, replaced as soon as a betterunderstanding of the phenomenon is acquired. But unless we can clearly and unam-biguously identify the phenomenon under investigation in the first place, to whatcan we attribute our results and the interpretations we ascribe to them? Moreover, towhat will others attribute them? The failure to carefully define the very subject of ourinquiry introduces a fundamental ambiguity that may prevent any sound conclusionfrom being reached.


1.2.4.8 Example: What Is an Analogue?

Consider a statute that prohibits a drug and any of its analogues. A substance is sent toa forensic chemistry lab for identification to see if it falls within this prohibition. Thelegislation clearly defines the prohibited drug by its chemical structure and makeupso that its identification by chemists is straightforward. The term analogue, on theother hand, is given its ordinary dictionary definition as being a substance that isstructurally similar to the one prohibited but differing slightly in composition.

To legislators and laypeople, this definition may seem quite clear and easilyapplied. To the chemists charged with determining whether a given substance consti-tutes an analogue, however, it is a different story. The phrases “structurally similar”and “differing slightly” are not scientific terms and do not tell the chemist what cri-teria they encompass, leaving the scientific enquiry to be engaged in under-defined.Although science yields the physical makeup of the substance in question, whether itconstitutes an analogue is not a scientific determination. It is simply a matter of opin-ion. Equally skilled and competent scientists might disagree on the meanings of thesephrases and hence about whether the substance in question constitutes an analogue.

Science requires strict and precise definitions. In particular, the properties, char-acteristics or criteria necessary to identify or distinguish the phenomena of interestmust be explicitly and specifically given. Failure to do so leads to ambiguity whichshort circuits the acquisition of scientific knowledge.

1.2.5 SPECIFIC PRINCIPLES OF REASONING: THE INFERENTIAL PROCESS

Returning to Newton’s description of the scientific method for a moment, we seethat before any knowledge can be gained, the information obtained must be “subjectto specific principles of reasoning.” Given the empirical nature of science and thatobservation, not human reason, is the final judge of truth, it might seem odd to somethat Newton would include such prominent mention of human reason in his briefdescription of the scientific method. After all, aren’t things such as principles andrules of reasoning the purview of philosophical logic rather than science?

Many naively believe that science, particularly that performed on a routine basis, islargely a mechanical process whereby a test, measurement or experiment is performedand the result provides a self-evident answer to the question asked. As discussedabove, this is not true. A result is simply one piece of information to be considered.To be able to ascribe meaning to it, we must consider it in context with the universeof available information relevant to the observation, measurement, or experiment asa whole. This will include information about the procedure used, the instrumentsemployed, and the conditions under which the observations were made. But eventaken together, this is still just a lump of information. Without more, our result couldstill mean anything. What we need is some way of extrapolating from this informationto the conclusions that are supported by our result.

The tool that allows us to transform information into knowledge is reasoned infer-ence. Inference is the process of reasoning from a set of evidence to a conclusion


believed to be supported by that evidence. A set of rules defining what consti-tutes valid reasoning facilitates arriving at sound conclusions. If there are such rulesgoverning the process of scientific reasoning, though, where do they come from?

1.2.5.1 Rules of Inference

Reflecting back over the discussion thus far, we have already established a few ofwhat might be considered primitive inferential rules:

• The only kind of evidence/information that can help us understand physicalreality is that which has been empirically obtained.

• We cannot know what is or is not, only what is more or less likely based onthe information we have.

• Any belief/knowledge that is contradicted by observation is deemed to beless likely.

• Successful prediction of novel phenomena makes belief/knowledge it isbased on more likely.

• Duplication of empirical result makes correctness of result more likely.• Duplication of empirical result by independent source makes correctness of

result more likely than if it is only confirmed by original source.

These are not written down like some sort of checklist that a scientist must followwhen analyzing an empirical result. They are simply examples of the type of informalrules of inference scientists typically apply. And though in our age of science theymay seem simple and obvious, they were not always so considered. Plato, one of thegreat minds of ancient Greece, taught that empirical information could not be trusted.Instead, he argued that our Universe could not only be perfectly understood throughreason alone, but that, that was the only way one could understand it. Against thisbackdrop, even the simple rules listed above may prove quite powerful.

An inferential rule commonly employed is simplicity: if two descriptions describea phenomenon equally well, then the simpler description is favored. This is whatbiologist E.O. Wilson termed the principle of economy.

Scientists attempt to abstract the information into the form that is both simplest and aes-thetically most pleasing—the combination called elegance—while yielding the largestamount of information with the least amount of effort [164].20

For example, by the time of the sixteenth century the Ptolemaic model of the SolarSystem had become quite complicated, festooned with epicycle upon epicycle. Then,in 1543, Nicholas Copernicus published his heliocentric model which placed the Sunat the center of the Solar System with the planets, including Earth, orbiting aboutit and the moon orbiting about the Earth. Removing the Earth from the center ofthe Universe was a radical idea at the time, but this model predicted the motionsof the planets at least as well as Ptolemy’s had. And, although Copernicus maintainedthe idea of uniformly circular motion which still required epicycles to account for themotions of the planets, there were far fewer of these ad hoc encumbrances. As a


result, the new heliocentric model was far simpler than the Ptolemaic model it soonreplaced.

Our bag of inferential tools contains more than these simple heuristics, though.Every observation, measurement, and experiment takes place against a backgroundof accepted scientific laws and principles which provide a formal framework ofinferential rules to work with. Referring to physical laws and principles as rules ofinference likely seems odd to most. But recall that these are simply descriptions of theregularities and relationships between phenomena that we observe in nature. Andwell-established regularities and relationships happen to make excellent inferentialtools.

1.2.5.2 Example: Chemistry and Rules of Inference

Consider the principles of chemistry. Amongst other things, they describe the chem-ical changes that occur when different substances are mixed together. Thus, if we aregiven a substance and told that it is the product of a chemical reaction, we may subjectthis information to standard chemical principles, acting as our rules of inference, toinfer what the substances combined to form it were.

As an example, assume a solution containing sodium hydroxide, a common base,is poured into a beaker containing an unknown liquid. The solution in the beaker islater found to have evaporated but a small amount of a solid substance has been leftbehind. After examination, the substance is found to be a salt, sodium sulfate. A well-known principle of chemistry tells us that when an acid and a base react, the cation ofthe base and the anion of the acid combine to form a salt. Subjecting our informationto this principle allows us to infer that the unknown liquid may have been an acid. Thesame principles tell us that when mixed with sodium hydroxide, different acids reactto produce different salts. When the cation of sodium hydroxide combines with theanion of sulfuric acid, the resulting salt is sodium sulfate. Subjecting our informationto this principle allows us to infer that the unknown acid may be sulfuric acid. Theconcept of a physical principle acting as an inferential rule is that simple.

1.2.5.3 Hierarchy of Inferential Rules

As you might guess, not all inferential rules are on the same footing. For exam-ple, physical laws that are firmly established will be considered more trustworthyfor inferential purposes than those still in their developmental stages. This seemsobvious. But some laws and principles are favored because the manner in which theyare expressed bestows far greater inferential power. More to the point, mathematicaldescriptions/models are generally favored over qualitative ones. Good mathematicalmodels make explicit everything that is being considered and trim away inexactitudesthat may be introduced by qualitative concepts. Moreover, they permit precise pre-dictions to be made about what will be observed under a variety of circumstancesthat qualitative descriptions generally cannot. This not only makes such models moreopen to being tested but, once they have been confirmed, far more useful in applyingour knowledge for both intellectual and practical purposes. In fact, absent mathemat-ical models, most, if not all, of the scientific achievements discussed in the openingparagraph would not have been possible.


The strength of such models can be seen by recalling the example above con-cerning Newton’s Law of Gravity and the discovery of Neptune. First, the Newtonianmodel yielded quantitative predictions that permitted the discrepancies between it andthe orbit of Uranus to be easily and precisely determined. Feeding this informationback into the mathematical machinery of the model, scientists were then able to inferthat, if the description of gravity were correct, there must be another planet orbitingthe Sun beyond Uranus waiting to be discovered. And not only was the inferencecorrect, leading to the discovery of Neptune right where the model had predicted,but it reaffirmed the Newtonian description of gravity and, hence, the inferential rulerelied upon.

1.2.5.4 Creation and Destruction of Inferential Rules

This raises another point. What if the prediction made by Newton’s model concerningthe existence of a new planet had been incorrect? Just as with the scientific methoddiscussed above, none of our rules of inference is absolute. There may be times whentwo such rules are in conflict and one must be chosen over the other. Or there maybe instances when a completely novel phenomenon can only be understood by theintroduction of a new rule. There may even be times when what was once considereda physical law, and hence one of our more concrete inferential rules, must be discardedaltogether. Nothing within science is ever finally safe from the possibility of beingdiscarded due to future observations or better understanding and the inferential ruleswe rely upon are no different.

1.2.6 EPISTEMOLOGICAL ROBUSTNESS OF SCIENTIFIC CONCLUSIONS

Finally, we return to one of the primitive inferential heuristics listed above: We cannotknow what is or is not, only what is more or less likely based on the information wehave. This is absolutely fundamental to scientific reasoning. One of the remarkableaspects of science, however, is the ability of scientists to rigorously characterize therelative strength of their inferences in quantitative terms. This includes the ability todetermine the limitations associated with the type, amount, and/or quality of informa-tion obtained from an observation, measurement, or experiment and the conclusionsthat can be drawn from it. A mature science includes the tools that permit scientiststo quantify in an unambiguous manner how much confidence one can have in theirconclusions given the information and inferences they are based on. Or, if you rather,to unambiguously quantify the limitations of those conclusions.

The quantitative characterization of the degree of confidence one can have in aparticular conclusion is typically expressed as an estimate of the relative likelihoodof that conclusion compared to others that could be drawn. This provides a measureof the epistemological robustness of scientific inferences representing the degree towhich our scientific knowledge is believed to be incomplete. In essence, this allowsus to convey what conclusions our current information and the inferential techniquesemployed may support and how strongly. Absent such measures, one is left to guesshow justified a belief in a particular conclusion is.


The tools relied upon to determine the level of confidence one can have inthe conclusions arrived at include measures of uncertainty and error. For example,“[n]umerical data reported in a scientific paper include not just a single value (pointestimate) but also a range of plausible values (e.g., a confidence interval, or intervalof uncertainty)” accompanied by an estimate of their likelihood [28].21 This is doneto ensure that the conclusions drawn are actually supported by the results obtained.

1.2.6.1 Example: Error Analysis and the Discovery of Planetary Laws

One of the first scientists to understand the importance of determining these limita-tions was the sixteenth century Danish astronomer, Tycho Brahe [109].22 Althoughthe heliocentric model of Copernicus was an improvement over the old Ptolemaic sys-tem in terms of simplicity, its predictions were often not much better. Brahe sought todevelop an improved model based upon very accurate and precise measurements ofplanetary motions. To do so, he developed new instruments and techniques to mea-sure the positions of celestial bodies of interest. Just as significantly, he is also thefirst scientist known to have determined both the systematic and random errors asso-ciated with his measurements. Beginning in the late 1570s, he spent over 20 yearsmeasuring planetary positions far more accurately and precisely than had ever beendone before. Unfortunately, Brahe died in 1601 before he finished his work leavingthat task to his assistant Johannes Kepler.

Retaining the accepted notion that celestial bodies must engage in uniform circularmotion, Kepler went about the task of trying to fit a model of planetary motion aboutthe Sun to Brahe’s observations. Despite his greatest efforts, he could not get a modelto fit these observations with an error of better than 8 min of arc. The problem withthis is that Brahe’s observations were precise to within 2 min of arc, meaning that thediscrepancy between Brahe’s observations and Kepler’s models amounted to 6 minof arc.

Now, there are 360◦ in a circle and a minute of arc is 1/60 of one degree abouta circle (see Figure 1.2). That means that 6 min of arc constitute only 1/3600 of theangular rotation about a circle!

Nonetheless, according to Kepler, “[b]ecause these 8 minutes of arc could not beignored, they alone have led to a total reformation of astronomy.”23 What a merediscrepancy of 6 min of arc eventually led to were Kepler’s three laws of planetarymotion.

First Law: Planets orbit the Sun in elliptical orbits with the Sun at one focus.Second Law: An imaginary line from the Sun to an orbiting planet sweeps over

equal areas in equal time intervals.Third Law: The ratio of the squares of the orbital periods of two planets is equal

to the cubes of their semimajor axes

[P2

1P2

2= R3

1R3

2

].

It is now over 400 years later and these three laws are still relied upon! YetKepler would not have stumbled upon them unless Brahe had not only made quan-titative measurements, but had also mathematically characterized the limitations ofthe information he had obtained and, hence, the inferences it permitted.


100° 80°60°

40°

20°

0°

340°

320°

300°280°260°

240°

220°

200°

180°

160°

140°

120°

FIGURE 1.2 Circle = 360◦.

1.2.7 A WORKING DEFINITION OF SCIENCE

So then, what is science? Simply stated, science focuses on the quest for, and acquisi-tion of, knowledge of the physical world as we experience it through empirical means.It requires the systematic collection of empirical information followed by an assess-ment of the strengths and weaknesses of that information. Where the informationpermits, relationships are discovered and inferences are made creating knowledge inthe form of a description or model of what has been or will be observed. Finally,based on our information, the significance of that knowledge is evaluated through arigorous determination of its limits, that is, the degree of certainty associated with it.

Ontologically, although science must be empirically based, its true subject mat-ter is simply a specific type of experiential information and the relationships weinfer from it that constitute our descriptions of the physical universe. Althoughwe may have certain beliefs as to what lies behind our information and descrip-tions, there is no way to physically prove it. In a sense, our descriptions simplymodel information–inference networks that correspond to empirical experience. Saidanother way, our descriptions map the relationships/patterns perceived within ourcollection of empirically obtained information.

Epistemologically,∗ even strictly within the confines of information and descrip-tion, science still does not deal in absolutes. Seldom will our information be completeand perfect, and even it were, it is unclear how we would know. Our scientific knowl-edge and inferences are always somewhat fuzzy. That is, the information obtained,relationships perceived and the conclusions drawn from observation, measurement,

∗ Epistemology is the study of knowledge and justified belief. Its focus is the nature and dynamics ofknowledge: what knowledge is, how it is created, and what its limitations are.


TABLE 1.1Epistemological Framework of Science

Information- Prior knowledge

What we know/believe about the phenomena of interest prior to measurement,observation, experiment (preexisting observation, models, etc.)

- Empirical in natureObtained via measurement, observation, experiment

- Information inputThe information delivered to the measurement, observation,experiment (experimental set-up, instrument settings, etc.)

- Information outputInformation received from measurement, observation experimentresults, and other observations

Inference- Transformation of information into knowledge

This is an active process of knowledge creation- Rule-based reasoning constrains set of conclusions

Physical laws, falsifiability, predictive power, etc.

Knowledge- Consists of beliefs concerning conclusion(s) arrived at

Can never know whether conclusion(s) is true, can only believebased upon information and inferences

- Justified beliefDetermination of relative likelihood of conclusions supported providesmeasure of epistemological robustness of each and our knowledge as a whole

and experimentation are necessarily soft edged to some extent, always containing amodicum of uncertainty. Although our degree of belief in a given description or pieceof information may be high, science can never absolutely prove it.

Science, then, does not tell us what is or is not true. Rather, through the “scientificmethod,” it represents a structured process by which empirical information can becollected and processed to create knowledge, in the form of beliefs concerning thephysical universe, that can be justified in a quantitatively rigorous manner providinga measure of the epistemological robustness of the conclusions they support.

This leads to a working definition of science that we will rely upon throughoutthe rest of the text. It is consistent with everything discussed thus far and does awaywith resort to needless and unprovable assumptions concerning any relationship tothe fundamental nature of physical phenomena. Rather, it is based on the idea that


the methodology of science is simply an applied epistemological framework basedon empiricism that permits us to determine what conclusions about the physical Uni-verse are supported by the information collected and inferential rules relied upon (seeTable 1.1). It is expressed as follows to give us a concise definition to refer to as wego forward.

Scientific knowledge consists of descriptions/models of physical phenomenathat are inferred from experiential information obtained by empirical methods and,which, are inherently uncertain to a degree that can be quantitatively determined andexpressed.

Revealed Truth, absolute and known, is not the domain of science. Rather, it is relativeinference. From observation and information, to the relationships alive therein, to vary-ing degrees of certitude never complete. That’s the promise of science . . . and the best itcan do.

1.3 FORENSIC SCIENCE AND THE LAW

This is not a textbook about science generally, though. Rather, it is a textbook aboutforensic science. Unlike other sciences, forensic science is specifically developed forthe investigation of crimes and intended to be relied upon in legal proceedings todetermine guilt or innocence. As a result, it may be subject to legal constraints thatother sciences are free of. So, while everything discussed so far may be correct forscience generally, the question that arises is whether any of it applies to the forensicsciences in particular.

1.3.1 SCIENCE IN THE COURTROOM

Owing to the increasing reliance upon scientific evidence in the courtroom, juristshave also had to address the question of what constitutes science. The approach,however, is not one of identifying science for the sake of science, but to establish cri-teria for when evidence or testimony purported to be scientific will be accepted andadmitted as such. This is critical given the enormous persuasive power that evidenceclaimed to be scientific carries.24

The admissibility of scientific evidence in the Federal Courts is governed byFederal Evidentiary Rule 702. It states that25:

A witness who is qualified as an expert by knowledge, skill, experience, training, oreducation may testify in the form of an opinion or otherwise if:

a. The expert’s scientific, technical, or other specialized knowledge will help thetrier of fact to understand the evidence or to determine a fact in issue;

b. The testimony is based on sufficient facts or data;c. The testimony is the product of reliable principles and methods; andd. The expert has reliably applied the principles and methods to the facts of the

case.


The U.S. Supreme Court interpreted the Rule in Daubert to require that whenevidence is offered as being scientific in nature, the subject of the testimony elicitedmust in fact consist of “scientific ... knowledge.”26 It explained that:

. . . in order to qualify as “scientific knowledge,” an inference or assertion must bederived by the scientific method. Proposed testimony must be supported by appropri-ate validation-i.e., “good grounds,” based on what is known. In short, the requirementthat an expert’s testimony pertain to “scientific knowledge” establishes a standard ofevidentiary reliability.27

Whether evidence is scientific under the Rule hinges on a determination of whetherthe principles and methods leading to it are scientifically valid. This determination isbased upon consideration of several factors including28:

• Whether the principles and methods can be and have been tested;• Whether the principles and methods have been subject to peer review and

publication;• The known or potential rate of error of the methods employed;• The existence and maintenance of standards governing the method’s use; and• Whether the principles and methods are generally accepted within the

scientific community.

The last factor in the Daubert analysis, general acceptability, comes from the pre-vious standard enunciated 70 years earlier in Frye v. United States.29 Although onlyone component of the Daubert analysis, it still stands as the standard for admissibilityof scientific evidence in a minority of states. Nonetheless, even in those minor-ity states the principles enunciated in Daubert have begun to inform the analysis.Whether a majority or minority state, though, the factors considered are intended toensure that evidence claimed to be scientific is in fact “ground[ed] in the methodsand procedures of science” generally.30

1.3.2 FORENSIC SCIENCE AS SCIENCE

Now, the forensic sciences did not, by and large, sprout from the same academic envi-ronment and motivations as the “pure” sciences. Instead, they grew from the creativegenius of law enforcement and those assisting them in trying to solve crimes. Thus,they are rooted in a different culture with different goals. The goal is not to knowthe physical world better for its own sake. Instead, it is to understand and determinehow science can be focused on physical evidence associated with suspected criminalactivity so as to determine whether a crime has been committed and/or who commit-ted it. Although new discoveries may be made and new fields created, however, it isall strictly for the aforementioned purpose. Nonetheless, despite its distinct origins,forensic science is not much unlike many other applied sciences that share similarcharacteristics.

What needs to be understood, though, is that regardless of context, the physicalworld behaves as it does without concern for the purpose of our activities. Gravity


behaves the same whether we are in a physics, chemistry, biology, or forensics lab,or even at the scene of a crime. And so it is for all of nature’s forces and laws. Andforensic science is no more exempt from the principles discussed above than any otherscience. If we are going to engage in an activity that we want to be scientific in nature,then it must satisfy those characteristics which define science. Failure to do so doesnot mean that the activity is not useful or worthy of practice. It does, however, meanthat it is not science. The cases above concluded the exact same thing with respect towhat constitutes scientific evidence in the courtroom. And this applies directly to theforensic sciences.

1.4 METROLOGY: THE SCIENCE OF MEASUREMENT

Generally speaking, activities aimed at gathering empirical information can begrouped into two categories based upon the type of information sought. An observa-tion is meant to collect qualitative information concerning an entity or phenomenon.Examples of qualitative activities include the identification of a substance throughchemical means or the biological classification of a newly discovered animal. The aimof measurement, on the other hand, is to determine the numerical value attributableto some property of a physical entity or phenomenon. These include determiningthe temperature of a substance utilizing a thermometer or the weight of an animalutilizing a scale. The focus of this text is measurement.

1.4.1 MEASUREMENT

Reliance upon measurement goes back to at least 3000 B.C. when the Egyptiansemployed it in the construction of the pyramids. And its importance as a tool inmodern society is hard to overstate:

Millions of analytical measurements are made every day in thousands of laboratoriesaround the world. There are innumerable reasons for making these measurements, forexample: as a way of valuing goods for trade purposes; supporting healthcare; checkingthe quality of drinking water; analyzing the elemental composition of an alloy to confirmits suitability for use in aircraft construction; forensic analysis of body fluids in criminalinvestigations. Virtually every aspect of society is supported in some way by analyticalmeasurement [53].31

Some consider measurement a necessary aspect of science. William Thompson,later known as Lord Kelvin and after whom the absolute temperature scale is named,summed it up this way:

In physical science, the first essential step in the direction of learning any subject isto find principles of numerical reckoning and practicable methods for measuring somequality connected with it. I often say that when you can measure what you are speakingabout, and express it in numbers, you know something about it; but when you cannotmeasure it, when you cannot express it in numbers, your knowledge is of a meager andunsatisfactory kind; it may be the beginning of knowledge, but you have scarcely in yourthoughts advanced to the state of science, whatever the matter may be.32


Measurement is favored over observation because it tends to generate resultswith higher and more structured information content. One advantage of measure-ment, as explained by biologist E.O. Wilson, is that “[i]f something can be properlymeasured, using universally accepted scales, generalizations about it are renderedunambiguous” [164].33

Consider the phases of water. Even absent some way to measure the temperatureof water, we can determine that when water gets cold enough it freezes, when it getshot enough it turns into steam and in between it exists as a liquid. Although this maybe useful to some extent, the information contained by the generalities “cold enough”and “hot enough” is quite limited. In fact, “cold enough” and “hot enough” may meandifferent things in different contexts. For instance, depending upon the ambient pres-sure, cold will have to be a little colder in some contexts to produce ice and hot hotterto produce steam. If we can measure the temperatures and corresponding pressuresat which these phase shifts occur, though, not only can we clearly communicate what“cold enough” and “hot enough” mean in these different environments, but we canbuild quantitative models that clearly describe these phenomena.

1.4.2 COMPONENTS OF MEASUREMENT

There are several components that must be considered whenever performing or evalu-ating a measurement. Absent attention to each, the reliability of any conclusions basedon a measured result is drawn into question. We will consider each briefly here.

1.4.2.1 The Quantity Intended to Be Measured

Measurement is the experimental process whereby one seeks to determine the quan-titative value attributable to some physical or phenomenological quantity. Commonquantities include things such as length, time, and weight. The quantity of interest,that is, “the quantity intended to be measured,” is referred to as the measurand.34

Thus, if we wish to determine how heavy an object is by measuring it, the weightof the object is the measurand. Before any measurement can be performed, themeasurand must be clearly identified.

1.4.2.2 An Exercise in Comparison

All measurement boils down to comparison. First, we identify the quantity of interest,our measurand. Then we choose a physical reference that is the same kind of quantityas our measurand but whose value is already known. The two quantities are thencompared and the measurand’s value is given in terms relative to the value of thereference used.

For example, if our measurand is the length of a steel rod, then the physical com-parator might be a ruler of known length. To measure the length of the steel rod, wesimply lay it side by side to our ruler and compare the two objects. The length of ourrod can then be reported as the multiple of rulers that can be laid end to end withoutexceeding the rod’s length. Consider, for example, Figure 1.3. There, we can reportthe rod’s length as 2.4 rulers.


Ruler1 1 0.4+ +

Ruler Ruler

FIGURE 1.3 Measuring length with ruler.

1.4.2.3 Universally Accepted Scales

A problem occurs, however, if we try to report the result of our measurement to some-one unfamiliar with our ruler. How long is 2.4 rulers? It depends on the length of therulers being used. These could be 12 inch rulers, meter sticks, or something else alto-gether. What if someone else is provided the same rod, and comes up with a value of4.8 rulers (see Figure 1.4)? Who is wrong?

The answer is that neither is necessarily wrong and, in fact, both may be right.The real question is, are they using the same ruler? If Lab B is utilizing a ruler thatis half the length of the ruler being utilized by Lab A, then the reported results arein complete agreement despite the apparent discrepancy. Therein lays the importanceplaced on the use of “universally accepted scales” indicated by E.O. Wilson above.By relying upon universally accepted scales, in this case a shared ruler, statementsabout the rod’s length are rendered unambiguous. That is, everybody can understandprecisely what the reported measurement refers to and means. This highlights the roleof a system of uniform weights and measures in the measurement process.

1.4.2.4 How to Measure

Even within a system of common weights and measures, there are many differentmethods of measuring. Some are as simple as using a common ruler while othersare as sophisticated as relying upon the oscillatory frequency of a cesium atom. Forexample, if we are trying to measure our weight in the morning, then the measurementprocess we engage in is weighing and we will likely utilize a typical bathroom scale.The sophistication of the measurement process will always depend upon the use towhich the measurement will be put. No matter how simple or sophisticated, though,to be scientifically acceptable it must be demonstrated that the method chosen canactually measure what it is being utilized to measure. The process of determining

Lab A

2.4rulers

4.8rulers

Lab B

FIGURE 1.4 Who is wrong?


whether a particular method is capable of measuring what it is intended to and whatits limitations in doing so are is known as method validation.

1.4.2.5 Performing the Measurement

Even a well-identified measurand, established system of weight and measures andvalidated measurement method will do little good if the measurement is poorly exe-cuted. To many this is obvious and so much focus is placed on the actual performanceof the measurement itself. Remember, though, that the goal of our measurement is tocollect information from which a reliable inference can be made. And every measure-ment takes place within a physical context that includes the procedures, instruments,and conditions attendant to it and which may impact the result in a multitude of ways.Reliable inference requires as complete a set of information as is possible concern-ing the whole context. Accordingly, the measurement system must be thought of asincluding all of these aspects and, to be properly executed, careful attention must bepaid to each.

This means that it is not enough to simply carry out the measurement itself cor-rectly. Any measuring instruments utilized must be properly maintained and preparedfor use. This would include proper calibration of instruments. Care must be takento account for any factors that may impact our result. For example, environmentalfactors such as temperature, pressure or even, in some cases, vibrations caused bypassing traffic.∗ Attention must be given to the entirety of the physical context withinwhich a measurement takes place. This requires a lab to engage in Good MeasurementPractices.

1.4.2.6 Conclusions Supported

A measured result can be properly interpreted only in the context of the entiremeasurement process, including each of the components discussed so far. Absentconsideration of the process as a whole, a result is little more than a number whosemeaning is vague at best. Even when taken as a whole, though, we can still neverknow what the true quantity value of a measurand is. The information obtained frommeasurement does not lead to inferences that are absolutely certain. The best onecan do is to determine what values are more or less likely based on the informationwe have.

But it is not the measurand’s true quantity value that is fuzzy. It is the inferencesthat we can make about it based on the information we have that are fuzzy. Con-sider, for example, weighing yourself with a bathroom scale. Everybody knows thatbathroom scales are not perfect. When you step on a scale in the morning beforeheading off to work, the result reported by it may overstate, understate, or pinpointyour weight. But you simply cannot know which of the three alternatives describesthe value reported by your scale. Thus, there is some doubt concerning the valuereported. You definitely have a well-defined weight, though, so that is not in doubt.

∗ Vibrations caused by traffic can be problematic when using instruments such as atomic force micro-scopes.


What is in doubt is the value you infer for your weight based on the result of yourmeasurement.

Fortunately, there are well-developed inferential rules for measurement that delin-eate the bounds of the conclusions a measurement supports. Referred to collectivelyas Measurement uncertainty, these inferential tools permit us to explicitly delimitthe boundaries of the fuzziness associated with our conclusions and unambiguouslyexpress how confident we are about them. In other words, uncertainty provides themeasure of epistemological robustness of the conclusions supported by a measuredresult. This feature of measurement greatly enhances its value as a tool for buildingknowledge. Measurement interpretation is the final step in the measurement process.

1.4.2.7 Information and Inference

Like the rest of science, measurement is an exercise in information and inference. Infact, throughout the rest of the text, measurement will be treated as an information–inference device. Information is obtained through and about the measurement processand from that information we can make an inference concerning the quantity valueof a measurand. As with any other inferential process, the conclusions we draw areno better than the information and inferences they are based upon. The disciplineproviding the tools needed to ensure quality information and sound inferential devicesis Metrology.

1.4.3 METROLOGY

Metrology is the “[s]cience of measurement and its application.”35 Deriving from theGreek metrologia, meaning theory of ratios, and metron, meaning measure, the word“metrology” was first recognized in the English language in the nineteenth century.Nonetheless, its roots as a science go as far back as formal measurement itself. Itincludes “all theoretical and practical aspects of measurement,” regardless of fieldor application, thereby providing the epistemological basis for both performing andunderstanding all measurements.36 As such, the fundamental principles of metrol-ogy provide a common vocabulary and framework by which one can analyze anymeasurement, whether for scientific, industrial, commercial, or other purposes. Andwhether realized or not, every measurement everywhere in the world is dependentupon these principles for scientific validity. Put simply, “if science is measurement,then without metrology there can be no science.”37∗

It is now recognized that metrology provides a fundamental basis not only for the phys-ical sciences and engineering, but also for chemistry, the biological sciences and relatedareas such as the environment, medicine, agriculture and food.38

Given the role that science and technology play in the world, the importanceof metrology is recognized by all technologically advanced nations. Its principles

∗ The authors do not subscribe to the view that qualitative observation cannot form the basis for scientificinvestigation. It is relatively uncontroversial to note, however, that when relevant and feasible, quanti-tative measurement provides higher content and more useful information. Thus, in any event, withoutmetrology, science would be far less advanced and accomplished.


provide the framework for technological aspects of international trade agreements,national and international laboratory accreditation requirements and formal scien-tific and industrial standards. In fact, most nations have a national metrology institutewhich provides the basis for competent measurement practices within its borders aswell as coordinating with those of other nations to establish a body of internationallyaccepted measurement practices. Quite literally, “[m]etrology has become a necessityfor trade, technical cooperation, scientific comparison and even simple exchange ofinformation” [136].39

Moving forward, we will break our discussion of metrology down into five majorcomponents: the measurand, weights and measures, method validation, good mea-surement practices, and result interpretation/measurement uncertainty. Our model ofmeasurement will be that of an information–inference device where metrology sup-plies the rules for generating information and making inferences from it. We willconnect each metrological concept to an aspect of forensic measurement and its usein the courtroom as we go. By doing so, it is hoped that the understanding necessaryto apply metrology in the context of the justice system will come more naturally.

1.4.3.1 Who Is a “Metrologist”?

According to the U.S. Department of Labor, a metrologist:

Develops and evaluates calibration systems that measure characteristics of objects,substances, or phenomena, such as length, mass, time, temperature, electric current,luminous intensity, and derived units of physical or chemical measure: Identifies mag-nitude of error sources contributing to uncertainty of results to determine reliability ofmeasurement process in quantitative terms. Redesigns or adjusts measurement capabil-ity to minimize errors. Develops calibration methods and techniques based on principlesof measurement science, technical analysis of measurement problems, and accuracyand precision requirements. Directs engineering, quality, and laboratory personnel indesign, manufacture, evaluation, and calibration of measurement standards, instruments,and test systems to ensure selection of approved instrumentation. Advises others onmethods of resolving measurement problems and exchanges information with othermetrology personnel through participation in government and industrial standardizationcommittees and professional societies.40

A metrologist may be a theoretician, experimentalist, or applied scientist, andeither be a generalist or specialize in any particular type of, or field utilizing,measurement.

1.4.3.2 Forensic Metrology

Forensic metrology is the application of metrology and measurement to the investi-gation and prosecution of crime.

It is practiced within the laboratories of law enforcement agencies throughout the world.Worldwide activities in forensic metrology are coordinated by Interpol (Internationalpolice; the international agency that coordinates the police activities of the membernations). Within the U.S., the federal Bureau of Investigation (FBI), an agency of the


Department of Justice, is the focal point for most U.S. forensic metrology activities[43].41

Forensic measurements are relied upon in determining breath and blood alcoholand/or drug concentrations, weighing drugs, performing accident reconstruction, andfor many other applications.

1.5 WHY FORENSIC METROLOGY FOR JUDGES, LAWYERS,AND SCIENTISTS?

The principles of metrology are fundamental and provide a tool for the performanceand critical evaluation of all measurements.

Forensic metrology provides forensic scientists a roadmap for how to develop,perform, and properly interpret scientifically sound measurements. Moreover, in thecourtroom it can facilitate the communication of results in a scientifically sound man-ner so that the conclusions that fact-finders arrive at based on evidence obtained byforensic measurements are actually supported by those results. This can go a longway in ensuring that when forensic work product and testimony are relied upon, con-fidence can be had in the integrity of our system of justice and the verdicts that issuefrom it.

Armed with a basic understanding of forensic metrology, even a nonscientistlawyer or judge can engage in critical analysis of forensic measurements across abroad spectrum without having to develop a separate expertise in each. It can enablelegal professionals to: better understand evidence from forensic measurements; betterprepare and present cases that involve such evidence; and give lawyer and judge alikethe ability to recognize poor measurement practices and play their respective roles inpreventing bad science from undermining the search for truth in the courtroom.

A basic understanding of forensic metrology will improve the practices of bothlegal and forensic professionals, help ensure the integrity of the legal system, its fact-finding functions and the doing of justice in the courtroom, and facilitate creation oflegal foundations that are sound in science as well as the law.

ENDNOTES

1. L. Peterson and Anna S. Leggett, The evolution of forensic science: Progress amid the pitfalls, 36Stetson Law Rev. 621, 660, 2007.

2. State v. O’Key, 899 P.2d 663, n.21 (Or. 1995).3. A Trial of Witches at Bury St. Edmonds, 6 Howell’s State Trials 687, 697 (1665).4. Nat’l Research Council, Nat’l Academy of Sciences, Strengthening Forensic Science in the United

States: A Path Forward, 87, 2009.5. Id. at 110.6. Rod Gullberg, Estimating the measurement uncertainty in forensic breath-alcohol analysis, 11

Accred. Qual. Assur. 562, 563, 2006.7. Bert Black, Evolving legal standards for the admissibility of scientific evidence, 239 Science 1508,

1512, 1988.8. Justice Stephen Breyer, Introduction to Nat’l Research Council, Nat’l Academy of Sciences,

Reference Manual on Scientific Evidence 1, 9, 3rd ed. 2011 (emphasis added).


9. Albert Einstein, Science and religion, in Science, Philosophy and Religion, A Symposium, The Con-ference on Science, Philosophy and Religion in Their Relation to the Democratic Way of Life, Inc.,New York, 1941. See also, Walter Isaacson, Einstein 390 (Simon & Schuster 2007).

10. Esther Salaman, A talk with Einstein, 54 The Listener, 370–371, 1955.11. Richard Feynman, The Character of Physical Law 129, 1965.12. John von Neumann, Method in the Physical Sciences, in The Unity of Knowledge (L. Leary ed.

1955), reprinted in The Neumann Compendium 628 (F. Brody and T. Vamos eds. 2000).13. Richard Feynman, The Meaning of it All 15, 1998.14. Karl Popper, Conjectures and Refutations 33–39, 1963, reprinted in Philosophy of Science 3-10

(Martin Curd & J.A. Cover eds. 1998).15. Richard Feynman, The Character of Physical Law, 1965.16. Thomas Kuhn, Logic of discovery or psychology of research?, in Criticism and the Growth of Knowl-

edge 4-10 (Imre Lakatos & Alan Musgrave eds. 1970), reprinted in Philosophy of Science 11–19(Martin Curd & J.A. Cover eds. 1998).

17. Imre Lakatos, Science and pseudoscience, in Philosophical Papers vol. 1, 1–7 (1977) reprinted inPhilosophy of Science 20–26 (Martin Curd & J.A. Cover eds. 1998).

18. Sir Isaac Newton, Philosophiae Naturalis Principia Mathematica (1687) as quoted in Nat’l ResearchCouncil, Nat’l Academy of Sciences, Strengthening Forensic Science in the United States: A PathForward, 111, 2009.

19. Percy W. Bridgman, On scientific method, in Reflections of a Physicist, 1955.20. Edward O. Wilson, Scientists, Scholars, Knaves and Fools, 86(1) American Scientist 6, 1998.21. Nat’l Research Council, Nat’l Academy of Sciences, Strengthening Forensic Science in the United

States: A Path Forward, 116, 2009.22. For a fuller account of the story which follows see, Malcolm Longair, Theoretical Concepts in

Physics, 21–32 (2nd ed. 2003).23. Malcolm Longair, Theoretical Concepts in Physics, 27 (2nd ed. 2003).24. It is well recognized by jurists that “an aura of scientific infallibility may shroud the evidence and

thus lead the jury to accept it without critical scrutiny” [65]. Paul Giannelli, The Admissibility ofNovel Scientific Evidence: Frye v. United States, a Half-Century Later, 80 Colum. L. Rev. 1197,1237 (1980); U.S. v. Addison, 498 F.2d 741, 744 (D.C. Cir. 1974); Reese v. Stroh, 874 P.2d 200, 205(Wash. App. 1994); State v. Brown, 687 P.2d 751, 773 (Or. 1984); State v. Aman, 95 P.3d 244, 249(Or. App. 2004).

25. Fed. R. Evid. 702.26. Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579, 589–590 (1993).27. Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579, 590 (1993). “Fairness to a litigant

would seem to require that before the results of a scientific process can be used against him, he isentitled to a scientific judgment on the reliability of that process.” Reed v. State, 391 A.2d 364, 370(Md. 1978).

28. Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579, 593–594 (1993).29. Frye v. United States, 293 F. 1013 (1923).30. Reese v. Stroh, 874 P.2d 200, 206 (1994); Chapman v. Maytag Corp., 297 F.3d 682, 688 (7th

Cir. 2002) (“A very significant Daubert factor is whether the proffered scientific theory has beensubjected to the scientific method”); State v. Brown, 687 P.2d 751, 754 (Or. 1984) (“The term‘scientific’. . . refers to evidence that draws its convincing force from some principle of science,mathematics and the like.”).

31. Eurachem, The Fitness for Purpose of Analytical Methods: A Laboratory Guide to Method Valida-tion and Related Topics § 4.1, 1998.

32. William Thomson (later Lord Kelvin), Electrical Units of Measurement, Lecture to the Institutionof Civil Engineers, London, May 3, 1883.

33. Edward O. Wilson, Scientists, Scholars, Knaves and Fools, 86(1) American Scientist 6, 1998.34. Joint Committee for Guides in Metrology, International Bureau of Weights and Measures, Interna-

tional Vocabulary of Metrology—Basic and General Concepts and Associated Terms (VIM), § 2.3,2008.


35. Joint Committee for Guides in Metrology, International Bureau of Weights and Measures, Interna-tional Vocabulary of Metrology—Basic and General Concepts and Associated Terms (VIM), § 2.2,2008.

36. Id.37. William Thomson, (later Lord Kelvin), Electrical Units of Measurement, Lecture to the Institution

of Civil Engineers, London, May 3, 1883.38. Terry Quinn, Director BIPM, Open letter concerning the growing importance of metrology and

the benefits of participation in the Meter Convention, notably the CIPM MRA, August 2003 at<http://www.bipm.org/utils/fr/pdf/importance.pdf>.

39. Dilip Shah, Metrology: We use it every day, Quality Progress, Nov. 2005 at 86, 87.40. U.S. Dept. of Labor, Dictionary of Occupational Titles 012.067-010.41. DeWayne Sharp, Measurement standards, in Measurement, Instrumentation, and Sensors Handbook

5–4, 1999.

crc press - c.ymcdn.comc.ymcdn.com/sites/ press is an imprint of taylor & francis group, an...

Documents