Quattrone C

enter for the Fair Administration of Justice

Quattrone Centerfor the Fair Administration

of Justice

2014 Spring Symposium:A Systems Approach to

Conviction Integrity

On Friday and Saturday, April 4-5, 2014, a group of 200 thoughtful, passionate people interested in improving the criminal justice system assembled at Penn Law for the inaugural Spring Symposium of the Quattrone Center for the Fair Administration of Justice. The topic, “A Systems Approach to Conviction Integrity,” brought a group of internationally known experts who have dramatically improved the safety and reliability of complex human systems such as healthcare, aviation, and laboratory research together with a wide spectrum of progressive criminal justice practitioners. It was the first step on our mission to improve the criminal justice system via an interdisciplinary, all-stakeholders approach that prizes a ‘just culture’ throughout its various participating agencies and emphasizes data sharing, data analysis and learning from error.

The results were extraordinary. Criminal justice professionals reacted with enthusiasm to the principles of quality control and safety explained by the healthcare, aviation and lab safety experts. The converse was also true, as experts in other industries discovered with great excitement how important their expertise could be to achieving much-needed criminal justice reform.

Our two-day conversation was the beginning of a longer journey. Your participation, engagement, and thoughtful comments inspired all who attended, made the conversation broader and deeper, and launched the Quattrone Center with great forward momentum. We look forward to working with you and learning from you in the months and years to come.

With great admiration and appreciation,

John HollwayExecutive DirectorQuattrone Center for the Fair Administration of JusticeUniversity of Pennsylvania Law School

THANK YOU.

We have today, truly, a galaxy of experts across multiple fields to really think about one simple question, which is a commitment to improving the criminal justice system. - Michael Fitts, Dean and Bernard G. Segal Professor of Law, Penn Law

We are taking a step back from the adversarial system . . . and talking about how the entire system can function together for one goal: how to get the right guy in the right way. - John Hollway, Executive Director, Quattrone Center

Welcome and Introduction

In health care, aviation, and scientific laboratories, errors can literally kill people - sometimes lots of people. The systems and processes used in these other settings may offer us some guidance or lessons about improving the integrity of the criminal justice system.- Cary Coglianese, Penn Law (top, seated, right)

I am not going to say one size fits all. But I am going to say . . . what drives all of these activities is the common starting point. Everybody sees the need to improve the system.- Christopher Hart, National Transportation Safety Board(top, standing)

We are not going to improve unless we start counting.- Lee Fleisher, Perelman School of Medicine at the University of Pennsylvania (middle)

If you want to prevent errors, and most importantly, the harm they cause, you have to quit trying to change the people, and change the system.- Lucian Leape, Harvard University School of Public Health (bottom)

What is essential is that there be a feedback loop between the observation and the response to the information. If the feedback loop is broken, it is not a system, or at least it’s not a well functioning system and therefore cannot learn and cannot improve itself.- Susan Silbey, Massachusetts Institute of Technology(top, standing)

Bottom: John Hollway (Penn Law) taking picture of Lucian Leape (Harvard University School of Public Health) and Madeline deLone (Innocence Project)

Session 1: Quality Control: A Comparative Analysis

When organizations want to learn about events, the best approach is to simply ask those people who are involved. People are generally willing to share their knowledge if their identities remain protected, there is no disciplinary or legal consequence.- Linda Connell, NASA (top)

We know there’s some uncounted number of false confessions, of flawed, fake forensics, etc., but we don’t really know how many . . . We don’t have a very good way of measuring what’s a successful prosecutor’s office or a successful public defender’s office. - Stephanos Bibas, Penn Law (bottom, standing)

Session 2: Fostering A Culture of Disclosure in Criminal Justice

Jeffrey Deskovic (right) was wrongfully convicted of murder and rape in January 1991 despite a DNA test of the victim that did not match his DNA. Based on a more sophisticated DNA test done in 2006, he was exonerated on the grounds of actual innocence and freed.

It’s important to understand . . . that often what happens outside of the courtroom can be just as important as what happens inside the courtroom.- Jeffrey Deskovic, Exoneree and Director, Jeffrey Deskovic Foundation for Justice

We want to now transition lessons learned into lessons applied.- Thomas Hicks, International Association of Fire Chiefs (left page, seated, second from right)

Leadership helps prevent the tone of the [case review] from being too acrimonious, and although the presentation of each case is backward looking, the real intent is to help all participants prevent or mitigate similar events in the future.- Steven Raper, Perelman School of Medicine at the University of Pennsylvania (left page, seated, second from left)

Lunch Keynote Address: Enforcement and Safety: Finding a BalanceKeynote Speaker - The Honorable Christopher Hart, National Transportation Safety Board

“System think” is understanding how a change in one subsystem will affect the other subsystems. And the way we did it in aviation is collaboration . . . if people come in it with their own myopic, my stovepipe is fine, but yours isn’t, it ain’t going to work. - Christopher Hart, National Transportation Safety Board

We don’t want to believe as professionals, or as lawyers, that you can reduce what we do - our talent, our judgment, our experience - into a checklist. But the reality is if you don’t, you run the risk of making a mistake that can prove fatal: fatal to your case, fatal to your client.- Jeff Adachi, San Francisco Public Defender’s Office (top)

I went and shook his hand and apologized to him for being convicted for something he didn’t do. The next day it was on the front page of the newspaper, and not only the newspaper in Dallas, but The New York Times, The Washington Post . . . that an elected DA went down and apologized to someone for being wrongfully convicted.- Craig Watkins, Dallas County DA’s Office (bottom, standing)

Session 3: Defining and Maintaining Quality inCriminal Justice

If I said to you, where you live, can you tell me where the good public school is? You could. And, if I said to you, where is the good hospital, you could probably break it down by specialty. But if I said to you, okay, how does the criminal justice system work in San Francisco and how does it compare to Boston and Manhattan? You would probably have no idea.- Amy Bach, Measures For Justice (top)

Really, internal affairs is such a terrible term because what it says to the community is this is our affair, it’s none of your business.- Karen Amendola, The Police Foundation (bottom)

Francisco “Frankie” Carrillo Jr. was wrongfully convicted of murder in 1992 based on false identification testimony. He was exonerated and released in 2011 after nearly 20 years in prison.

These are people who are there to enforce the law. And when I hear you say that they can’t even enforce their own policies, it bothers me. Because it reminds me of what happened to me. - Frankie Carrillo, Exoneree (left page)

You need somebody to look in periodically and we need audits. And, not only do we need audits of labs, we need audits of what goes on in the courtroom.- Jules Epstein, Widener University School of Law (top, left)

Procedures that are used day in, day out in all 18,000 police departments across this country are often times inadequate. They’re rarely standardized, the training is somewhat limited, the oversight is problematic.- Dan Simon, USC Gould School of Law (top, right)

Our view of forensics, if we are going to be doing our jobs as well as we can, has to change. It has to evolve, because science is changing.- Michael Ambrosino, US Attorney’s Office for District of Columbia (middle)

To be overconfident about a system like our criminal justice is certainly a fault in itself . . . We should be skeptical. We should continue to ask for the right kind of data. And we should continue to think in the future, how do we measure - how do we measure the types of reforms that we are talking about, will they really work?- David Rudovsky, Penn Law (bottom)

Session 4: Errors in Criminal Justice - What We Know, and What We Don’t

Accountability is what we need.- Rick Walker, Exoneree (right page)

The more we learn about different categories of exonerations, the more that will teach us how to avoid them. - Samuel Gross, University of Michigan Law School (top)

Interrogation training in the U.S. is like teaching a doctor a surgical procedure without teaching them the possible hazards, side effects, or how to recognize and deal with them.- Jim Trainum, Mid-Atlantic Innocence Project (bottom, standing)

Quedillis Ricardo “Rick” Walker was wrongfully convicted of murder in December 1991. In 2003, new testimony from eyewitnesses and new DNA evidence led to his release and exoneration. He was incarcerated for 12 years.

If you really want to help us change, you need to be a part of the implementation plan as much as the drafting plan. The drafting is the easy part.- Francis Healy, Philadelphia Police Department (top, middle)

The whole idea of a double blind is that the person who is doing the lineup doesn’t know who the suspect is.- Phil Kohn, Clark County Public Defender’s Office, NV (bottom, standing)

Session 5: Challenges to Implementing Reform

We represent the victims, the police, the community. Who else do we represent? We represent the defendants. That’s because as prosecutors, we represent everybody.- Kevin Steele, Montgomery County DA’s Office (top)

How do we actually go about making change? It’s one thing to have evidence about a new policy or a new policy that’s suggestive of a different approach. Yet, we don’t really a good basis in science for understanding how to go about actually making effective change.- John MacDonald, Department of Criminology, University of Pennsylvania (middle, right)

Is there a strategic plan for the change? If there is not a plan, it’s just not going to succeed, absolutely not.- Rosemary DeMenno, International Association of Chiefs of Police Research Center (bottom, middle)

I’ve been told by people that they were instructed by the city lawyers to not say they were sorry. To not admit that there was a mistake. And maybe not even to admit that there had been a real wrongful conviction out of fear of the civil implications that would flow from that. As long as you have all of that going on, it’s going to be very, very hard to create a safe space.- Madeline deLone, The Innocence Project (top, left)

If the Sentinel Events review approach is viewed as simply another criminal, internal affairs or civil review, it’s not going to work. Because the people who are going to be involved in the review process are not going to feel safe . . . to be open. - Sean Smoot, Police Benevolent & Protective Association of Illinois (middle, middle)

The National Institute of Justice was able to engage with three different jurisdictions . . . These beta sites have self-identified “sentinel events” in their jurisdictions and have brought a team of stakeholders together to conduct a review. They will be planning and conducting that review through 2014 with some technical assistance from the National Institute of Justice.- Maureen McGough, National Institute of Justice (bottom)

Session 6: Identifying and Improving System Weaknesses in Criminal Justice

When these things go wrong, they are not single cause events, they are organizational accidents where individual mistakes combine with latent system weaknesses.- James Doyle, Law Offices of Bassil, Klovee & Budreau, P.C. (top)

You have to change the relationship between defenders and prosecutors early on in the system. Many of these mistakes . . . would not occur if there was an open sharing of information . . . early on in this process. - John Chisholm, Milwaukee County District Attorney’s Office, WI (bottom, left)

How do we do randomized experiments in the criminal justice world? How do we work with agencies to get that done? This is critical if we want new ideas and proof that they work to be available to people who work in this field.- David Abrams, Penn Law (top)

What the Quattrone Center really wants to do is try to generalize the knowledge that people are producing in their day-to-day life in a way that we can spread best practices . . . much more rapidly.- Anne Piehl, Rutgers University

Session 7: Successful Design of Rigorous Field Experiments in Criminal Justice

It’s been said many times, nobody here wants to convict an innocent person. And I think most of your exonerees will just say, duh! But there are bad actors on both sides of the line.- Michael Morton, Exoneree (next page, far left)

A strong research design usually . . . trumps a fancy technique.- James Anderson, Pardee RAND Graduate School (top, right)

We want to have studies that are done by people who really do have the scientific expertise to get them done right when you’re doing studies in the field.- Barry Scheck, The Innocence Project (bottom)

Michael Morton (above, left) was wrongfully convicted of murdering his wife in 1987. He spent 25 years in prison before exculpatory evidence improperly withheld by the prosecutor was revealed, and DNA testing proved he was not the murderer. In a highly unusual result, the prosecutor was incarcerated for his actions - for 10 days.

If we don’t measure, we don’t know. - Lucian Leape, Harvard University School of Public Health

The Quattrone Center thanks its generous benefactors, Frank Quattrone and Denise Foderaro.

SPEAKERS

David Abrams, Ph.D.University of Pennsylvania Law SchoolJeff AdachiSan Francisco Public Defender’s OfficeMichael AmbrosinoUS Attorney’s Office for the District of ColumbiaKaren Amendola, Ph.D.The Police FoundationJames M. AndersonPardee RAND Graduate SchoolAmy BachMeasures for JusticeStephanos BibasUniversity of Pennsylvania Law SchoolJohn ChisholmMilwaukee County District Attorney’s OfficeCary Coglianese, Ph.DUniversity of Pennsylvania Law SchoolLinda ConnellNASA Ames Research CenterMadeline deLoneThe Innocence ProjectRosemary DeMennoInternational Association of Chiefs of Police Research CenterJames M. DoyleOf Counsel, The Law Offices of Bassil, Klovee & Budreau, P.C.Jules EpsteinWidener University School of LawLee A. Fleisher, MDPerelman School of Medicine at the University of PennsylvaniaSamuel R. GrossThe University of Michigan Law SchoolChristopher HartNational Transportation Safety Board

 

Francis HealyPhiladelphia Police DepartmentE. Thomas Hicks IVInternational Association of Fire ChiefsJohn HollwayUniversity of Pennsylvania Law SchoolPhillip KohnClark County Nevada Public Defender’s OfficeLucian Leape, MDHarvard University School of Public HealthJohn MacDonald, Ph.D.University of PennsylvaniaMaureen McGoughNational Institute of JusticeAnne Piehl, Ph.D.Rutgers UniversitySteven E. Raper, MDPerelman School of Medicine at the University of PennsylvaniaDavid RudovskyUniversity of Pennsylvania Law SchoolBarry ScheckThe Innocence ProjectSusan Silbey, Ph.D.Massachusetts Institute of TechnologyDan SimonUniversity of Southern California Gould School of LawSean SmootPolice Benevolent & Protective Association of IllinoisKevin SteeleMontgomery County District Attorney's Office, PAJames TrainumMid-Atlantic Innocence ProjectCraig WatkinsDallas County District Attorney’s Office, TX