receiver operator characteristics
DESCRIPTION
Receiver Operator Characteristics. What is it and where does it come from Statistical aspects Use of ROC. Early radar signals. Is this an enemy plane?. Signal Noise Ratio. The problems of decision. Sound the alarm when the signal is very small Advantages - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Receiver Operator Characteristics](https://reader034.vdocuments.us/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/1.jpg)
Receiver Operator Characteristics
• What is it and where does it come from
• Statistical aspects
• Use of ROC
![Page 2: Receiver Operator Characteristics](https://reader034.vdocuments.us/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/2.jpg)
Early radar signals
Is this an enemy plane?
Signal Noise Ratio
![Page 3: Receiver Operator Characteristics](https://reader034.vdocuments.us/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/3.jpg)
The problems of decision• Sound the alarm when the signal is very small
– Advantages• Plenty of time to get the fighters off the ground• Reduce the number of bombers reaching the target
– Disadvantages• Lots of false alarms• Waste of gasoline, wear and tear on fighter planes• Exhaust fighter pilots
• Sound the alarm when the signal is unmistakable– Advantages
• No waste, no wear and tear, no exhaustion
– Disadvantages• More bombers get through, more bombs, more destructions
![Page 4: Receiver Operator Characteristics](https://reader034.vdocuments.us/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/4.jpg)
Solution to decisions
• Code Yellow
– Signal suggests possible incoming bomber
– Pilots get dressed, fighter planes get loaded with gasoline and ammunition
• Code Orange
– Signal suggests incoming bomber likely
– Fighter planes towed to runway, pilots goes to the planes
• Code Red
– Signal is unmistakable
– Fighter planes take off
![Page 5: Receiver Operator Characteristics](https://reader034.vdocuments.us/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/5.jpg)
Receiver Operator Characteristics
False alarms
Sen
sitiv
ity
Useless operatorSensitivity and false alarm rate changes together
Perfect operator100% Sensitive0% false alarms
Most operators
![Page 6: Receiver Operator Characteristics](https://reader034.vdocuments.us/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/6.jpg)
Receiver Operator Characteristics
False alarms
Sen
sitiv
ityCode Yellow
Code Orange
Code Red
Increasingsignal
strength
![Page 7: Receiver Operator Characteristics](https://reader034.vdocuments.us/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/7.jpg)
False alarms
Sen
sitiv
ityCode Yellow
Code Orange
Code Red
Increasingsignal
strength
Receiver Operator Characteristics
![Page 8: Receiver Operator Characteristics](https://reader034.vdocuments.us/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/8.jpg)
Receiver Operator Characteristics
• What is it and where does it come from
• Statistical aspects
• Use of ROC
![Page 9: Receiver Operator Characteristics](https://reader034.vdocuments.us/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/9.jpg)
![Page 10: Receiver Operator Characteristics](https://reader034.vdocuments.us/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/10.jpg)
![Page 11: Receiver Operator Characteristics](https://reader034.vdocuments.us/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/11.jpg)
![Page 12: Receiver Operator Characteristics](https://reader034.vdocuments.us/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/12.jpg)
![Page 13: Receiver Operator Characteristics](https://reader034.vdocuments.us/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/13.jpg)
![Page 14: Receiver Operator Characteristics](https://reader034.vdocuments.us/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/14.jpg)
![Page 15: Receiver Operator Characteristics](https://reader034.vdocuments.us/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/15.jpg)
![Page 16: Receiver Operator Characteristics](https://reader034.vdocuments.us/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/16.jpg)
Statistical ROC• A measurement is normally distributed in two groups,
those outcome negative and those outcome positive
• Using a cut off level to make a decision will create a number of TP, FN, FP, and TN. From these Sensitivity and Specificity is calculated
• If the cut off value changes
– TP,FN,FP,TN changes
– Sensitivity and Specificity changes
• The relationship between Sensitivity and Specificity over the range of the measurement defines the ROC
![Page 17: Receiver Operator Characteristics](https://reader034.vdocuments.us/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/17.jpg)
Statistical ROC
![Page 18: Receiver Operator Characteristics](https://reader034.vdocuments.us/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/18.jpg)
![Page 19: Receiver Operator Characteristics](https://reader034.vdocuments.us/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/19.jpg)
![Page 20: Receiver Operator Characteristics](https://reader034.vdocuments.us/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/20.jpg)
![Page 21: Receiver Operator Characteristics](https://reader034.vdocuments.us/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/21.jpg)
Receiver Operator Characteristics
• What is it and where does it come from
• Statistical aspects
• Use of ROC
![Page 22: Receiver Operator Characteristics](https://reader034.vdocuments.us/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/22.jpg)
Advantages of using ROC
• It defines the quality of a test or prediction using a measurement without specifying a cut off value for decision making
• Assuming Normal distribution– The mean and Standard Error can be estimated
– The 95% CI can be estimated
– Statistical significance can be determined
– Whether one test is better than another can be determined
![Page 23: Receiver Operator Characteristics](https://reader034.vdocuments.us/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/23.jpg)
23
چهار ترکیب ممکن برای تصمیم و خطا
درص�د افراد غی�ر مبتال که مدل :FPمثب�ت نادرس�ت•داده تشخیص مبتال را آنه�ا اشتباه ب�ه تص�میم
است.نادرس�ت• :FNمنف�ی� مدل که مبتال افراد درص�د
تص�میم ب�ه اشتباه آنه�ا را غیرمبتال تشخیص داده .است
:TNمنف�ی درس�ت• مبتال که افرادی غی�ر در ص�د مدل تصمیم به درستی آن را تشخیص داده است.
درص�د افرادی ک�ه بیماری دارند :TPمثب�ت درس�ت•داده تشخیص را آ�ن درس�تی ب�ه تص�میم مدل و
است.
TP+FN=100%FP+TN=100%
هدف مدل تصمیم حداقل کردن
)FN وFP (خطاهاست
![Page 24: Receiver Operator Characteristics](https://reader034.vdocuments.us/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/24.jpg)
24
نکته
و FNدرص�د FP مستقل ص�ورت ب�ه شود حداقل نمی
افزای�ش م�ی یابد و FN ک�م شود FPاگ�ر برعکسمی ROCنمودار پیدا را بهین�ه مقدار
کند.
![Page 25: Receiver Operator Characteristics](https://reader034.vdocuments.us/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/25.jpg)
25
:مثال
است. افراد در باال خون فشار تشخی�ص هدف سه در افراد خون فشار کار ای�ن انجام برای
گیری می شود. گروه فرضی اندازه
اند. - جمعیت سالم که تحت مراقبت قرار گرفته1- جمعی�ت بیماران مشمول مراقب�ت اولی�ه ک�ه به 2
خاطر شکایات جزئی تحت بررسی هستند.بستری 3 قلب�ی بخ�ش در ک�ه بیماران�ی - جمعی�ت
هستند.
![Page 26: Receiver Operator Characteristics](https://reader034.vdocuments.us/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/26.jpg)
26
![Page 27: Receiver Operator Characteristics](https://reader034.vdocuments.us/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/27.jpg)
27
ROC رسم منحنی
![Page 28: Receiver Operator Characteristics](https://reader034.vdocuments.us/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/28.jpg)