"This book presents the technology evaluation methodology from the point of view of radiological physics and contrasts the purely physical evaluation of image quality with the determination of diagnostic outcome through the study of observer performance. The reader is taken through the arguments with concrete examples illustrated by code in R, an open source statistical language."
- from the Foreword by Prof. Harold L. Kundel, Department of Radiology, Perelman School of Medicine, University of Pennsylvania

"This book will benefit individuals interested in observer performance evaluations in diagnostic medical imaging and provide additional insights to those that have worked in the field for many years."
- Prof. Gary T. Barnes, Department of Radiology, University of Alabama at Birmingham

This book provides a complete introductory overview of this growing field and its applications in medical imaging, utilizing worked examples and exercises to demystify statistics for readers of any background. It includes a tutorial on the use of the open source, widely used R software, as well as basic statistical background, before addressing localization tasks common in medical imaging. The coverage includes a discussion of study design basics and the use of the techniques in imaging system optimization, memory effects in clinical interpretations, predictions of clinical task performance, alternatives to ROC analysis, and non-medical applications.

Dev P. Chakraborty, PhD, is a clinical diagnostic imaging physicist, certified by the American Board of Radiology in Diagnostic Radiological Physics and Medical Nuclear Physics. He has held faculty positions at the University of Alabama at Birmingham, University of Pennsylvania, and most recently at the University of Pittsburgh.

Dev P. Chakraborty received his PhD in physics in 1977 from the University of Rochester, NY. Following postdoctoral fellowships at the University of Pennsylvania (UPENN) and the University of Alabama at Birmingham (UAB), since 1982 he has worked as a clinical diagnostic imaging physicist. He is American Board of Radiology certified in Diagnostic Radiological Physics and Medical Nuclear Physics (1987). He has held faculty positions at UAB (1982 - 1988), UPENN (1988-2002) and the University of Pittsburgh (2002-2016). At UPENN he supervised hospital imaging equipment quality control, resident physics instruction and conducted independent research. He is an author on 78 peer-reviewed publications, the majority of which are first-authored. He has received research funding from the Whittaker Foundation, the Office of Women's Health, the FDA, the DOD, and has served as principal investigator on several NIH RO1 grants.

Preliminaries

Introduction

Clinical tasks

Imaging device development and its clinical deployment

Image quality vs. task performance

Why physical measures of image quality are not enough

Model observers

Measuring observer performance: four paradigms

Hierarchy of assessment methods

Overview of the book and how to use it

PART I

Introduction

Decision vs. truth: the fundamental 2x2 table of ROC analysis

Sensitivity and specificity

Reasons for the names sensitivity and specificity

Estimating sensitivity and specificity

Disease prevalence

Accuracy

Positive and negative predictive values

Example: calculation of PPV and NPV

PPV and NPV are irrelevant to laboratory tasks

Introduction

Decision variable and decision threshold

Changing the decision threshold: Example I

Changing the decision threshold: Example II

The equal-variance binormal model

The normal distribution

Demonstration of the concepts of sensitivity and specificity

Inverse variation of sensitivity and specificity

The ROC curve

Assigning confidence intervals to an operating point

Variability in sensitivity and specificity: the Beam et al study

Introduction

The ROC counts table

Operating points from counts table

Relation between ratings paradigm and the binary task

Ratings are not numerical values

A single "clinical" operating point from ratings data

The forced choice paradigm

Observer performance studies as laboratory simulations of clinical tasks

Discrete vs. continuous ratings: the Miller study

The BIRADS ratings scale and ROC studies

The controversy

Introduction

The empirical ROC plot

Empirical operating points from ratings data

AUC under the empirical ROC plot

The Wilcoxon statistic

Bamber's theorem

The importance of Bamber's theorem

Appendix 5.A: Details of Wilcoxon theorem

Introduction

The binormal model

Least-squares estimation

Maximum likelihood estimation (MLE)

Expression for area under ROC curve

Appendix 6.A: Expressions for partial and full area under the binormal ROC

Introduction

Three sources of variability

Dependence of AUC on the case sample

Estimating case-sampling variability using the DeLong method

Estimating case-sampling variability of AUC using the bootstrap method

Estimating case-sampling variability of AUC using the jackknife method

Estimating case-sampling variability of AUC using a calibrated simulator

Dependence of AUC on the reader's expertise

Dependence of AUC on the modality

Effect on empirical AUC of variations in thresholds and numbers of bins

Empirical vs. fitted AUCs

PART II

Introduction

Hypothesis testing for a single-modality single-reader ROC study

Type-I errors

One-sided vs. two sided tests

Statistical power

Some comments on the code

Why is alpha chosen to be 5%

Introduction

Random and fixed factors

Reader and case populations and data correlations

Three types of analyses

General approach

The Dorfman-Berbaum-Metz (DBM) method

Random-reader random-case (RRRC) analysis

Fixed-reader random-case (FRRC) analysis

Random-reader fixed-case (RRFC) analysis

DBMH Analysis: Example 1

DBMH Analysis: Example 2

Validation of DBMH analysis

Meaning of pseudovalues

Introduction

The single reader multiple treatment model

The multiple reader multiple treatment model ORH model

Special cases: fixed-reader and fixed-case analyses

Example of ORH analysis

Comparison of ORH and DBMH methods

Introduction

Statistical power

Sample size estimation

Dependence of statistical power on estimates of model parameters

Formulae for random-reader random-case (RRRC) sample size estimation

Formulae for fixed-reader random-case (FRRC) sample size estimation

Formulae for random-reader fixed-case (RRFC) sample size estimation

Example 1

Example 2

Details of the sample size estimation process

Cautionary notes: the Kraemer et al paper

Prediction…