Miscellaneous
Perceptions of Paradox Illusions of Professional Skepticism Perceptions of Paradox: Illusions of Professional Skepticism When I saw Beverly Hall's picture, I was reminded of my favorite elementary school teacher, Mrs. Foreman. Hall was Jamaican- American and her smile, like Mrs. Foreman's, seemed extraordinarily kind. She was the superintendent of the Atlanta public schools and named the national superintendent of the year in February 2009 by the American Association of School Administrators. The honor had been earned by raising test scores, including those on standardized tests that had been designed to hold teachers accountable as part of the No Child Left Behind Act. By October the test scores were under scrutiny after being deemed statistically improbable. The truth was, there was cheating going on. A lot of cheating, but this time it wasn't by students. Erasure patterns were discovered on answer sheets that made it appear as though educators had corrected student's answers immediately after the testing. Students that had learning disabilities suddenly became proficient in math and reading, and students who were considered gifted went from exceptional to perfect. In the fall of 2010, fifty agents with the Georgia Bureau of Investigation started questioning teachers about the conspiracy to falsify test scores. They considered Hall to be the ringleader. An accusation she vehemently denied. She allegedly offered cash bonuses to teachers who could meet the minimum score targets. The culture in the schools became one of unrelenting pressure to raise scores, and the cheating had reportedly been an open secret for years. All in all, investigators identified 178 teachers and administrators that had fraudulently corrected student's answers. Hall was indicted in March of 2013 and on April 1, 2015, eleven teachers were convicted on racketeering charges and led from the courtroom in handcuffs in what had become one of the largest standardized test cheating scandals in U.S. history. As I was reading research into the Atlanta cheating case I felt a little twinge of pushback when I learned that Beverly Hall was a target in the investigation. What I'd imagine it would feel like if someone had suggested to me that Mrs. Foreman was a cheater. This is a tell-tale sign of implicit bias that can sabotage your reasoning. I speak for experts that make high-stakes decisions as a routine part of their jobs and when I talk with them about professional skepticism, they generally see it as the opposite of being gullible. This view may seem intuitive but comes with serious risks of misjudgment when making critical decisions. In the real world, many of our judgments force us to choose between competing alternatives, not to simply evaluate a single item. If you need to make a choice, simply doubting all of the factors isn't helpful. Also, consider that the way we apply doubt is biased, we apply it unevenly. Evidence that meets our assumptions generally gets a free pass, while evidence that challenges our preconceived beliefs receives more scrutiny. If your view of professional skepticism is simply setting a higher bar before you believe something, your judgment process is like a ticking time bomb. You're bound, at some point, to make a critical error and here's why. When we talk about skepticism, we almost always mean avoiding believing something is true when it isn't. This is a false positive or a type 1 error. We forget that a false negative, believing something is false when it's actually true, can be just as harmful. Effective skepticism should be a constant process. A screening process kind of like a signal to noise ratio that acts as a gauge for the relevancy of evidence and how we update our beliefs. Experts have a variety of tools designed to guide their judgment. Physicians use algorithms that inform them which treatment regimen could be the most effective for a particular patient. Engineers use root cause analysis to separate symptoms...