by mermann » Mon Sep 30, 2019 8:39 am
How Do I Read My Fail Report?
I just got off the phone with NCARB’s Vice President for Examination. I asked him what he wanted to communicate with the testing community. His response: Looking at the testing record of individuals, he finds a pattern whereby folks sit for a few tests, they pass those tests, and then fail a division, give up, and stop testing altogether. If you’ve read my previous posts, you know that failing a division is a feature, not a bug in this process, and that, unless you failed spectacularly, you should schedule a retake for the earliest possible allowed date. But what to study while you are waiting to retake? And how to use the fail report to guide that decision?
When looking to the fail report, you’ll want to read not only the level of achievement you earned on a section, but, more importantly, what percentage of questions come from that section.
So at first glance it might seem like you should study Content Area 5, because that is where their performance was demonstrably weakest. But look at the highlighted “Section %.” Content Area 5 only accounts for 2% to 8% of the exam (maybe three to nine questions), and because your pass depends on only your total score for the whole exam regardless of the content area, and because every question is worth one point, and construction cost estimation is a large field that would take forever to study and it is one that he clearly knows little about, Content Area 5: Construction Cost Estimates is probably the last content area he should study. Likewise, looking at Content Area 1 and Content Area 2, you might assume that because he achieved a “Level 2,” meaning he did well on those sections, that those would be areas he doesn’t need to study. Yet, he actually should study Content Area 1 and Content Area 2, because they together account for between two-thirds and three-quarters of the questions on the division (maybe 80 or 90 questions). The irony is that he likely has far more wrong questions, and therefore more room for improvement, on the content areas where the report says he did better!
Okay, so now you know what to study. No! You don’t. How do you study for a content area as large as “Content Area 1: Integration of Building Materials & Systems,” or “Content Area 2: Construction Documentation?” These areas are way too broad and not specific enough to guide you to retake preparation. If NCARB’s original sin in distributing these misleading reports is to inadvertently fool you into believing that you need to pass a certain number of content areas (rather than a certain number of questions, regardless of the content area), its new sin, adopted with ARE 5.0, is to inadvertently group the questions and title the content areas with language so vague an unspecific, as to render them almost useless. So study with a test prep program that knows the test and can weight subject matter the way the exam does, and worry about learning about architectural acoustics and swales rather than spending your valuable time studying the way the test makers populate vague content areas.—Michael Ermann, The Amber Book