|
In the Ospre Part I examination, whether
it is to the rank of Sergeant or to Inspector, candidates will
have to answer 150 multiple-choice questions. However, only 140
of those questions will count towards a final mark. This is because
10 of the questions in the examination are validation questions.
Validation questions are 'test' questions
inserted at random points within the examination paper. Effectively,
they are included in an examination as a test of the question
itself (and not of the candidates' law knowledge) to establish
if the actual question is good enough to be included in a future
examination. Another way of thinking about these questions is
that they represent a form of quality assurance for questions.
If, after validation, the question matches set statistical criteria
then it will be introduced into a future examination as a 'real'
question and will count as one of the 140 questions that make
up a candidates' final score.
Many candidates criticise the inclusion
of validation questions and, on the face of it, it is easy to
understand why as these 'test' questions can have a detrimental
effect on candidate performance. For example:
|
Let
us imagine that question 24 in the 2010 Sergeants' examination
is a verification question. Of course, this fact is not
known to a candidate. The question is problematic and somewhat
lengthy and because of these factors a candidate gets into
difficulty with the question and it takes them 5 minutes
to formulate an answer. The drawback is that with 150 questions
in an examination, spending five minutes on one question
is going to have a detrimental effect on time management.
It is also highly likely to dent the candidates' confidence
after having spent so long on a question. And all
of that effort and time has been wasted as the question
is a verification question and will not count towards the
candidates' eventual mark.
|
Considering this scenario, it is reasonable
that candidates may strongly feel that verifications questions
should not be placed in an examination. However, there is a far
weightier counter-argument in favour of the inclusion of verification
questions in the examination.
How would a candidate feel if they revised
over a period of six months, spent a considerable amount of time,
effort and money in the process and then took a 140 question examination
where the questions had just been thrown together because they
were relevant to the syllabus but had never been strictly analysed
to make sure they were fair and then, as a result, that candidate
failed the examination? Can you imagine the amount of thoroughly
justified and warranted complaints this would create? This would
mean that a candidate fails the examination because of the large
number of poor quality questions and not a lack of knowledge and/or
understanding of the law. This danger far outweighs the problems
that answering verification questions in the examination creates
for an individual candidate.
One suggestion I have heard is that as
the NPIA examination question bank must contain hundreds if not
thousands of multiple-choice questions, the need to verify new
questions is no longer required. Such an approach could be taken
but it would mean that the examination question bank would remain
static and be unable to ask questions on new law as it was introduced.
In addition, as 'old' law fell away, so would the multiple-choice
questions on that 'old' law in the examination and the NPIA question
bank would offer less and less questions for examiners to choose
from. Consequently, within a short period of time the examination
would become stale and verification questions would have to be
re-introduced.
A further point that a candidate may
make is to ask why verification questions cannot be verified somewhere
else and not within the Ospre Part I examination? This is because
the only place that accurate feedback on multiple-choice questions
can be obtained is in the actual examination. If NPIA were to
offer candidates the facility of sitting through 50 verification
questions, how many candidates' would take up the offer? Even
if they did, there is no guarantee that the questions would be
taken seriously and therefore the feedback could be called into
doubt. Placing verification questions in the examination enables
realistic and large-scale feedback to be obtained to ensure that
the question can legitimately be introduced at a later stage having
undergone a strict testing regime.
In conclusion, candidates would be far
worse off if verification questions were not included as part
of the examination process. They are an absolute necessity to
ensure that questions are not only current but also fair (please
see the 'Myth, Rumour and Barrack-room Advice'
section for further information on verification questions).
|
