A job task analysis (JTA) must be used to create a testing process for promotion to higher rank in a police department. The job task analysis is a systematic method of defining what knowledge, skills, abilities, and/or personal traits (KSAPs) are critical to performing a job (Sergeant, for example) successfully over a period of time. Used by the person(s) constructing the promotional process, the JTA helps to develop an evaluation method that will predict who among the candidates is able to demonstrate identifiable KSAPs. We then use a nominal scale (the number 100 as the highest score achievable) and rank candidates based on their achievement in the testing process. So, what’s the problem?
Cut-off Scores
The people creating the examination process uses the job task analysis (along with other variables) to construct the testing process. Some form of written examination (multiple choice test, in-basket, etc.) is used to determine to what degree candidates are able to demonstrate requisite knowledge (such as supervisory, management, legal, policy, and procedure) in areas deemed important. However, because oral and written communications skills consistently rate very high on the JTA, the written examination is followed by some sort of oral test (such as a traditional oral board, supervisor/subordinate exam, leaderless discussion, along with some type of writing component).
The person creating the testing process uses the JTA to apply “weights” to each component of the testing process based on what the JTA has determined to be most critical. So, the written part of the process might be “weighted” at 50 percent of the candidate’s final scores, the oral part of the process 30 percent of a candidate’s final score and the writing component 20 percent of a candidate’s final score. Depending on the department and the rank, the parts and weights of the examination process will vary. In other words, being a Sergeant in a small town or city may require candidates possess KSAPs different than a Sergeant in a larger city. The promotional process may then have to be different as would the parts and weights of the examination.
In the vast majority of the cities in which I am hired to create and administer the promotional testing process, it has already been pre-determined that a “hurdle” approach will be used. Candidates must score a 70 on the written part of the examination in order to be eligible to participate in the remaining elements of the testing process. I have not yet been given a satisfactory explanation for why 70 percent is used as a cut-off score or what methodology was used to come up with this number. I’m usually told it is past practice or is part of the union/management contract. When I explain that the JTA should determine the parts and weights of the entire process and that the written examination only is a part (50 percent, for example) of testing a candidates ability to perform at the next level, I hear the refrain “that’s how we have always done it.”
So, a candidate for the position of police Sergeant takes the first part of the promotional examination process and receives a score of 69 percent. The candidate is informed that he/she is out of the process and may not proceed to the other parts of the test. Meanwhile, another candidate scores 70 percent on the written examination (which would be worth 35 points if the written exam is weighted at 50 percent) and is allowed to take the oral examination. The candidate then scores another 70 percent (which is 21 points if weighted at 30 percent), then takes an essay examination and scores a 70 percent (which is 14 points if weighted at 20 percent) for a final score of 70.
If the candidate who had scored the 69 (34.50 points) was allowed to go forward and take the rest of the examination process which the JTA indicated had variables either “very important” or “critical” to being able to do the job-and scored a 95 percent on the oral (28.50) and a 95 on the essay component (19 points), then the candidates who was not allowed to participate in the entire testing process would have received a final score of 82 – 12 points ahead of the candidate who received the 70 percent on the written examination and was allowed full participation in the rest of the promotional process.
In my opinion, not allowing candidates to participate in all parts of the testing process that the JTA has determined to be important often results in the department not promoting candidate who would have done an excellent job. If this is the case, then why don’t cities and towns allow all candidates to proceed to the different parts of the testing process?
The primary reason is that it is expensive. In a small town there may only be five officers eligible to take a Sergeant’s exam and regardless of the score a candidate achieves on the written examination an oral process with a writing component can be done in a single day. If, however there were 140+ candidates eligible to take the Sergeant’s examination, and everyone who took the written exam was allowed to proceed to an oral examination (30-35 minutes per candidate) it would take six days (24 per day) and three different panels.
From the point of view of the city, this would be a logistical nightmare. The oral test questions would get out. There might be a challenge to the test because different panel members are scoring candidates. Besides, where would the city get panel members who could commit six days to the enterprise?
As a person who creates and administers these tests (and as a former police officer) I don’t buy any of the above reasons. I have administered quality police examinations with multiple panels that lasted a week or more – the testing process was valid and reliable. There is no more important function in government than selecting those individuals entrusted to investigate, supervise, and manage its citizens and enforce its laws. There is too much at stake for the candidates and the city or town not to allow full participation in the promotional process.
If you are in a position to influence your department to allow the job task analysis to determine the parts and weights of the promotional process and for all candidates to be able to participate in all phases of the test (without the cut-off score) I recommend you do so.
