The Commission organises “dry-run” or mock tests which volunteers sit under exam conditions. The main purpose of this is to enable the Commission to improve the tests. But they also provide valuable experience for the volunteers. Most people thinking of applying for recorderships or other judicial offices will not have sat an exam for at least 15 years. They are unlikely to remember their last experience with much affection. Volunteering to sit a dry-run test provides an opportunity to practise basic exam technique, in a way which should improve your chances if you think that one day you may be doing the real thing.
The selection procedure explained
To explain why, it may help if I say something about the purpose of the tests, which is widely misunderstood.
The selection procedures used by the Commission are thorough and extremely resource-intensive. In the larger exercises, there may be several hundred candidates. It is not possible to carry out a full assessment on every one of them. Some short-listing is essential, and perfectly normal in any competitive appointment procedure. Written qualifying tests are a short-listing tool.
They are designed to select between 2.5 and 3 candidates for each vacancy who will proceed to the next stage. Once the short-list has been compiled the purpose of the tests has been largely exhausted, and the results are rarely used for final selection. By then the Commission and its panellists and staff expect to have much better information about candidates than anything which could be revealed by the written test.
Qualifying tests are a blunt instrument. Many of us will know of outstanding practitioners who would be likely to make fine judges, who have nevertheless failed the qualifying test and have therefore not proceeded to a full assessment of their merits. So why use them? For practical purposes there is only one alternative, which is to short-list on the basis of references and the candidate’s own self-assessment.
What’s the alternative?
In the early part of the Commission’s existence short-listing was done by a “paper sift” of this kind. It proved to be an adequate method of identifying candidates at the very top and the very bottom of the ability, but quite hopeless in the middle of the range where, in the nature of things, most candidates will be found.
References are valuable evidence, and the great majority of referees take a great deal of trouble over them. But candidates will have different referees, who will differ in their styles, their scales of value and the degree of detail which they provide. In large exercises references are not a suitable method for comparing one candidate with another, by an assessor who has seen neither of them. Their use as a short-listing tool, in the absence of any other evidence about the candidate apart from what he/she has said in the application form would be, and in practice was arbitrary and unfair. It also lacks the objectivity of an impersonal test.
I was at first an opponent of written qualifying tests. I still regard them as imperfect. But experience has persuaded me that they are less imperfect than any reasonable alternative.
It is important to appreciate that there is no official pass mark for qualifying tests. The Commission decides the maximum number of people that the panels can interview, and short-lists that number of top scorers in the test. If candidates are closely bunched about the line, the Commission tries, resources permitting, to increase the number of people interviewed. If the results of the qualifying test show surprising or off-the-wall results in terms of certain groups being excluded, the scripts are carefully reviewed and the marks adjusted if appropriate.
First class failures
Why then do some first class practitioners fail? I recently went through the test scripts in a large exercise for which I was the assigned commissioner. A number of excellent candidates from the Bar failed to make the short-list. I wanted to discover why. The commonest reasons were generally the most trivial:
- candidates did not study properly the legal materials provided in advance;
- or they failed to read the questions carefully;
- or (the commonest problem of all) they failed to plan the use of their time, spending too long on a few questions, often those carrying the fewest marks (the number of marks is shown on the paper).
In one sense all this is dismaying. But in another, it is reassuring, because the factors which cause good candidates to fail are eminently correctable. You can look up old tests on the JAC’s website to see how they are set. You can read the markers’ assessment of the results of each test on the website. Above all, you can volunteer for a mock test through the website, thereby helping us to make the tests better and building up valuable experience to polish up your examination technique.
Obviously, volunteers who sit the mock test must not be candidates in the exercise in question. In addition, through the JAC website you can volunteer to present yourself at a mock interview - part of the training process for the interview panels - or to take part in mock role-plays. If you dislike the JAC’s procedures, you may be converted, but if not you will at least have helped to improve them.
Jonathan Sumption QC is a JAC Commissioner
Written qualifying tests
- The tests are not designed to assess legal knowledge and they are not memory tests.
- They are designed, generally by judges in the relevant jurisdiction, to provide a preliminary assessment of your legal reasoning, forensic judgment and ability to make and explain decisions.
- They also seek to avoid giving an advantage to practitioners from any particular professional background or specialism.
- Before they are used, they are tried out on a representative group of volunteers who sit the test and give their views about whether these objectives have been achieved, as well as about issues such as timing and clarity. A good deal of work then goes into adjusting them for any anomalies which have come to light.
Mock Selection Exercises
If you think that you might apply for a judicial appointment in future, then take the opportunity to take part in a dry run.
Visit the JAC website at: www.judicialappointments.gov.uk/