Counsel asked readers who had sat one or more of the qualifying test papers to submit what they thought of the process and how they had coped with it (see tables on pp 17 and 18). The respondees will be referred to as Jane, Stephen, Rory and Thomas.
- Jane is a civil practitioner who successfully passed her first test.
- Thomas is a criminal and licensing barrister, who was successful on his first time.
- Stephen is civil practitioner and sat more than one test.
- Rory, a criminal practitioner, was unsuccessful.
According to figures released by the JAC on 9 February 2010, 982 lawyers applied for 128 posts as fee paid Recorders on the South East Circuit in 2009. Of those:
- 31 per cent of applicants, 28 per cent of shortlisted candidates and 37 per cent of those selected were women.
- 14 per cent of applicants, 12 per cent of shortlisted candidates and 13 per cent of those selected were from a black or minority ethnic (“BME”) background.
- 23 per cent of applicants, 11 per cent of shortlisted candidates and 11 per cent of those selected were solicitors.
- 3 per cent of applicants, 3 per cent of shortlisted candidates and 3 per cent of those selected were disabled people.
A sample question
The last Qualifying Test Paper, designed for the South Eastern Circuit competition, stated the aims:
- Intellectual capacity—the ability quickly to absorb and analyse information.
- Authority and communication skills—the ability to explain procedure and any decisions reached clearly and succinctly to all those involved.
Here is one question, based on the premise that juries have been replaced by a panel of six lay assessors:
The somewhat over-familiar and interfering long-serving usher at the court draws to your attention that he has heard ‘those useless lay assessors’ squabbling in the retiring room and expresses the view that two of them are ganging up on ‘the only one with any sense’.
What do you decide and what if any action do you take?
[Answer in bullet points.]”
Q1 What were the expectations of those who sat the test?
I am a civil practitioner and was applying to sit in the criminal jurisdiction. Prior to the test candidates were informed of the qualities and abilities which were to be tested and were assured that specialist knowledge was not required.
As one who never enjoyed exams in his youth, the prospect of taking one almost 16 years after the end of Bar finals did not particularly appeal. I am sure I was not alone in that view. The introduction of a formal written exam as part of the recorder selection process was a shock to many.
I prepared for the test primarily by familiarising myself with the authorities and guidance in relation to juries and questions of bias since I worked out (correctly) that these were areas which were likely to be covered. I also had the benefit of doing judicial shadowing previously and of hearing recorders in chambers discuss cases and JSB training exercises.
What was particularly helpful was that an example of a previous paper was available [on the JAC website], and it did seem to me that consideration of this was key as far as preparation for this part of the selection exercise. I took the opportunity to practice with this example and to discuss with others the potential answers. A preparatory contemplation of the competencies that were being tested seemed sensible too, giving thought in advance as to how they could be best demonstrated within a written test paper.
Q3 On the day?
The test itself was plainly designed to test time management skills and judicial qualities. I was strict in allocating equal time to the two parts of the paper. I was also alert to the fact that different questions had very different marks and allocated the time accordingly. By applying common sense exam techniques (although it was 18 years since I had last taken an exam), I was able to complete all of the questions although I would have benefited from greater time to check answers. I also benefited from being able to use a computer and therefore revise answers.
I believe that my experience in writing advices in civil work assisted in tasks such as giving reasons for decisions. I felt a little disadvantaged in not knowing the finer points of criminal procedure, but this did not prove to be fatal. The main aim of the test (and of the role play exercises) seemed to be to ensure that those appointed had appropriate people skills. It was necessary to be assertive and decisive whilst always listening and ensuring fairness in proceedings.
The first test was far more challenging. It involved substantial reading material. Due to time pressure, I was unable to complete the test. The subsequent two tests involved a greater emphasis on writing and less on reading. Although they involved a smaller reading bundle, candidates were given less time to complete the latter two tests. I was nevertheless able to complete both.
One major problem was that after being presented with a factual scenario, candidates would often be asked open questions such as “What enquiries would you make?”, “What questions would you ask?” and “What further submissions would you invite?” The candidate was then asked to state the decision he/she would make and the reasons for it. However, it was impossible to do this because the decision was entirely dependent on what the further enquiries, submissions and questions might reveal.
Another major problem was that a number of questions required a candidate to write a fully reasoned judgment. However, there was insufficient information in the test papers to even form a preliminary view, let alone provide a fully formulated decision.
They hired expensive hotels and equipment and the numbers involved would make a cattle truck look comfortable.
On the test day some basic exam technique dredged up from the past was of use. The marking scheme in relation to each question was set on the paper and I took it that time management was going to be the key to ensuring those marks were maximised. My priority was to ensure however basic it sounds, that I answered every question irrespective of whether it was by means of a lengthy considered answer or a series of bullet points. Taking time at the start to apportion the available time between questions seemed to pay off.
Q4 Did it serve its purpose?
As a civil practitioner who appears in courts all over England and Wales, I find it difficult to identify judges who know my work well. I therefore welcomed a selection method which did not depend on “who you know”.
I am aware of many senior practitioners “rubbishing” the process on the basis that some very good criminal practitioners failed the test. However the implication of the complaints is that those unsuccessful candidates were more deserving of appointment than those who were in fact selected. That is deeply insulting to those of us who were successful. In any system there will be those who succeed and those who do not. In a competition in which there are many more applicants than places … there has to be an objective way of filtering candidates at the outset.
A successful criminal Silk or senior junior, however good they are in that capacity, does not necessarily have (or demonstrate) the very different skills and abilities which have been identified as necessary for judicial appointment.
I did observe, however, that at my Induction Course, there was only one Silk in the whole group and he was a very new Silk. I would accept that it is possible that the test favours the younger candidate.
I am a barrister specialising in civil work. I did not feel that any of the tests favoured criminal practitioners. I felt the tests patently failed to assess whether a candidate met the “competencies” the JAC identified as the selection criteria. Many of the questions were confusing and lacked any real direction. Many involved novel scenarios that would rarely be encountered in real life. Other questions had such an array of answers that could be construed as “correct”, that it was difficult to see how they could discriminate between more and less able candidates.
As is often said, academic ability is not the main quality of a judge. There is more to it than that and no account is taken of a person’s service as a part time judicial holder at all in the initial selection, which cannot be right. It seems to me that if the skills of a judge are generic, then if they insist on using a test method there should be one across the board test used as a factor of the assessment process. Or they could go back to using references and application forms—a little more time consuming but maybe a tad more fair.
If the system is to be fair, then like most other bodies there should be a standard and if a person hits the mark then they are through to the next round, not “we’ll take the top x number and that’s it.” The new system doesn’t seem to be producing anywhere near the range and variety of appointees coming from the more unorthodox routes of practice.
As far as I could divine the exam was designed to place time pressure upon the applicant, and to test their ability to make reasoned decisions in a short space of time. It didn’t seem to me that it was seeking perfection but rather an ability to grasp the essential issues in the problem and to lay out a template for their resolution. I have no doubt that I was helped by reminding myself of the basics of the Criminal Procedure Rules and by considering the Equal Treatment Bench Book.
Q5 Did it play a part in the rest of the selection process?
The answers given in the test were not the subject of any specific questions at the interview/role play stage.
I could see no evidence that it played any further role in the selection process. Certainly during interview, I was asked no questions arising from the test.
THE JAC RESPONSE
Qualifying tests were much debated within the JAC before they were adopted, and I confess that I was initially an opponent. But experience has persuaded me that although there are difficulties about any method of short-listing, this one is fairer, more objective and more reliable than any alternative.
What do the results show?
Preliminary analyses suggest two tentative conclusions:
- First, the JAC’s tests, which are designed to test intellectual capacity and communication skills rather than legal knowledge, appear to have no significant bias for or against any particular category of candidates.
- Secondly, the tests have proved to be relatively good predictors of performance in the later, more exacting, stages of the process. Of course, it is difficult for candidates most of whom are in their 40s and 50s to sit an exam for the first time in many years. But it is just as difficult for everyone else, and there is plenty of information about the tests on our website to help you prepare (visit: www.judicialappointments.gov.uk).
The decisive factor for me was the experience of short-listing candidates on their self-assessments and references, which is probably the only practical alternative. While this method works quite well at the top and bottom of the ability range, it can be exceptionally difficult to apply fairly in the middle where in the nature of things most candidates are to be found. It is also seen to be unfair by candidates, often from non-traditional backgrounds, who do not have access to referees with the right kind of supporting evidence.
Jonathan Sumption QC is a JAC Commissioner