ncbex-blog

[ncbex-blog]

Part II: Ontario

When a licensing authority decides to implement a new standardized lawyer licensure exam, what factors help shape that decision? In last month’s blog post, we examined the introduction of the new Solicitors Qualifying Examination, one of several changes to the solicitor licensure process in England and Wales scheduled to be implemented in 2021. This month, we’re shifting our attention closer to home to look at the standardized, open-book lawyer licensure examinations that were introduced a little over a decade ago in Ontario.

Just as each United States jurisdiction determines its own process and rules for admission to the bar, in Canada a law societyfor each province or territory determines that province’s or territory’s licensure process. In Ontario, lawyer licensure is governed by the Law Society of Ontario (known until recently as the Law Society of Upper Canada).

Lawyers in Ontario, as in most of Canada, are licensed as both solicitors and barristers, in contrast with the distinct licensure processes for solicitors and barristers that exists in England and Wales. Since 2006, candidates for the bar in Ontario have been required to take two summative licensure exams, the Barrister Examination and the Solicitor Examination. These exams, offered in both English and French, cover federal and Ontario law as well as ethics and professional responsibility (the different areas covered on each exam reflect a distinction between entry-level barrister and solicitor competencies). Each exam is seven hours long and includes 240 multiple-choice questions; the exams are given on paper and held two weeks apart, three times per year. Candidates can take the two exams in any order and may choose to take one exam or both during a given administration. Candidates must complete a law degree from an approved program before taking the exams; prior to being licensed, they must also complete a period of articling (apprenticeship) or alternative experiential training, as well as meeting a good character requirement. 1

The Barrister and Solicitor Examinations are administered as open-book exams, reflecting a longstanding practice in Ontario’s bar licensure process. 2 The Law Society develops a set of study materials on an annual basis for the subjects that will be examined that year; purchase of the study materials is mandatory for all candidates, and candidates may not share, sell, or give away their materials. 3 Candidates are permitted to bring these materials, as well as their own notes or texts, into the test room to use during the exams, but they are not subsequently permitted to remove any materials, books, or notes from the test room. The Law Society’s study materials may be used for all exam administrations offered in a particular year, but unsuccessful candidates must purchase a new set of materials if they retest in another year. 4

At the time the Barrister and Solicitor Examinations were first proposed in 2003, Ontario’s bar admissions process had remained essentially unchanged since 1957. That process, then known as the Bar Admission Course (BAC), required law graduates to complete both an articling placement and a pair 1 of postgraduate courses that lasted between one and three months each, covering lawyering skills and substantive and procedural law topics. These courses, run by the Law Society and taught largely by practitioners, included both skills assessments and law exams; while the types of questions used on the exams varied, the questions were not developed in consultation with psychometric experts or formally validated. 5

In making the case for reforming the BAC requirements, Ontario’s Task Force on the Continuum of Legal Education argued that substantial changes in both legal education and the legal profession had rendered the postgraduate courses less necessary and more burdensome than they had once been, and that it was redundant to provide legal instruction both as part of a degree program and in a postgraduate setting. The adoption of the recommended reforms, they argued, would update Ontario’s bar admissions process for the 21st century, “remov[ing] unnecessary barriers to admission and respect[ing] the principles of equity to which the Law Society is committed,” while also reducing costs for candidates and continuing to serve the public interest. 6 And they argued, further, that standardized examinations would “more effectively assess candidate competence” than the varied exams and assessments that had previously been used as part of the BAC courses. 7 The Barrister and Solicitor Examinations, created in response to these recommendations, are based upon lists of competencies required for entry-level practice that were developed (and later revised) with the help of psychometricians after extensive consultation with legal practitioners, following best practices for professional licensure. 8

It is interesting to compare the changes that took place in Ontario with those that are scheduled for implementation in England and Wales. In both cases, a shift away from postgraduate educational requirements toward standardized testing is seen as a necessary step to ensure that the licensing process effectively evaluates the competencies required for new lawyers, while also reducing the burden placed on candidates and creating a fairer and more equitable process. More broadly, both cases affirm a central role for high-quality licensure exams in the midst of a changing legal profession. As the Testing Task Force moves forward with its study of the bar examination in the U.S., it, too, must consider how the exam can best keep pace with a dynamic profession, while always ensuring that it continues to be a valid and reliable measure of the minimal competencies required for new lawyers.

https://lso.ca/becoming-licensed/lawyer-licensing-process.

George Hunter, et al., Task Force on the Continuum of Legal Education: Report to Convocation (2003).

The cost of the materials in both digital and paper form, Can$150 per exam, is included in the examination fees of Can$750 per exam. https://lso.ca/becoming-licensed/lawyer-licensing-process/fees-and-forms/fees-schedule.

https://lso.ca/becoming-licensed/lawyer-licensing-process/study-materials.

Dialogue on Licensing Topic 3: Reference Materials. Licensing Examinations: Assessment of Entry-level Competence 34–35.

supra note 2, 4–5.

Id. at 20.

supra note 5, 36–37. In late 2018, Ontario completed a new review of its bar admissions process, the Dialogue on Licensing / Dialogue sur l’accès à la profession, motivated in part by a persistent shortage of articling positions for candidates. A number of options for reform were considered, including doing away with the articling requirement altogether (none of the options included changes to the Barrister and Solicitor Examinations) (Peter Wardle, et al., Professional Development and Competence Committee: Options for Lawyer Licensing (2018)). The recommendations that were ultimately adopted by the Law Society’s governing body will maintain the articling process, with some changes; the addition of a new skills examination or assessment remains under consideration, although the details of what it might involve are still unknown. The recommended changes are scheduled to be implemented in 2021.

[ncbex-blog]

How Do We Know What We Need to Test?

If you were designing the bar exam from the ground up, how would you begin? Would you try to determine what type of questions to ask, how many questions the exam should include, how hard the questions should be? These are all important topics to consider. But you wouldn’t be able to get far without answering more fundamental questions, including what exactly the bar exam should be testing, and why—in other words, what do you want to be able to do with bar exam scores?

The bar exam, like all licensure tests, is intended first and foremost to assist in protecting the public by helping ensure that those who are licensed to practice a particular profession—in this case, the legal profession—are competent to do so. For the legal profession, public protection also includes protecting the rule of law and promoting the values of the profession, given that lawyers are officers of the court and play an important role in the judicial branch of government. The legal licensure process, of which the bar exam is just one part, must determine whether applicants to the bar possess the knowledge, skills, and abilities that are necessary for minimal competence as a new lawyer.

But how do we know which knowledge, skills, and abilities new lawyers will need? This requires, first of all, that we reach out to stakeholders to gather their opinions and input. Additionally, any claim that the knowledge or skills being assessed by an exam are necessary for the safe or effective performance of a job should typically be supported through a job or practice analysis.* NCBE conducted a national job analysis in 2012, and a new practice analysis will be a significant component of the Testing Task Force’s study of the bar examination.

A practice analysis is a systematic process for collecting and analyzing information about work activities and requirements using tools such as focus groups and surveys. A typical practice analysis begins by identifying the activities required to perform the relevant job, as well as the environmental context in which those activities are performed. It then determines the knowledge, skills, and abilities necessary to perform those activities. Because a license to practice a profession authorizes the individual to work in a variety of settings and positions or practice areas, it must include an appropriately broad sample of respondents and job positions. Because jobs can and do change over time, it is appropriate to repeat a practice analysis periodically, with the period for doing so informed by the pace and extent of change taking place within the job or profession in question. The Task Force’s practice analysis will identify the current job activities of newly licensed lawyers, but it will also consider how those activities are changing or are reasonably expected to change in the next five to ten years.

The Task Force’s research consultants have already begun conducting stakeholder research and laying the groundwork for a practice analysis. As always, we encourage you to sign up to receive updates from the Task Force as this work progresses.

* AERA et al, Standards for Educational and Psychological Testing 182 (2014).

[ncbex-blog]

International Lawyer Licensure Part II: Ontario

When a licensing authority decides to implement a new standardized lawyer licensure exam, what factors help shape that decision? In last month’s blog post, we examined the introduction of the new Solicitors Qualifying Examination, one of several changes to the solicitor licensure process in England and Wales scheduled to be implemented in 2021. This month, we’re shifting our attention closer to home to look at the standardized, open-book lawyer licensure examinations that were introduced a little over a decade ago in Ontario.

Just as each United States jurisdiction determines its own process and rules for admission to the bar, in Canada a law societyfor each province or territory determines that province’s or territory’s licensure process. In Ontario, lawyer licensure is governed by the Law Society of Ontario (known until recently as the Law Society of Upper Canada).

Lawyers in Ontario, as in most of Canada, are licensed as both solicitors and barristers, in contrast with the distinct licensure processes for solicitors and barristers that exists in England and Wales. Since 2006, candidates for the bar in Ontario have been required to take two summative licensure exams, the Barrister Examination and the Solicitor Examination. These exams, offered in both English and French, cover federal and Ontario law as well as ethics and professional responsibility (the different areas covered on each exam reflect a distinction between entry-level barrister and solicitor competencies). Each exam is seven hours long and includes 240 multiple-choice questions; the exams are given on paper and held two weeks apart, three times per year. Candidates can take the two exams in any order and may choose to take one exam or both during a given administration. Candidates must complete a law degree from an approved program before taking the exams; prior to being licensed, they must also complete a period of articling (apprenticeship) or alternative experiential training, as well as meeting a good character requirement.1

The Barrister and Solicitor Examinations are administered as open-book exams, reflecting a longstanding practice in Ontario’s bar licensure process.2 The Law Society develops a set of study materials on an annual basis for the subjects that will be examined that year; purchase of the study materials is mandatory for all candidates, and candidates may not share, sell, or give away their materials.3 Candidates are permitted to bring these materials, as well as their own notes or texts, into the test room to use during the exams, but they are not subsequently permitted to remove any materials, books, or notes from the test room. The Law Society’s study materials may be used for all exam administrations offered in a particular year, but unsuccessful candidates must purchase a new set of materials if they retest in another year.4

At the time the Barrister and Solicitor Examinations were first proposed in 2003, Ontario’s bar admissions process had remained essentially unchanged since 1957. That process, then known as the Bar Admission Course (BAC), required law graduates to complete both an articling placement and a pair1 of postgraduate courses that lasted between one and three months each, covering lawyering skills and substantive and procedural law topics. These courses, run by the Law Society and taught largely by practitioners, included both skills assessments and law exams; while the types of questions used on the exams varied, the questions were not developed in consultation with psychometric experts or formally validated.5

In making the case for reforming the BAC requirements, Ontario’s Task Force on the Continuum of Legal Education argued that substantial changes in both legal education and the legal profession had rendered the postgraduate courses less necessary and more burdensome than they had once been, and that it was redundant to provide legal instruction both as part of a degree program and in a postgraduate setting. The adoption of the recommended reforms, they argued, would update Ontario’s bar admissions process for the 21st century, “remov[ing] unnecessary barriers to admission and respect[ing] the principles of equity to which the Law Society is committed,” while also reducing costs for candidates and continuing to serve the public interest.6 And they argued, further, that standardized examinations would “more effectively assess candidate competence” than the varied exams and assessments that had previously been used as part of the BAC courses.7 The Barrister and Solicitor Examinations, created in response to these recommendations, are based upon lists of competencies required for entry-level practice that were developed (and later revised) with the help of psychometricians after extensive consultation with legal practitioners, following best practices for professional licensure.8

It is interesting to compare the changes that took place in Ontario with those that are scheduled for implementation in England and Wales. In both cases, a shift away from postgraduate educational requirements toward standardized testing is seen as a necessary step to ensure that the licensing process effectively evaluates the competencies required for new lawyers, while also reducing the burden placed on candidates and creating a fairer and more equitable process. More broadly, both cases affirm a central role for high-quality licensure exams in the midst of a changing legal profession. As the Testing Task Force moves forward with its study of the bar examination in the U.S., it, too, must consider how the exam can best keep pace with a dynamic profession, while always ensuring that it continues to be a valid and reliable measure of the minimal competencies required for new lawyers.

https://lso.ca/becoming-licensed/lawyer-licensing-process.

George Hunter, et al., Task Force on the Continuum of Legal Education: Report to Convocation (2003).

The cost of the materials in both digital and paper form, Can$150 per exam, is included in the examination fees of Can$750 per exam. https://lso.ca/becoming-licensed/lawyer-licensing-process/fees-and-forms/fees-schedule.

https://lso.ca/becoming-licensed/lawyer-licensing-process/study-materials.

Dialogue on Licensing Topic 3: Reference Materials. Licensing Examinations: Assessment of Entry-level Competence 34–35.

supra note 2, 4–5.

Id. at 20.

supra note 5, 36–37. In late 2018, Ontario completed a new review of its bar admissions process, the Dialogue on Licensing / Dialogue sur l’accès à la profession, motivated in part by a persistent shortage of articling positions for candidates. A number of options for reform were considered, including doing away with the articling requirement altogether (none of the options included changes to the Barrister and Solicitor Examinations) (Peter Wardle, et al., Professional Development and Competence Committee: Options for Lawyer Licensing (2018)). The recommendations that were ultimately adopted by the Law Society’s governing body will maintain the articling process, with some changes; the addition of a new skills examination or assessment remains under consideration, although the details of what it might involve are still unknown. The recommended changes are scheduled to be implemented in 2021.

For more information, visit testingtaskforce.org.

[ncbex-blog]

The Role of Bar Examinations in International Lawyer Licensure Part 1: England and Wales

What can we learn from looking at lawyer licensure examinations in other countries? The question may seem to be beyond the scope of the Testing Task Force’s work. After all, the Task Force is studying the bar examination as it is administered by United States jurisdictions, and the legal licensure processes of other countries are necessarily dictated by the legal systems and needs of those countries. Nevertheless, it can be instructive to consider foreign bar examinations and the role played by those examinations within each country’s respective licensure process.

In this month’s blog post, we’re taking a look at the new Solicitors Qualifying Examination, which will be required for prospective solicitors in England and Wales beginning in 2021. Next month, we’ll turn our attention to Ontario, where significant reforms to lawyer licensing requirements were made in 2006, including the introduction of the licensing examinations now in use there. In both cases, postgraduate educational requirements have been or will be replaced with new licensure examinations.

Lawyers in the United Kingdom are either solicitors or barristers; a pair of centralized regulatory bodies, the Solicitors Regulation Authority (SRA) and the Bar Standards Board (BSB), authorize and regulate solicitors and barristers, respectively, in England and Wales. In 2013, following the completion of a major study of the education and training requirements for new lawyers in England and Wales, the SRA began planning significant changes to its regulation of solicitors, which are scheduled to take effect beginning in 2021. These changes include the introduction of a standardized exam and other changes to the solicitor licensure process.1

Under the existing (soon to be former) system of licensure, those who wish to be admitted as solicitors in England and Wales must complete a multi-step process that includes both undergraduate and postgraduate educational requirements and supervised work (training) requirements, as well as a character assessment; there is no bar examination. Under the new system, a period of supervised work will continue to be required, though with changes; the character assessment will also remain. However, the undergraduate educational requirements will become more flexible, and the postgraduate educational requirements will be eliminated. In their place, a new examination, the Solicitors Qualifying Examination, will be required.2

While the details of the new examination are still being finalized, as currently envisioned it will have two stages.3 Candidates will most likely take the first stage of the exam soon after completing an undergraduate degree, while the second stage of the exam would be taken following the required two years of supervised work experience.

One portion of the first stage of the exam will be administered via computer-based testing and will use objective questions (including multiple-choice, single best answer, and extended matching questions) to test candidates’ ability to apply substantive and procedural legal knowledge, as well as knowledge of legal ethics, to specific scenarios. This stage will also include written exam components designed to test candidates’ legal research and writing abilities.

The second stage of the exam will test legal skills (e.g., client interviewing, advocacy and persuasive oral communication, and legal research), requiring candidates to complete several different legal tasks such as interviewing a simulated client or using a database to perform legal research.

In a recent issue of the Bar Examiner, Julie Brannan, the Director of Education and Training for the SRA, writes that the introduction of the new exam will result in “a more fair, objective process” for admission, as “all solicitors will [be] assessed against a single, consistent standard.”4 Brannan notes, among other things, the difficulty in the current system of ensuring fair and consistent standards across the wide range of educational providers and training supervisors who are charged with assessing the performance of solicitor candidates. Additionally, the SRA hopes that the new requirements will help ease a backlog of candidates who have completed their educational requirements but cannot find trainee positions, and will make the choice of solicitor as career path more accessible to those for whom the cost of the Legal Practice Course (currently around $20,000, per Brannan) presents a significant barrier.

It is important to note that, in redesigning its licensure program, the SRA began by reviewing and updating the list of competencies that new solicitors should possess.5 Only once core competencies are established can a regulatory authority determine how best to verify that candidates possess those competencies; this is why the Testing Task Force’s study begins with stakeholder research and a practice analysis, all designed to determine the knowledge, skills, and abilities that newly licensed lawyers need.

But the SRA’s reforms also make clear that it is necessary to determine not only which competencies need to be verified, but how best to verify them—and, if an examination will be used, then what type of examination (format, timing, etc.) is best. In the case of solicitor regulation in England and Wales, a wide-ranging exam was determined to be preferable to a series of undergraduate and postgraduate educational requirements. Check back next month for our post about a similar shift by the Law Society of Ontario from a postgraduate educational component to standardized testing as a means of verifying the competence of candidates seeking admission to the bar.

https://www.sra.org.uk/sra/news/press/sqe-ensure-high-consistent-standards.page; https://www.sra.org.uk/sra/news/press/sqe-launch-2021.page.

Currently, in addition to passing a character assessment, aspiring solicitors must successfully complete (1) an undergraduate law degree that meets certain requirements for subject-matter coverage OR an undergraduate degree in another field and a one-year postgraduate course in law; (2) a one-year postgraduate Legal Practice Course, which covers “core practice areas” such as wills and estates, business law, and property law; skills such as  legal research, writing, and advocacy; and more specialized substantive topics, such as employment and family law; (3) a two-year Period of Supervised Training in an approved law office or legal department; and (4) a short Professional Skills Course, generally taken as part of the Period of Supervised Training. See also https://www.sra.org.uk/students/resources/student-information.page. Beginning in 2021, (1) the undergraduate law degree requirement will be replaced with a requirement that candidates have an undergraduate degree in any field or an equivalent qualification or experience; (2) the Legal Practice Course will be eliminated; (3) the Period of Supervised Training will be replaced with a requirement that candidates have two years of “qualifying legal experience” (which might include, e.g., working as a paralegal or in a university legal services office); (4) the Professional Skills Course will be eliminated; and (5) the Solicitors Qualifying Examination will be required. See also https://www.sra.org.uk/sra/policy/sqe/solicitor-persona.page.

The description of the exam components given here is based on information presented in Julie Brannan, “Training for Tomorrow: The Reform of Education and Training Requirements for English and Welsh Solicitors,” 86(4) The Bar Examiner(Winter 2017-2018) 17, 24-26.

Id. at 17, 18.

http://www.sra.org.uk/solicitors/competence-statement.page; see also Brannan, supra note 3 at 20.

For more information, visit testingtaskforce.org.