# data[].attributes.attachment_link.description
The National Adult Literacy Agency (NALA) has developed four new workbooks for literacy learners at Levels 1 to 2. These workbooks are for adults living in Ireland who want to improve or practise new words, reading, writing and spelling skills.

# data[].attributes.body
Easier English Wiki provides free, adaptable ESOL teaching materials, including ready-made lessons, simplified news articles, and quizzes on global topics.

It supports teachers with practical ideas for all levels and offers resources suitable for volunteer or refugee contexts, making it ideal for engaging, skills-based learning in adult and community ESOL classrooms.

# data[].attributes.attachment_link.readable_text
and are you interested in the world? read these Easier English articles, or choose one of the QUIZZES.

- more than 125, free Ready Lessons to help learners develop speaking, reading, writing, vocabulary and grammar. Also see More teaching ideas for many more teaching ideas.

Look at our latest issues:

Issue 530 - March / April 2021 - "Democracy on the edge"

Issue 529 - January / February 2021 - "The biodiversity emergency"

Issue 528 - November / December 2020 - "A caring economy: what would it take?"

Issue 527 - September / October 2020 - "Lessons from the pandemic"

Issue 526 - July / August 2020 - "The Kurds: Betrayed again"

Issue 525 - May / June 2020 - "Take a deep breath: the fight for clean air"

Issue 524 - March / April 2020 - "How we make poverty"

Issue 523 - January / February 2020 - "Borders"

Issue 522 - November / December 2019 - "China in charge"

Issue 521 - September / October 2019 - "Who owns the sea?"

Issue 520 - July / August 2019 - "The right to the city"

Issue 519 - May / June 2019 - "How to avoid climate breakdown"

Issue 518 - March / April 2019 - "Building a new internationalism"

Issue 517 - January / February 2019 - "Trade in turmoil: A chance for justice?"

Issue 516 - November / December 2018 - "The dirt on waste"

Issue 515 - September / October 2018 - "Making peace in a world at war"

Issue 514 - July / August 2018 - "The next financial crisis"

Issue 513 - June 2018 - "A better media is possible"

Issue 512 - May 2018 - "Public ownership rises again"

Issue 511 - April 2018 - "Who cares?"

Issue 510 - March 2018 - "No justice, no peace"

Issue 509 - January / February 2018 - "What's left for the young?"

Issue 508 - December 2017 - "Clampdown!"

Issue 507 - November 2017 - "Humans vs robots"

Issue 506 - October 2017 - "Brazil's soft coup"

Issue 505 - September 2017 - "Bad Education"

Issue 504 - July / August 2017 - "The Equality Effect"

Issue 503 - June 2017 - "Homelessness"

Issue 502 - May 2017 - "West Papua: Freedom in sight?"

Issue 501 - April 2017 - "Populism rises again"

Issue 500 - March 2017 - "Our 500th issue - brave people"

Issue 499 - January / February 2017 - "Sisters and Brothers: Three decades of change in an African village"

Issue 498 - December 2016 - "The Coming War on China / Tax Avoidance"

Issue 497 - November 2016 - "Peace in Colombia?"

Issue 496 - October 2016 - "World Fiction Special"

Issue 495 - September 2016 - "Trade Unions"

Issue 494 - July / August 2016 - "Smily-faced monopolists"

Issue 493 - June 2016 - "After Ebola"

Issue 492 - May 2016 - "Technology justice"

Issue 491 - April 2016 - "Forests"

Issue 490 - March 2016 - "Saudi Arabia and the West"

Issue 489 - January / February 2016 - "Migration"

Issue 488 - December 2015 - "10 Economic Myths"

Issue 487 - November 2015 - "Paris Climate Summit"

Issue 486 - October 2015 - "The Transgender Revolution"

Issue 485 - September 2015 - "Syria's Good Guys"

Issue 484 - July/August 2015 - "Alternatives to Capitalism"

Issue 483 - June 2015 - "Fundamentalism: Power, politics and persuasion"

Issue 482 - May 2015 - "Global Banking"

Issue 481 - April 2015 - "Total control - is Monsanto unstoppable?

Issue 480 - March 2015 - "Renewable energy"

Issue 479 - January / February 2015 - "Democracy in the digital age

Issue 478 - December 2014 - "NGOs: do they help?"

Issue 477 - November 2014 - "Ending the Oil Age"

Issue 476 - October 2014 - "The New Cuba"

Issue 475 - September 2014 - "Gold Trouble"

Issue 474 - July / August 2014 - "Feminism"

Issue 473 - June 2014 - "Save our speech - the politics of language loss"

Issue 472 - May 2014 - "Organ trafficking"

Issue 471 - April 2014 - "The war on whistleblowers"

Issue 470 - March 2014 - "Commodities"

Issue 469 - January / February 2014 - "Why are we locking up migrants?"

Issue 468 - December 2013 - "Fracking"

Issue 467 - November 2013 - "We are able; time to rethink disability"

Issue 466 - October 2013 - "Where have all the girls gone?"

Issue 465 - September 2013 - "Pirates"

Issue 464 - July / August 2013 - "Debt - a global scam"

Issue 463 - June 2013 - "Argentina's challenge"

Issue 462 - May 2013 - "Land grabs"

Issue 461 - April 2013 - "Why is housing in such a state?"

Issue 460 - March 2013 - "What has development done for me?"

Issue 459 - January / February 2013 - "The Feral Rich"

Issue 458 - December 2012 - "Internet Showdown"

Issue 457 - November 2012 - "Bad Medicine"

Issue 456 - October 2012 - "Youth Rising"

Issue 455 - September 2012 - "Drugs Legalization"

Issue 454 - July / August 2012 - "How Co-operatives can save the planet"

Also get these NEW BOOKLETS AT 3 LEVELS: Pre-intermediate, Intermediate and Upper Intermediate:

As the articles are simplified, the words, text structure and quotes may have been changed. For the originals, please see: newint.org

Follow us on Twitter: @EasierNewInt for new articles / contact us at: [email protected]

# data[].attributes.attachment_link.title
New Internationalist Easier English Wiki

# data[].attributes.attachment_link.description
and are you interested in the world? read these Easier English articles, or choose one of the QUIZZES . - more than 125, free Ready Lessons to help learners develop speaking, reading, writing, vocabulary and grammar. Also see More teaching ideas for many more teaching ideas.

# data[].attributes.body
An innovative new English language teaching programme for migrant mums with bumps and babies

# data[].attributes.headline
Link to a Padlet of mixed level resources on study skills

# data[].attributes.subject
Link to a Padlet of mixed level resources on study skills

# data[].attributes.headline
ESOL Driving test materials downloaded (ESOL UK)

# data[].attributes.subject
ESOL Driving test materials downloaded (ESOL UK)

# data[].attributes.headline
Learning English podcast series: Talking about...

# data[].attributes.subject
Learning English podcast series: Talking about...

# data[].attributes.attachment_link.title
BBC Learning English - Real Easy English

# data[].attributes.attachment_link.description
Real conversations in easy English to help you learn English. BBC Learning English presenters talk about a new topic each week and explain words to help you learn.

# data[].attributes.body
An innovative approach to English language learning that has been designed to extend learning beyond the traditional classroom. This fully interactive, Northern Ireland-based online programme empowers learners with engaging digital lessons that build English proficiency and digital literacy.

# data[].attributes.attachment_link.description
Simple, Practical, Effective ESOL Digital Pathways is an innovative approach to English language learning that has been designed to extend learning beyond the traditional classroom. This fully interactive, Northern Ireland based online programme empowers learners with engaging digital lessons that

# data[].attributes.body
See column 13 on Speaking and Listening for Podcast suggestions

# data[].attributes.attachment_link.description
Hi everyone, I'm Lizzy! I'm a native English speaker and I've been an ESOL teacher in Oldham, Greater Manchester for five years. All of my videos are linked to the ESOL Core Curriculum.

# data[].attributes.body
Lauren is an experienced ESOL tutor:

# data[].attributes.attachment_link.keywords[]
learn english study improve speak listen read write practise

# data[].attributes.attachment_link.description
Hi and Welcome to Lauren's English Corner! Lauren's English Corner is here to help you improve in your English speaking and listening skills and difficult English grammar. With Lauren's English Corner you can request classes in the comments boxes on each of the videos and we will endeavour to get those videos made for you!

# data[].attributes.attachment_link.description
Improve your English with our fun and exciting learning apps! Designed for all the family, our games, podcasts, videos and quizzes will help you learn English at home or on the move.

# data[].attributes.attachment_link.provider_name
IDEA: International Dialects of English Archive | free dialect and accent recordings for the performing arts

# data[].attributes.attachment_link.readable_text
Listen to accents and dialects of England. There are currently 113 samples from England, organized into nine regions: Southwest, Southeast, London, East, West Midlands, East Midlands, Yorkshire and Humber, Northwest, and Northeast. There is one additional group for samples that don't belong in any of the regions. (At the bottom of this page, you will also see the samples listed in numeral order.) Also see Jamaica 15.
Southwest
This region includes the counties of Somerset, Bristol, Gloucestershire, Oxfordshire, Wiltshire, Dorset, Devon, and Cornwall. Some major cities are Bath, Bristol, Bournemouth, Cheltenham, Exeter, Gloucester, Oxford, Plymouth, Swindon, Taunton, Torbay, Weston-super-Mare, and Weymouth. Please select a sample from the list below. (Also see Devon History 1 on the Oral Histories page.)
Southeast
This region includes the counties of Berkshire, East Sussex, Hampshire, Isle of Wight, Kent, Surrey, and West Sussex. Some major cities are Brighton, Worthing, Portsmouth, Southampton, Canterbury, and Reading. Please select a sample from the list below.
London
This region includes all of Greater London. Please select a sample from the list below.
East
This region includes the counties of Essex, Hertfordshire, Bedfordshire, Buckinghamshire, Cambridgeshire, Norfolk, Suffolk, and parts of Lincolnshire. Some major cities are Chelmsford, Colchester, Welwyn Garden City, Hatfield, St. Alban's, Bedford, Cambridge, Norwich, Ipswich, King's Lynn, Luton, and Bedford. Please select a sample from the list below. (Also see Suffolk History 1 on the Oral Histories page.)
East Midlands
This region includes the counties of Derbyshire, Nottinghamshire, Lincolnshire (part only), Leicestershire, and Northamptonshire. Some major cities are Derby, Nottingham, Lincoln, Northampton, and Leicester. Please select a sample from the list below.

England 6 male, 40s, 1950s, white, Rushden (Northamptonshire)
England 66 male, 40s, 1962, white, Gainsborough (Lincolnshire) & Yorkshire
West Midlands
This region includes the counties of Herefordshire, Shropshire, Staffordshire, Warwickshire, West Midlands, and Worcestershire. Some major cities are Birmingham, Wolverhampton, Dudley, Walsall, West Bromwich, Coventry, Stoke-on-Trent, Stratford-upon-Avon, and Sutton Coldfield. Please select a sample from the list below.
Yorkshire and Humber
This includes the counties of South Yorkshire, West Yorkshire, North Yorkshire, East Riding of Yorkshire, and Lincolnshire (part only). Some major cities are Sheffield, Rotherham, Barnsley, Doncaster, Wakefield, Bradford, Leeds, Scarborough, York, Halifax, Hull, and Scarborough. Please select a sample from the list below.
Northwest
This includes the counties of Cheshire, Cumbria, Greater Manchester, Lancashire, and Merseyside. Some major cities are Manchester, Liverpool, Chester, Crewe, Macclesfield, Warrington, Barrow-in-Furness, Carlisle, Bolton, Wigan, Preston, Lancaster, Blackpool, and Blackburn. Please select a sample from the list below.
Northeast
This region includes the counties of Northumberland, Tyne and Wear, County Durham, and North Yorkshire (part only). Some major cities are Newcastle, Sunderland, Tyneside, Wearside, Teesside, Durham, Darlington, Stockton-on-Tees, Middlesborough, and Hartlepool. Please select a sample from the list below.
Received Pronunciation and Miscellaneous
This group includes Standard British English (RP) speakers of unknown history and others who do not logically fit in any of the above regional categories. Please select a sample from the list below.

England 1 male, 50s, 1942, white, central and southern England
England 2 female, 20s, 1978, white
England 7 male, white
England 63 female, 1954, white, Surrey (and abroad)
England 64 female, 20, 1987, white, Bury (Manchester) and Germany
Samples in numerical order
This is a list of all England subjects in numerical, rather than geographical, order. Please select a sample from the list below.

For more information on the Estuary dialect, we suggest listening to the July 2020 edition of the In a Manner of Speaking podcast. And for information on Received Pronunciation, we suggest the November 2019 episode featuring renowned linguist David Crystal.

# data[].attributes.attachment_link.title
Accents and Dialects of England | IDEA: International Dialects of English Archive

# data[].attributes.attachment_link.description
Listen to accents and dialects of England. There are currently 113 samples from England, organized into nine regions: Southwest, Southeast, London, East, West Midlands, East Midlands, Yorkshire and Humber, Northwest, and Northeast. There is one additional group for samples that don't belong in any of the regions.

# data[].attributes.headline
A Tour of the Accents of England - YouTube video

# data[].attributes.subject
A Tour of the Accents of England - YouTube video

# data[].attributes.headline
A Tour of the UK and Ireland in accents - You Tube video

# data[].attributes.subject
A Tour of the UK and Ireland in accents - You Tube video

# data[].attributes.attachment_link.title
A tour of The UK and Ireland in accents.

# data[].attributes.attachment_link.description
Exam suitable dictionaries are "word to word" or "one to one" dictionaries. In other words, there are no definitions, example sentences or usage notes, there are just straightforward direct translations.As the name this makes these dictionaries perfect for exam conditions. Please ensure however these meet the rules of

# data[].attributes.attachment_link.description
Buy Read and Write every day ESOL Literacy Teacher’s Book by Butler, Dave (ISBN: 9781738449729) from Amazon’s Book Store. Everyday low prices and free delivery on eligible orders.

# data[].attributes.headline
Curriculum For SIFE (Students With Interrupted Formal Education) Students

# data[].attributes.subject
Curriculum For SIFE (Students With Interrupted Formal Education) Students

# data[].attributes.body
The SIFE Equity Project and Oakland Unified School District, supported by CalNew, Partnered to create a free Open-Source curriculum for SIFE.

# data[].attributes.body
ESOL courses

ESOL Queen

Twinkl (for kids but suitable).

ISL Collective.

BC, LearnEnglish Teens.

Twinkl - display photos.

Lingua House.

# data[].attributes.headline
Accessible Drug and Alcohol information for People form Ethnic Backgrounds Living in Wirral. 

# data[].attributes.subject
Accessible Drug and Alcohol information for People form Ethnic Backgrounds Living in Wirral. 

# data[].attributes.attachment_link.title
DRUG TALK: harm reduction for people from ethnic backgrounds living in Wirral

# data[].attributes.attachment_link.description
Accessible drug and alcohol harm-reduction information for people from ethnic backgrounds living in Wirral. Available in over 100 languages.

# data[].attributes.headline
Supporting ESOL in Devolved Authorities  - AoC & Bell Foundation

# data[].attributes.subject
Supporting ESOL in Devolved Authorities  - AoC & Bell Foundation

# data[].attributes.attachment_link.title
ESOL Report - Supporting ESOL in Devolved Authorities

# data[].attributes.body
Search Lauren's English Corner on YouTube for dozens of useful videos to play in class or refer for self-study.

# data[].attributes.attachment_link.readable_text
Thousands ready to play online
We have the best collection of word search puzzles online, with new ones being added regularly.
They are fun to play, but also educational, in fact, many teachers make use of them.

Puzzles are 100% free to play and work on desktop pc, mac, mobile and tablet. Or you can go old school and print them to enjoy offline later.

Plus, if you're feeling a little more adventurous, why not create your very own with our simple to use Word Search Maker, and then share them with your friends.

To get started playing, just select a game from below. Best of luck.

# data[].attributes.attachment_link.description
Thousands of word search puzzles and games to play online or print out, covering a mix of both fun and educational topics. You can even create your very own game in just a few simple steps.

# data[].attributes.headline
ESOL course themed around health and wellbeing. 

# data[].attributes.subject
ESOL course themed around health and wellbeing. 

# data[].attributes.body
By Bromley Bow Centre

# data[].attributes.attachment_link.title
ESOL Resources English for Speakers of Other Languages

# data[].attributes.attachment_link.description
We have designed and tested innovative ESOL resources themed around health and wellbeing. Useful for tutors looking for course materials to help learners.

# data[].attributes.attachment_link.description
Home " ELT Materials " Pronunciation materials

# data[].attributes.headline
Video examples of skills abilities from E1 to L2.

# data[].attributes.subject
Video examples of skills abilities from E1 to L2.

# data[].attributes.body
A series of videos providing examples of speaking, listening, reading and writing from learners with skills at ESOL Entry Level 1 (E1) to Level 2 (L2).

Courtesy of Hackney Education.

# data[].attributes.attachment_link.readable_text
abc English: Student Books & Teacher's Guides
​
​ Book One focuses on alphabet sounds, one syllable words, and sight words in pattern sentences. Book Two adds multi-syllable word practice and more sight words in pattern sentences. The Teacher's Guide for each book contains suggestions for interactive games and activities to introduce each page in the student book. Choose any book image below to see a full preview. ​Images in the printed books are grayscale, to keep printing costs as low as possible. The PDF files are in full color.

Each book contains fourteen high-interest stories about relevant topics for adolescent and adult English Language Learners, ordered by reading difficulty level. The stories in Book One range from Lexile Level 100 to 300, and the stories in Book Two range from Lexile Level 300 to 500. Choose any book image below to see a full preview. ​Images in the printed books are grayscale, to keep printing costs as low as possible. The PDF files are in full color. ​​

Copy for linking on the same page:

https://www.abceng.org/books.html?data-scroll-to-anchor=EER

Copy for linking from an external page:

https://www.abceng.org/books.html?data-anchor-link=EER

Easy English Readers have simple vocabulary and color photos and are designed with English learners in mind. Each book is 26-32 pages long and sized 8x8" with high quality color printing. Choose any book image below to see a full preview . ​​

Copy for linking on the same page:

https://www.abceng.org/books.html?data-scroll-to-anchor=abcPD

Copy for linking from an external page:

https://www.abceng.org/books.html?data-anchor-link=abcPD

The abc English Picture Dictionary pairs well with any of the above books, and also stands alone as a useful literacy instruction tool. The first part of the dictionary contains sets of words and images to teach various phonics concepts. The second part of the dictionary contains sets of common vocabulary words and images. The Picture Dictionary is printed in full color.

​English for Work contains a series of lessons to prepare adult English learners for the workplace, especially learners who have emerging literacy skills and low-beginning English levels. Each lesson also contains suggestions for how to expand or adapt the lessons for higher level learners. In the lesson instructions, you'll learn how to access supplemental resources (image slideshows, worksheets, etc.) on a companion website. Click on either image to see a preview.
To make a bulk purchase using a purchase order, please contact me.

# data[].attributes.attachment_link.description
abc English: Student Books & Teacher's Guides​​ Book One focuses on alphabet sounds, one syllable words, and sight words in pattern sentences. Book Two adds multi-syllable word practice and more sight words in pattern sentences. The Teacher's Guide for each book contains suggestions for interactive games and activities to introduce each page in the student book.

# data[].attributes.attachment_link.readable_text
Lockdown Special Offers!

During the current Coronavirus pandemic, some resources, suitable for online teaching, will be FREE!

Buy a printed book and request the download free (email your request and a copy of your invoice as evidence of the purchase)

Buy The Drop-In Series download - formatted for printing into books and receive an e-book version free

The offers will end on October 1st 2020

Forward With Phonics aims to provide training, advice, support and resources for teachers of older children, adults and ESOL learners, who want to use phonics to help improve literacy skills.

Services

A comprehensive training in synthetic phonics is provided in the 4-day Certificated Sounds-Write course. For more information about the programme go to www.sounds-write.co.uk

We provide taster sessions and workshops which explain the concept of synthetic phonics in a fun and practical way. The sessions can be adapted to suit the target audience, be it childcare workers in playgroups and nurseries or adult education providers in Job Centres or Prisons.

Forward with Phonics provides individual tuition on request from schools and private tuition for children and adults.

Resources

There are many phonics resources available for use with young children. Forward with Phonics aims to provide a variety of resources suitable for teenagers and adults.

# data[].attributes.attachment_link.description
During the current Coronavirus pandemic, some resources, suitable for online teaching, will be FREE!

# data[].attributes.headline
Benchmarking ESOL in the East Midlands - Report

# data[].attributes.subject
Benchmarking ESOL in the East Midlands - Report

# data[].attributes.body
Courtesy of East Midlands Councils.

# data[].attributes.attachment_link.provider_name
East Midlands Councils - East Midlands Councils represents the interests of local councils to Government and national organisations. We enable local councils to work together on key

# data[].attributes.attachment_link.description
East Midlands Councils (EMC) is working with the Migrant English Support Hub (MESH) to develop deeper understanding of the challenges facing English for Speakers of Other Languages (ESOL) providers and migrant English learners in the East Midlands. We wanted to ‘benchmark’ ESOL in the East Midlands, to gain insight into opportunities to provide support and coordination to

# data[].attributes.headline
Life in the UK Test 2020 - dyslexia friendly book 

# data[].attributes.subject
Life in the UK Test 2020 - dyslexia friendly book 

# data[].attributes.attachment_link.title
Life in the UK Test 2020 - dyslexia friendly: Official Material and Test Questions for British Citizenship

# data[].attributes.attachment_link.description
Buy Life in the UK Test 2020 - dyslexia friendly: Official Material and Test Questions for British Citizenship by Lewis, Hugh (ISBN: 9781911538240) from Amazon’s Book Store. Everyday low prices and free delivery on eligible orders.

# data[].attributes.headline
Link to a separate Padlet on teaching pronunciation (including specific language difficulties)

# data[].attributes.subject
Link to a separate Padlet on teaching pronunciation (including specific language difficulties)

# data[].attributes.body
Note: some of the links below are repeated on this Padlet.

# data[].attributes.headline
Article: Teaching visually impaired students online: six key learnings.

# data[].attributes.subject
Article: Teaching visually impaired students online: six key learnings.

# data[].attributes.attachment_link.readable_text
Teaching visually impaired students online can be challenging for even the most experienced teachers. We asked British Council teacher, Elsa O'Brien, to share what she has learned while teaching a new class.

Some months ago, I was offered a new online English class of visually impaired students. It seemed like the perfect fit for me, because, as well as being a teacher, I am one of the Special Educational Needs Coordinators at the British Council.

I was lucky to have the best handover I could wish for from the previous teachers, who were always supportive. Despite this, I must admit my initial excitement was mixed with a little apprehension. I have always seen myself as a very creative teacher and now, for the first time, I was afraid of improvising and not following the materials to the letter.

As Nigerian author Chimamanda Ngozi Adichie would put it, I was trapped by The danger of a single story. I had very broad and scant information about what visually impairment meant. And, for this reason, instead of having lots of questions about it, I seemed to be unable to picture what a class would be like.

Thankfully, 'when one teaches, two learn' (Robert A. Heinlein). The learning curve was a steep and exciting one. In every lesson, I learned things about my students' lives, their difficulties and strengths and about their goals and how I could help them get there.

Teaching visually impaired students online is probably as convenient as it can be challenging, both for the teacher and the learner. These are the key things I've learned to bear in mind when teaching visually impaired students online:

Visually impairment can mean a lot of different things

I was quite surprised to find out that only one of my students was actually blind. The rest had a vast array of visual impairments with their own specificities. Some students might have an impairment in one eye, some might have reduced vision in both, other students have monochromacy and you might even come across students whose visual impairment might overlap with another kind of impairment such as auditory.

Depending on the origin of the visual impairment, some students' working memory or attention span could also be affected. Having this kind of information is vital to be able to help students reach their full potential.

Talk to your students - they are the best source of information

Unlike other specific learning needs, visually impaired students can be quite used to talking about their needs. They are probably very aware of what these are and very able to explain how you can help them.

As being visually impaired can mean many different things, the support students can benefit from is varied. Knowing whether your students use text-to-voice screen readers, a zooming software, Braille or can read 'touchable ink', can really help when it comes to setting up activities and working out how much time you should allocate to each stage.

Avoid visual resources

Although it might seem like the most obvious tip, nowadays our resources are highly visual to facilitate language acquisition, and we're so used to working in a certain way that it's difficult to change our mindset.

When using resources that have been created for sighted students, think about the adaptations you'll need to make. Will you have to use translation instead of description? Can you replace a speechless video with a more eloquent one?

Be a narrator

You'll have to find the right balance between not overwhelming your students with teacher talking and making sure your voice is there to guide them. You will need to let your students know of things that you're doing that they can't see, such as looking for a file to share, or creating breakout rooms. But more importantly you will have to explicitly welcome students in, even if they're late, provide information of who else is in class, who has their hand up and whether they need to wait for their turn, and who they'll be paired with in breakout rooms.

I think this is one of the most difficult things to get used to doing as sighted people heavily but inadvertently rely on visual cues to understand the world.

You might have to adapt your time expectations

This will largely depend on your students' age and on their level of mastery using a computer, but you must remember that students who use screen-reading software to navigate will need extra time to transition between tasks.

Time is especially important when doing a listening activity. If the students are only using digital materials, they will most likely access them with a screen-reading programme. This means they will need time to listen to the questions before the actual listening activity starts.

During the activity, students won't be able to check the questions as they listen, so it's important to break down the task to reduce the amount of information they need to remember at one time.

Although we should never do this, it's particularly important to make sure that the teacher doesn't speak while students are screen reading a question, as they won't be able to listen to both things at the same time.

Using online game websites

A popular online game website like Bamboozle is easy to use with visually impaired students, as long as the teacher reads the questions out loud. Don't forget to click 'share sound' if on Zoom. Sound effects add to the excitement!

Kahoot claims to be highly accessible for students with different types of visual impairment. However, if you really want to use a new website in class, think about its implications, give it a try and ask your students for feedback before planning a long activity around it.

And have fun

An extra piece of advice, as with any class, is to have fun! Going back to tip number two, involve your students in the learning process, find out about their interests, and choose together what types of activities and games they'd like to include in the lessons.

# data[].attributes.attachment_link.title
Teaching visually impaired students online: six key learnings

# data[].attributes.attachment_link.description
We asked British Council teacher, Elsa O'Brien, to share what she has learned while teaching a new online English class for visually impaired students.

# data[].attributes.headline
Supporting the achievement of deaf children who use English as an additional language (EAL)

# data[].attributes.subject
Supporting the achievement of deaf children who use English as an additional language (EAL)

# data[].attributes.body
deaf children who use English as

an additional language (EAL)

# data[].attributes.headline
Supportive practices and guidance in working with deaf ESOL learners

# data[].attributes.subject
Supportive practices and guidance in working with deaf ESOL learners

# data[].attributes.body
learners

# data[].attributes.headline
Teaching English to Hearing Impaired Learners

# data[].attributes.subject
Teaching English to Hearing Impaired Learners

# data[].attributes.attachment_link.title
Teaching English to Hearing Impaired Learners | Learning Difficulties | EnglishClub

# data[].attributes.attachment_link.description
An introduction to hearing impairment from mild loss of hearing to total deafness, how it affects language learners, and practical strategies for teachers of English and other languages. Author: Tara Benwell

# data[].attributes.attachment_link.readable_text
How do you describe colours to the blind?

Are colours important to them? Are they fashion conscious?

Understanding degrees of blindness

Understanding the background

Setting up a readers service
Teaching tips
I have used talking books and taped dialogues for reading comprehension lessons.

I use real objects in my lessons. I brought in kites when the reading passage was about kites.

I took my students to the kitchen and we had a lesson on how to prepare sandwiches and how to make tea.

I took my senior students to a lower court to record the proceeding of court cases. I was able to do all these things because there were usually only eight to nine students in a class.

Reactions of other students

Is he or she totally blind?

Does he or she have some sight left that he or she may be able to read big print?

Eighty percent of learning is through sight.

How do you teach poetry to them?

How do they conceive concepts?

Can they learn science?

Do you know how to read Braille?

Of all the subjects teaching English to the blind is the easiest. Having said so there are one or two things teachers who are not trained to teach the blind may need to know before they can be an effective teacher to a blind student.

Whatever amount of residual vision that is left of a blind person should be utilised to the maximum. So students with partial vision should be encouraged to read big print books.

Textbooks could never be brailled in time for my blind students. I always had to cope with one chapter at a time. But from my experience I can say that setting up a readers service for the blind is never too difficult. Just spread the word that there is a need for a pool of volunteers to read books into tapes or to blind students and there will be many volunteers.

Blind students may be visually impaired but their other senses are intact, so teachers of the blind should utilise the other senses. Learning a language is very much tied up with culture, exposure and experiences. Blind students may not be able to acquire exposure and experiences the same way as sighted students. So teachers of the blind may have to do a little more than other teachers. Bring experiences and exposure to the blind students.

# data[].attributes.attachment_link.description
How do you describe colours to the blind?Are colours important to them? Are they fashion conscious?How do you teach poetry to them?How do they conceive concepts?Can they learn science?Do you know how to read Braille?Of all the subjects teaching English to the blind is the easiest.

# data[].attributes.headline
Teaching English to Visually Impaired Learners

# data[].attributes.subject
Teaching English to Visually Impaired Learners

# data[].attributes.attachment_link.title
Teaching English to Visually Impaired Learners | Learning Difficulties | EnglishClub

# data[].attributes.attachment_link.description
An introduction to visual impairment from low vision to blindness, how it affects language learners, with useful strategies for teachers of English and other languages. Author: Tara Benwell

# data[].attributes.headline
Resources for Teaching English as a Second Language to Learners with Blindness or Visual Impairment

# data[].attributes.subject
Resources for Teaching English as a Second Language to Learners with Blindness or Visual Impairment

# data[].attributes.body
See also this video.

# data[].attributes.attachment_link.readable_text
As the number of students from non-English speaking backgrounds increases, teachers of students with visual impairment will see a corollary rise in their caseloads of children with language learning needs. The following information may be helpful to share with ESL teachers working with those students.

General Ideas

Perhaps a primary issue is that ESL instruction typically makes heavy use of visual materials such as photos, pictures, movies and videos, flash cards, demonstrations and pantomime. These strategies work very well with sighted learners because they connect the language with concrete, real-life examples.

The challenge is that these strategies rely on materials that are presented at a distance from the learner and are accessed though vision and hearing. Individuals who are blind, however, require hands-on materials that can be explored by sense of touch; distance materials by themselves will not provide much meaningful information.

There are accommodations in instruction that can be helpful. For example, students can use "real" objects, for example an actual pot or pan instead of a picture of a pot or pan, when learning vocabulary for cooking. By touching and exploring the object, the student forms a connection between the word and the object. This is approach is actually one that will work well for all students, regardless of degree of vision or blindness, in learning the language. For objects that are too large fit into a classroom, such as a car or bus, miniatures can be used. However, learning will be enhanced if the miniature is initially paired with a visit to the parking lot to experience an actual car or bus. This type of "real world" experience is also a great way to provide meaningful, context-rich educational experiences.

In regards to the written form of English, it is essential that the student have some access to braille, either with a braille writing machine or a slate and stylus. It is extremely difficult to learn the written rules of any language through listening only. Again, there must be a hands-on, direct experience with the letters and words. There is a braille code for most major languages and while these may be quite different, the differences are no more difficult to learn than in learning the visual writing systems. If a student is fluent in the braille code to his or her first language, he or she should be able to begin learning the code in English.

Internet Links

Sharon Monthei works at BLIND, Incorporated, one of the three training centers operated by the National Federation of the Blind, where she teaches English to refugees and immigrants.

Chock Sing answers questions about teaching English asked by people who have no experience working with the blind.

Chen Guangcheng describes his experiences as a blind learner of English as a second language.

From the ESL website "English Club", info about teaching English to the visually impaired (VI). You will find a brief description of different types of VI learners as well as some challenges of and tips for teaching them.
Paths to Literacy: English Language Learners
Website devoted to instructional strategies for learners who are blind or visually impaired which has a number of articles on English as a Second Language.

English Language Learners

Books

A couple of books on braille instruction from American Foundation for the Blind address the needs of the blind ESL student. They can be ordered from American Printing House for the Blind or from Amazon.

Instructional Strategies for Braille Literacy. AFB Press. Chapter 6: "Teaching Braille Reading and Writing to Students Who Speak English as a Second Language".

Beginning with Braille. AFB Press. "Teaching Braille to English Language Learners" (pp. 302-309)

# data[].attributes.attachment_link.title
Resources for Teaching English as a Second Language to Learners with Blindness or Visual Impairment - Paths to Literacy

# data[].attributes.attachment_link.description
Paths to Literacy Resources for Teaching English as a Second Language to Learners with Blindness or Visual Impairment. For teachers, families, and others interested in literacy for children and youth with visual impairments

# data[].attributes.body
Resources to support ESOL leaners with hearing and sight impairments.

# data[].attributes.body
Educoot

# data[].attributes.body
Skillsworkshop

# data[].attributes.attachment_link.readable_text
Five scenarios and tasks designed to practice for the discussion part of the speaking exam (NCFE task 2; new spec).

Topics include watching sport, fitness clubs, life skills, transferable work skills and alternative school provision (learning at home and in the community).

Is desired, the sheet can be cut up into 5 different tasks to use on different occasions / with different groups.
Editor's note

# data[].attributes.attachment_link.description
Five scenarios and tasks designed to practice for the discussion part of the speaking exam (NCFE task 2; new spec). Topics include watching sport, fitness clubs, life skills, transferable work skills and alternative school provision (learning at home and in the community).

# data[].attributes.headline
ESOL Resources (Listening and Speaking)

# data[].attributes.subject
ESOL Resources (Listening and Speaking)

# data[].attributes.attachment_link.title
LibGuides: FET Resources for Ukrainians in Ireland: ESOL (Listening and Speaking)

# data[].attributes.attachment_link.description
LibGuides: FET Resources for Ukrainians in Ireland: ESOL (Listening and Speaking)

# data[].attributes.body
ESOL Literacy Workbook - SAMPLE UNIT Courtesy of Dave Butler

# data[].attributes.attachment_link.title
Read and Write every day Workbook sample unit (1)

# data[].attributes.body
News lessons, at three levels, are sourced from articles published by The Guardian. They include downloadable worksheets and teacher's notes for you to use in class.

# data[].attributes.body
On National Archives site

# data[].attributes.attachment_link.description
This Page is [ARCHIVED CONTENT] and shows what the site page http://www.dcsf.gov.uk/curriculum_preentry/progressionxcurr/1/ looked like on 15 Apr 2010 at 08:08:40

# data[].attributes.attachment_link.readable_text
This course was originally written by Christopher J Hall with Rachel Wicaksono in 2013, and fully updated and revised in 2019 and 2023. The original version was supported by a British Council ELT Research Partnership award and a grant from the York St John Business School.

The course is an output from the ELT Research Partnership award scheme funded by the British Council to promote innovation in English language teaching research. The views expressed are not necessarily those of the British Council.

This work is licensed under a Creative Commons Attribution 4.0 International License. You are free to download all material in MS Word format here.

# data[].attributes.attachment_link.title
About Changing Englishes | An Online Course For Teachers

# data[].attributes.attachment_link.description
The course is an output from the ELT Research Partnership award scheme funded by the British Council to promote innovation in English language teaching research

# data[].attributes.headline
Argument essays, graphs, other writing, and speaking for IELTS, PET, and TOEFL

# data[].attributes.subject
Argument essays, graphs, other writing, and speaking for IELTS, PET, and TOEFL

# data[].attributes.attachment_link.readable_text
Welcome to Writefix.com! The site is aimed at people who are learning English and taking the IELTS or TOEFL exams in writing or speaking. Learn about IELTS essay writing here! There are several parts in this site:

How to write Argument and Opinion IELTS essays and lots of example essays

Writing about Graphs and other essays, including Problem and Solution/Cause and Effect

Speaking test practice: IELTS Speaking and PET Speaking

Your IELTS essays! Hundreds of IELTS essays and comments by test-takers like you.

There are also many other links and activities. We hope you enjoy looking around!

IELTS: Parts 1, 2 and 3 of the IELTS Speaking test, with topics and tips

PET: Practice talking about a picture. Time yourself!

Thanks for dropping by!

Tell us if something is missing!

Try the new SpeakingFrog.com website: videos of speeches, with transcripts. Read while you watch! Good for longer IELTS Listening e.g. Part 3 and 4

Should college students stay at home? (very short version)Many university students live with their families while others need to live away from home because their universities are in different cities. What are the advantages and disadvantages in both situations?

Is ambition positive or negative?Is ambition a positive or negative characteristic? Is is necessary for success?

Who is valued most in society - old or young? (Short version)In some countries old age is highly valued, while in other youth is emphasized. Which viewpoint do you agree with?

Who is valued most in society - old or young? (Long version)In some countries old age is highly valued, while in other youth is emphasized. Which viewpoint do you agree with?

Should college students stay at home? (Very long version)Many university students live with their families while others need to live away from home because their universities are in different cities. What are the advantages and disadvantages in both situations?

From the forums...

See other people's IELTS essays and comments!

# data[].attributes.attachment_link.description
Welcome to Writefix.com! The site is aimed at people who are learning English and taking the IELTS or TOEFL exams in writing or speaking. Learn about IELTS essay writing here!

# data[].attributes.attachment_link.readable_text
IELTS preparation involves understanding the test format, how it is marked and then making a plan to achieve your desired score. To do this, you will need to improve your English, practice test skills and join a preparation course to check your progress before registering for the test.
How to Prepare for the IELTS Test

Below is a quick checklist to help with your IELTS preparation:

Understand the test format

Know how the test is marked

Surround yourself with English

Make a plan of action

Join a preparation course

Practice with sample questions

Perfect your IELTS skills

Check your progress

Register for the test

Join the Best IELTS Preparation Course.
IELTS Preparation: Pass 3x Faster

One of the most popular questions I get on my YouTube Channel is:

Chris, how can I prepare for my IELTS exam quickly?

Truthfully, the answer to this question will be different for every person reading this article, which makes it difficult for me to give one solution that will help everyone.

However, I can tell you how to prepare for your exam PROPERLY!

Find out how to improve your preparation and accelerate your IELTS journey here:

Why is IELTS Preparation So Difficult?

Quite simply, so much information is out there, and students do not know which sources they can trust. IELTS can take months (sometimes years!) of hard work and thousands of dollars in tuition and test fees, so there is a lot of pressure to get it right the first time.

There are three main things you need to improve:

General level of English (Slow)

Test skills (Medium)

Knowing how the test is marked (Fast)

Improve Your General Level of English

IELTS is essentially an English language test; therefore, the higher your level of English, the better your score should be. Simply studying IELTS skills and doing practice tests will not be enough if your level of English is not high enough.

As indicated above, this is probably the slowest part of your preparation. Most English schools recommend at least 6 months to improve a student's level of English by the equivalent of 0.5- 1 band score.

I will show you how you can improve your general level of English at home below.

Perfect Your Test Skills

There are four parts to the IELTS test- Writing, Speaking, Reading and Listening. Each of these parts has many different possible questions, each of which has a specific skill you need to acquire.

For example, academic students can expect to see one of seven different types of questions in Task 1 Writing. Each of these different types of questions requires differing skills.

The Listening and Reading tests have over 10 different types of questions each. Again, all of these questions require a different strategy and set of skills.

Most of your IELTS preparation should be spent learning these skills, and when these are combined with a high level of English, it should result in a high score.

Learning these skills does not take as much time as learning the language itself, but does take a significant amount of time. Most schools recommend spending 2-3 months learning these skills.

I will address how to improve these at home below.

Know How the Test is Marked

The fastest and most effective way to improve your score is knowing exactly what the examiners want and giving it to them. Most of my teachings are based on this principle.

However, you can't simply learn this and get a high score; they should be combined with improving your level of English and test skills.

Below we will look at the official marking criteria to help us understand what the examiner wants.

Step-by-Step Approach to IELTS Preparation

1. Decide if you are doing General Training or Academic

2. Understand the Test Format

3. Set Realistic Goals

The keyword here is 'realistic'. There is a big difference between the score you want and the score you will get.

Effective preparation will help you succeed and ensure you get the best score possible, but it does not ensure miracles.

Click here to read our article: How Long Will it Take Me to Get the Score I Need?

4. Understand Marking Criteria

The article below will explain the band scores, how examiners decide them and the official marking criteria used to mark your test. Click the link below.

5. Understand the Different Question Types

There are more than 10 different types of questions for Reading and Listening. Again, there are more than 10 different kinds of questions on the Writing test.

Knowing these will give you a huge advantage because each of them requires a different approach and strategy. I have them all for you in each of the skills sections below.

6. Perfect Your IELTS Skills

As mentioned above, there are many different approaches and strategies that you need to succeed in IELTS. For example, do you know how to write an effective introduction and conclusion? Do you know how to develop your answers in the speaking test? Do you know how to quickly locate the correction information in the reading test?

All of these skills will be addressed below.

7. Improve Your Vocabulary

Vocabulary is a huge part of the IELTS test. It covers 25% of your total mark in Speaking and Writing. It is also tested in the Reading and Listening tests.

You should implement a vocabulary improvement plan as quickly as possible. Click below to download one now.

8. Practice English Every Day

When it comes to improving your English, there really is no substitute for practising a little every day.

There are hundreds, if not thousands, of ways you can practice every day, and it doesn't have to be doing boring IELTS tests. My most successful students found something they enjoyed doing in English and then did this regularly. The more you use English; the more your skills will improve and the higher your chance of getting the score you need.

Below are lots of ways to improve your English at home:

You can also combine learning English with the most common IELTS topics. Some topics, such as the environment, education, and technology, come up again and again. Reading and listening to these common topics is a very powerful technique that will help you improve your vocabulary and your ideas while improving your English.

Below you will find links to the most common IELTS topics.

9. Practice Tests

You should do IELTS practice tests to establish your current band score and familiarise yourself with the test.

However, there are two things that I should warn you about before doing these.

The first thing is that there are lots of fake tests. These tests can be found online or in your local bookstore. Where I live, there are hundreds of books in my local store, all claiming to offer 'official' tests. The problem with this is that they are often written by people who know nothing about IELTS and are simply looking to make a few dollars.

Fake tests can be very misleading and often prevent students from understanding the real test.

You should only do tests from official and trustworthy sources. The best sources of past papers are the Cambridge Past Papers books.

Below are other reliable sources of practice tests:
British Council IELTS Essentials
The second thing I want to warn you about is not making these the only part of your IELTS preparation. Lots of students that I know do IELTS practice tests all day, every day, and most of them improve very little. They should only be used as a test of your current ability. You should spend most of your time improving your level of English and your IELTS skills.

If you are practising all week, you should only be really doing 1 or 2 practice tests. In other words, they should be a very small but important part of your preparation.

You'll find a full list of practice study materials in our resource guide and our practice guide.

10. Get Your Speaking and Writing Assessed

You must get a qualified teacher to assess your speaking and writing. They will be able to tell you your current level, but more importantly, they will be able to tell you what your weaknesses are.

This is probably the most important part of your IELTS preparation. If you do not know your weaknesses, you are wasting a lot of time because you have no idea what to focus on. You must focus on things you are not good at to improve.

The most effective strategy for IELTS preparation is to find out your weaknesses, go away and work on them and then come back and have your work assessed by an experienced IELTS teacher. They can then tell you if you have improved and advise you on what to focus on next.

Please click the link below if you want me to assess your writing.
Essay Correction Service
How will I know that I'm ready for the IELTS exam?

Hundreds of thousands of students fail the IELTS exam each year for one simple reason:

They took the test before they are ready for it!

In my experience, there is only one question you need to ask yourself to decide whether you're ready for the IELTS exam.
Find out what it is here.
IELTS Writing Task 2 Preparation

Writing Task 2 Skills

Below are links to complete guides on the essential Writing Task 2 skills.

Full Lessons

Below you will find complete lessons on each of the different Task 2 question types.

Sample Answers

Below you will find a link to lots of Band 9 sample answers. Use these to compare and assess your writing.

Summary

IELTS Writing Task 1 (Academic) Preparation

Writing Task 1 Lessons

Below are links to complete guides on the essential Writing Task 1 skills.

IELTS Writing Task 1 (General) Preparation

Below you will find all the help you need to prepare for Task 1 if you are doing General Training.

IELTS Speaking Preparation

Part 1

In part 1, you will be asked questions on familiar topics. Click below for lots of sample questions and answers.

Part 2

This test part allows you to speak for longer on a given topic.

You will be given a card with a particular topic on it, including key points that you should discuss.

You will be given one minute to plan, and then you will talk for between 1-2 minutes.

Below are some tips that I give to all of my IELTS Speaking classes:

Part 3

In part 3, the examiner will ask further questions linked to the topic discussed in part 2.

This part of the test is designed to allow you to talk about things more abstractly.

IELTS Speaking Skills

Below you will find several posts designed to help you improve your IELTS speaking exam technique:

IELTS Speaking Criteria

You must understand the criteria examiners use to assess your speaking. Here is a guide:

IELTS Reading Preparation

There are several different questions on the IELTS reading test, and each has its own problems and strategy. Click on any of the links below for a full step-by-step guide.

IELTS Listening Preparation

Below is a list of resources to help you prepare for the IELTS listening test at home.

Test Day Advice

You are now ready for your test, and your IELTS Preparation is complete. Here is a checklist to ensure you do your best on test day:

Get to bed early. You'll need lots of sleep.

Eat a good breakfast. Eat more than you normally do so that you won't get tired and hungry. Protein and fat are better than simple carbohydrates.

Wear comfortable clothes. This isn't a time for showing off your fashion sense.

Don't forget to bring your ID and essential documents.

Make sure you know where the test centre is and how to get there.

Get to the test centre early. Do not throw away all of your IELTS preparation by being late.

Find out where everything is, including your test room and the toilets.

Bring some water and some sugary food or drinks. Don't drink too much because bathroom breaks will take up time.

Do not panic. Stress is normal. Use that nervous energy to focus, but do not allow it to control you.

Do not hesitate to ask the staff for help if you need anything. They are there to help you.

Action Plan

Now that you know what to do, it's time to make a study plan.

Everyone will use do this differently; however, below are some of the things you need to consider when making your study schedule:
1. The date of your test:
How many days/weeks before your test?

Is it flexible, i.e. can you change the date?
2. How much time do you have per day to study?
You should aim to be able to study in peace and totally focus on what you are doing.

Do you have more time on particular days, such as Saturday and Sunday?

What other commitments do you have?

Are there any commitments you can stop doing until you take your IELTS test? This will free up a lot of time.

You should be realistic about this. Be honest with yourself.
3. What is your current English level?
The higher your general level of English, the faster you can complete this course.
4. What is your learning style?
Can you normally understand everything the first time, or do you have to go away and think about it alone?

Do you prefer to study with other people or by yourself?

I recommend sitting down with a calendar and filling out all of the days you want to study with the things you need to do above. This will help you stay organised and reduce your stress about preparing for the test. This will also give you an idea of how much work you will have to do each study day.

You can do this with a paper calendar or use one of the many online apps.

The more organised you are now, the easier your preparation will be.

IELTS Band Scores

Full IELTS Band score calculators are available here.

We are very lucky to have many successful students who have shared their IELTS stories with you.
Click here to watch them now.
Below you'll find testimonials from just some of our successful students.

From Failure to Band 8.5 in Just 1 Month

I am very proud of Sana. She failed to get the scores she needed the first time but kept going and got an amazing Band 8.5 overall.

She was kind enough to share her experience with you all in a detailed video.

I crowdsourced the questions from you guys, so thanks very much for that. In the video, we look at:

- How to identify your areas of weakness.
- How to boost your level of English.
- How much time did it take to reach 8.5?
- Thoughts on the 'perfect' essay.
- How to cope with stress.
- + her advice for listening, reading, speaking and writing.

How This Doctor Went from Band 5.5 to 7.5 in IELTS

Doctor Aidrous is one of the students I am most proud of. He worked extremely hard to get the score he needed and moved from 5.5 to 7.5.

He showed up to all of my classes and steadily improved with each session.

He has kindly agreed to share his experience and inspire you.
Click here to watch his video now.
From IELTS Band 6.5 to Band 7 in 2 Weeks

Daniela was in the same position many of you are in. She got 6.5 in IELTS Writing but needed a 7.

She tried going to a big IELTS school in her hometown but didn't get the help she really needed.

In just 2 weeks, we helped her get to a Band 7 level.

She kindly shared her experience with you in this video and gave great advice to anyone struggling with their writing.
Click here to watch her success story now.
How Crystal Achieved Band 8 After Failing her IELTS Test Many Times

I am proud of all my students, but I have to admit I have a soft spot for students who show massive amounts of perseverance, determination and consistency.

Crystal achieved a Band 8 overall after trying and failing many times.

She was kind enough to share her story with you all. In this video, we look at:

- How she identified her writing weaknesses.
- How perfectionism was ruining her speaking performance.
- How she achieved a Band 9 in Listening.
- How she improved her vocabulary.
Click here to watch it now.
IELTS Preparation Courses

All of the students above attended one of my IELTS preparation courses.

You are probably thinking, 'What makes you different?'

In a nutshell, our student's success is our success, so we strive to create the best IELTS courses and services globally.

What also makes us different is our unique system.

We first help you identify the exact reasons you are not getting the score you need. We do this by looking at your work and giving you feedback.

We then teach you how to fix and turn weaknesses into strengths. We do this through our course video platform.

You will then put what you have learned into practice, and we will continue giving you personalised feedback on what you are currently not doing well enough.

We will help you refine and fix any problems until you can do everything effectively. You are then ready for the test.

For more information about our IELTS Preparation courses, please visit our VIP Academy. If you'd like to hear what our students have to say, you should read our VIP Academy Reviews. If you have any questions about joining, please feel free to email me- chris@ieltsadvantage.com

Related Articles
How To Prepare For The IELTS Test Does Teacher Ability Affect Your Test Score How To Prepare for IELTS At Home IELTS Band Score Calculator IELTS Syllabus
(Click to expand answers)

IELTS Preparation FAQs

# data[].attributes.attachment_link.title
IELTS Preparation- ✍️ Everything You Need to Know

# data[].attributes.attachment_link.description
Discover your one-stop resource for IELTS preparation, filled with effective strategies, tips, and techniques to elevate your score.

# data[].attributes.attachment_link.readable_text
Welcome to my Free IELTS TEST PREPARATION WEBSITE: containing IELTS tips, model answers, topics, practice lessons, and videos created from many years of experience teaching IELTS. My name is Liz and I am the author and teacher of this website.

Please read this whole page carefully to learn about IELTS and learn how to use this free website for your preparation. Most of the lessons and tips are for people aiming for IELTS band score 7, 8 & 9. Although, people who only need band 6 can also benefit.

IELTS = International English Language Testing System. It is the world's biggest English language test with test centres in most countries. This means it is a test which accepts international English, not only British English.

There are four parts to the IELTS test:

IELTS Speaking - a face to face informal test with an examiner which lasts between 11 and 14 mins and is made up of 3 parts. All IELTS candidates take the same test.

IELTS Listening - a 30 minute audio recording with 40 questions. All IELTS candidates take the same test.

IELTS Reading - 3 reading passages with 40 questions which lasts for 1 hour. This test is divided into Academic and General Training papers.

IELTS Writing - this is divided into task 1 and task 2. Task 1 Academic is a short report of a chart, graph, map table etc. Task 1 GT is a letter. Task 2 Academic & GT is a formal essay. IELTS writing lasts for 1 hour in total.

For more information about band scores, GT IELTS, computer IELTS etc see the TEST INFO button on the RED MENU BAR at the top of the website. The Test Info FAQ page will give you information about using capital letters, using highlighter pens etc - make sure you read that page.

At present, there are over 300 PAGES OF FREE IELTS LESSONS & TIPS on this site. All pages are accessed lower down this page or through the RED MENU BAR at the top of the website. You will find IELTS model answers, lessons, videos, tips, topics, vocab, band score info, etc all for free to help you prepare at home for your test.

You can also find more information in the comments section of many pages. I am not able to answer all comments posted to me, but you can learn from the answers I have already provided to others.

How to Prepare for IELTS

Watch the video to learn the essential steps to prepare successfully for your IELTS test. When you prepare for IELTS, there is no particular place to start. Just start where you want to start. This video will help make sure you cover all the vital aspects of preparation before your test.

.

If you prepare in the right way, you will be able to maximise your results. Having both good English and a clear understanding of the IELTS test is essential. Be clear minded and strategic in your approach and training.

Free IELTS Tips & Practice Lessons

I have prepared IELTS tips, practice lessons, model answers and topics for you to use for FREE at home. Click below to start learning.

-----------
FREE SUBSCRIBE to get New Lessons & Tips by Email
-----------

ADVANCED IELTS E-books & Lessons

Professionally designed IELTS lessons for serious candidates who need a high score in Writing Task 2. By IELTS Liz.

IELTS Grammar E-book

My Grammar E-book is now available to buy in my online store. Click here: Visit Store

# data[].attributes.attachment_link.title
IELTS Liz – IELTS Preparation with Liz: Free IELTS Tips and Lessons, 2023

# data[].attributes.attachment_link.description
IELTS Preparation with Liz: Free IELTS Tips and Lessons, 2023

# data[].attributes.headline
Grammar Practice Activities Paperback with CD-ROM :A Practical Guide for Teachers

# data[].attributes.subject
Grammar Practice Activities Paperback with CD-ROM :A Practical Guide for Teachers

# data[].attributes.body
Cambridge Handbooks for Language Teachers

# data[].attributes.attachment_link.readable_text
A Practical Guide for Teachers

🚚 Free UK delivery on books (excluding sale). T&Cs apply.

Free click & collect on all orders.

First published in 1988, Grammar Practice Activities is a language teaching classic. This revised version combines tried and tested activities with up-to-date content including a CD-ROM. This is an invaluable resource for any teacher who wishes to combine grammar teaching with a broadly communicative methodology. The book contains a collection of imaginative and interesting grammar practice activities suitable for a wide range of levels and ages. Clear teaching notes accompany each activity ensuring they are easy to use. In addition, the accompanying CD-ROM provides print-ready photocopiable worksheets which can be put to immediate use in the classroom.

# data[].attributes.attachment_link.description
First published in 1988, Grammar Practice Activities is a language teaching classic. This revised version combines tried and tested activities with up-to-date content including a CD-ROM. This is an invaluable resource for any teacher who wishes to combine grammar teaching with a broadly communicative methodology. The b

# data[].attributes.headline
New home for the Excellence Gateway teacher resources. 

# data[].attributes.subject
New home for the Excellence Gateway teacher resources. 

# data[].attributes.attachment_link.description
The Education Training Foundation online learning classes

# data[].attributes.headline
Understanding ESOL Skills for Life qualifications

# data[].attributes.subject
Understanding ESOL Skills for Life qualifications

# data[].attributes.body
Assessment properties, comparability, core curriculum and CEFR alignment

# data[].attributes.attachment_link.readable_text
Authors

Acknowledgements

We would like to thank the many people without whose input this study would not have been possible:

all our participants, who devoted a lot of their time and enthusiasm to work on this study and share their expertise and opinions

colleagues at Ofqual who have helped in different ways (with IT support, admin, discussions and analytical support, and various ad hoc and last-minute requests for help)

List of acronyms

AO - Awarding Organisation

AC - Assessment criteria

CBA - Competence-based assessment

CEFR - Common European Framework of Reference for Languages

CJ - Comparative judgement

EAL - English as an Additional Language

EFL - English as a Foreign Language

ESFA - Education and Skills Funding Agency

ESL - English as a Second Language

ESOL - English for Speakers of Other Languages

FSE - Functional Skills English

GCR - General Conditions of Recognition

GLH - Guided learning hours

IELTS - International English Language Testing System

LO - Learning outcome

MFL - Modern Foreign Language

NARIC - National Academic Recognition Information Centre

NSAL - National Standards for Adult Literacy

NQF - National Qualifications Framework

NVQ - National Vocational Qualification

SFL - Skills for Life

QCA - Qualifications and Curriculum Authority

QCF - Qualifications and Credit Framework

RQF - Regulated Qualifications Framework

SELT - Secure English Language Test

VTQs - Vocational, technical and other general qualifications

List of tables

Table 1 Proportions of ratings characterising assessments as fully (Y), mostly (M), partly (P) or not appropriate (N) - Entry 1

Table 2 Proportions of ratings characterising assessments as fully (Y), mostly (M), partly (P) or not appropriate (N) - Entry 3

Table 3 Proportions of ratings characterising assessments as fully (Y), mostly (M), partly (P) or not appropriate (N) - Level 1

Table 4 Content mapping at component level - Entry 1

Table 5 The CEFR scales represented in SfL components - Entry 1

Table 6 Content mapping component levels - Entry 3

Table 7 The CEFR scales represented in SfL components - Entry 3

Table 8 CEFR levels and sub-levels used in standard linking

Table 9 Numerical rating scale categories - CEFR sub levels

Table 10 Task score ranges - CEFR sub-levels

Table 11 Task score ranges - CEFR levels

Table 12 Example frequency table

Table 13 ICCs based on initial ratings

Table 14 ICCs based on final ratings

Table 15 Descriptive statistics for task sub-level ratings by test

Table 16 Indicative CEFR mapping

Table 17 Number of CEFR benchmarks by level

Table 18 SfL script sample by level

Table 19 Key features of the judging allocation design

Table 20 SSR and separation coefficients

Table 21 Overall model fit

Table 22 Original mark (band) correlation with script measure - benchmark scripts

Table 23 Script score-measure correlations - SfL scripts

Table 24 CEFR benchmark descriptive statistics by level

Table 25 Descriptive statistics for each AO script sample by level

Table 26 Relationship between SfL pass grade standard, ESOL exemplars and the CEFR

List of figures

Figure 1 Box plots showing task sub-level rating distribution by test

Figure 2 Comparability between CEFR benchmarks

Figure 3 SfL sample borderline script check - AO1

Figure 4 SfL sample borderline script check - AO2

Figure 5 SfL sample borderline script check - AO3

Figure 6 SfL sample borderline script check - AO4

Figure 7 Comparability between SfL scripts by level

Figure 8 Box plots showing the SfL standards mapping onto the CEFR

Executive summary

Background

In the corporate plan 2019-2022, Ofqual committed to considering the effectiveness of the regulatory requirements relating to qualifications covering English for speakers of other languages (ESOL).

This study focuses on the ESOL Skills for Life (SfL) qualifications. These were developed as part of a suite of basic skills qualifications to help migrants resident in this country to integrate better with English-speaking communities and to access education and employment, helping them to fulfil their potential.

The SfL qualifications were accredited by Ofqual in 2014, against the 2011 ESOL criteria and the rules of the Qualifications and Credit Framework (QCF). They must meet the requirements of the National Standards for Adult Literacy (NSAL) (QCA, 2000) and demonstrate a clear relationship to the Adult ESOL Core Curriculum (DfES, 2001). They are government-funded qualifications, offered at levels Entry 1, Entry 2, Entry 3, Level 1 and Level 2 of the Regulated Qualifications Framework (RQF), delivered by several awarding organisations (AOs).

The SfL qualifications operate in a challenging context of language assessment for migration and integration. The challenge is the sheer variety of the experience of the target cohort for these qualifications in terms of literacy levels, language and educational backgrounds, life histories including trauma and other vulnerabilities, aspirations and needs within the UK society. This often calls for flexibility in delivery and assessment approaches. On the other hand, employers, education providers and other users may rely on the results of these qualifications and will need to know there is sufficient comparability between different awarding organisations (AOs).

Study aims and research questions

The purpose of this research was to gain an understanding of how effective our current regulatory requirements are for these qualifications in making sure they function as intended in relation to their purposes and to inform decisions on our requirements in future. We explored and evaluated a sample of SfL qualifications in terms of their design and assessment properties, alignment with the core curriculum, inter-AO comparability and validity. Specifically, we wanted to develop a clearer understanding of:

The research involved a sample of 4 AOs, which account for a large proportion of the certifications in ESOL SfL, with 2 additional AOs included in interviews. Due to resource and time limitations, we focused on a subset of qualification levels. The research was conducted in several strands:

Key findings

This report brings together the findings from our 4 studies and suggests some potential areas for improvement. This section is organised according to the key themes that emerged across different studies or from our background research.

Issues of recognition

It is clear from the views expressed in our interviews with AO staff, as well as from the literature, that the SfL qualifications are seen by practitioners as having enormous value to learners and society as a whole. The AO staff saw the SfL qualifications as similar in their enabling function to ESOL International qualifications. Some of these are widely recognised, at competence levels similar to Entry 3, Level 1 or Level 2, by reputable higher education institutions or for job recruitment.

In contrast, our interview participants felt that the potential of SfL qualifications to enable learner progression was not sufficiently recognised. In our interviews, as well as in the relevant literature, there was also a sense that these qualifications are undervalued in terms of funding. The AOs told us that this affected user attitudes towards the qualifications, including centres, employers, higher education institutions, and learners themselves.

Multiple qualification purposes

The views expressed in our interviews as well as in the literature suggest that the SfL qualifications potentially have multiple purposes and there is insufficient clarity about their relative importance to different users (for instance, learners, centres or employers). Their purposes are currently not clearly articulated in Ofqual ESOL criteria.

Using Newton's (2017) perspectives framework for thinking about assessment purposes, some of SfL qualification purposes can be seen from an engagement perspective, helping to ensure that learning actually occurs, inasmuch as they are seen as valuable for (linguistic) integration of migrants and for removing barriers to accessing employment or education. Given their required alignment with the core curriculum, it appears that some of the purposes relate to the expertise perspective, providing evidence of a level of mastery of the specified content domain, as well as boosting confidence and motivation to build on skills and expand the expertise within the domain. The SfL qualifications are seen as both socially and personally valuable in relation to these 2 perspectives. What is less clear, based on the issues with recognition of these qualifications for progression to employment or education, is how far they have (or could have) purposes related to information perspective that are valuable to stakeholders beyond learners or qualification providers as the basis for making any decisions.

This situation may indirectly impact on overall quality and appropriateness of provision and reduce the extent to which any of the intended purposes are truly fulfilled, potentially reducing the usefulness of these qualifications to users. It is important to seek clarity about the intended purposes of these qualifications and their relative importance. This is necessary to enable coherent considerations of the most appropriate assessment models, as well as approaches to regulation, and any trade‑offs.

Assessment models

A wide range of assessment models and approaches are used across the 4 AOs in our sample. Some of the differences in approaches appear to stem from a desire to offer flexibility of choice to learners and centres. The AOs in our interviews considered the flexibility of the SfL qualification structure, its modularity and the variety of assessment models (such as an option to use internal or external assessment at all levels) as beneficial and justified by the wide range of learner and centre contexts.

Our assessment review, however, highlighted quality issues with some of the assessment approaches and assessment instruments. In some cases, the assessments were not entirely appropriate for the construct of assessment or were poorly executed in practice. The AO interviews suggested that some of the assessment design choices were historical and not always reflective of best practice in language assessment but AOs were not always able to provide a clear rationale for them beyond citing original requirements that had not been updated.

Curriculum alignment

Our reviewers thought that the curriculum was largely fit for purpose, sufficiently clear and indicative of the appropriate qualification levels. However, the review suggests that the SfL qualifications may not be sufficiently well aligned with the curriculum. This was primarily in terms of demand, aspects of coverage (for instance, narrowing of the curriculum) and interpretation of the curriculum messages.

At Entry 3 and Level 1, the demand of assessments and pass grade standards were deemed to be too low in most cases except in speaking. Entry 1 was seen as broadly appropriate, if occasionally too high in relation to the curriculum standards. The overall quality and appropriateness of coverage and relative demand of the assessments seem to decline from Entry 1 to Level 1 for most of the AOs in our sample, with insufficient stretch at higher levels, suggesting threats to appropriate candidate progression.

CEFR alignment

The CEFR describes second language ability in terms of a 'can do' scale that reflects increasing complexity and range of language competence from basic user (A1, A2), independent user (B1, B2) to proficient user (C1, C2) (see Appendix 1 for the CEFR global scale descriptors). The indicative CEFR content mapping produced in our studies suggests there is meaningful alignment of the SfL qualifications with the CEFR in many respects, and thus also with ESOL International qualifications.

The CEFR content mapping placed Entry 1 reading and writing components at A1/A1+ level. Entry 1 speaking was judged to be at A1+/A2 level. For writing, the standard linking exercise additionally indicated that the pass grade standard is at about A1+ level. However, there is a separate question as to whether A1+ level or higher is suitable for an initial Entry level qualification (even if it is deemed appropriate according to the curriculum). Given that Entry 1 pass grade standard appears to be significantly higher than threshold A1 CEFR level at least in writing, this may suggest a potential need for separate qualifications at somewhat lower level ('pre-entry') for learners who are starting from the point of no English at Entry 1.

Entry 3 components were judged in the content mapping to be at B1/B1+ level. However, the results of our standard linking study for reading comprehension suggest that actual pass grade standards of some exams are lower, at A2+, and in all cases closer to B1 threshold level than B1+. Similarly, the standard linking for the writing components indicated that the pass grade performance standard at Entry 3 is A2+/low B1 at most, while Level 1 was mapped to B1+/low B2.

Perhaps the most significant finding from the CEFR standard linking study for writing is that there does not appear to be enough differentiation between higher levels of the SfL qualifications at the pass grade threshold, particularly between Entry 2 and Entry 3. Both of these levels were mapped onto A2+ CEFR level for some qualifications. This is also the case to some extent between Entry 3 and Level 1 writing qualifications. Thus, the findings from the CEFR standard linking largely confirm the findings from the curriculum review, namely, that Entry 3 and Level 1 SfL qualifications may not be sufficiently demanding given the demand implied in the curriculum and relative to the demand of the lower levels. This lack of stretch at Entry 3 in particular may be contributing to the perceived "large jump" in standards at Level 1, which was noted by our curriculum reviewers.

The CEFR content mapping highlighted the discrepancies with the CEFR in relation to assessment of literacy in SfL qualifications, for example, assessment of Latin alphabet knowledge, emphasis on orthographic accuracy at Entry 1 or emphasis on awareness of the writing process at Entry 3, which could be seen as too advanced for learners at those levels, particularly in the migration context. Such requirements in the domain of literacy in SfL qualifications are likely related to their link to the NSAL, which are primarily targeted at learners for whom English is their first language. This potentially raises questions about whether mapping qualifications such as SfL, which is an English as a second language qualification, to standards such as NSAL, intended for English as the first language qualifications (for instance, Functional Skills) is entirely appropriate.

Comparability of pass grade standards

Pass grade standards in Entry 3 reading comprehension assessments were broadly comparable and at around threshold B1 level for 3 out of 4 AOs. For one AO, the pass grade standard was mapped to A2+, and is thus lower than the rest. This is despite the overall profile and demand of the tests from the 4 AOs being reasonably aligned.

Similarly, some comparability issues were noted in our CEFR linking study of pass grade standards in writing, at Entry 1 level. All 4 AOs, however, were mapped to A1 level on the CEFR, despite some discrepancies at the pass grade within this level. These comparability issues, therefore, are of somewhat less concern than those identified in reading comprehension.

Potential improvements

In light of these findings, Ofqual (and other relevant stakeholders where appropriate, for instance, the DfE) should consider how the qualifications could be improved to address some of the issues identified. Some areas for improvement are listed below:

The range, relative priority, and interactions of SfL qualification purposes should be clarified. Given the complexity of SfL learner needs and the importance of engagement-related purposes of these qualifications, revisiting the trade-off between these and any information-related purposes seems particularly important as a starting point. This should help to inform the choice of the most appropriate assessment models and any trade-offs between the extent of standardisation and flexibility in assessment design and delivery.

There are quality and validity issues in some individual approaches and assessment instruments, which need to be addressed. In addition, appropriateness of approaches that may be historically motivated rather than meaningfully linked to qualification purposes and constructs should be reconsidered. Assessment approaches should be sufficiently justified in terms of validity and best assessment practice.

The way the curriculum is covered and interpreted in some assessments may need to be improved to reflect its intention and implied demand.

Insufficient comparability between Entry 3 reading comprehension pass grade standards should be addressed if comparability were deemed important for certain purposes of SfL qualifications. Wider checks on comparability (between other components, levels and AOs) may also be warranted depending on the extent of comparability required.

Differentiation and progression between SfL qualification levels should be improved to ensure optimal progression for learners.

Mapping SfL qualifications to the CEFR might clarify to users how the skills at each RQF level relate to the skills of ESOL International learners, who might be competing for the same jobs or education courses upon entry to this country, and improve recognition of SfL qualifications for similar purposes.

Introduction

In the corporate plan 2019-2022 (Ofqual, 2019), Ofqual committed to considering the effectiveness of the regulatory requirements relating to qualifications covering English for speakers of other languages (ESOL). This was originally in response to the government's English Language Strategy (Integrated Communities Strategy green paper, 2019). This work is now feeding into the DfE review of Level 2 and below qualifications.

There are 2 types of ESOL qualifications currently on the Ofqual register:

ESOL Skills for Life (SfL) qualifications. These are suitable for primarily over-16 learners whose first language is not English, who are resident in the UK, and wish to increase their English language knowledge and proficiency for life, work or further study. They were accredited by Ofqual in 2014, against the 2011 ESOL criteria and the rules of the Qualifications and Credit Framework (QCF). These qualifications must meet the requirements of the National Standards for Adult Literacy (henceforth, NSAL) (QCA, 2000) and demonstrate a clear relationship to the Adult ESOL Core Curriculum (DfES, 2001). They are government-funded qualifications, offered at levels Entry 1, Entry 2, Entry 3, Level 1 and Level 2 of the Regulated Qualifications Framework (RQF).

ESOL International qualifications. These are suitable for primarily over-16 learners whose first language is not English and who are living abroad. These qualifications are recognised for entry to higher education in the UK. Some ESOL International qualifications have been recognised as Secure English Language Tests (SELTS), which can be used to satisfy visa, residency and citizenship requirements. ESOL International qualifications are primarily delivered abroad, but also within the UK. These qualifications reference the Common European Framework of Reference for Languages (CEFR) (Council of Europe, 2001) as well as the RQF levels and have a minimum of 75% external assessment.

The focus of the current research was on ESOL SfL qualifications. This focus was partly motivated by the fact that these qualifications are government funded, as well as by the fact that Ofqual criteria for them are fairly minimal.

The purpose of this research was to explore and evaluate a sample of SfL qualifications in terms of their design and assessment properties, alignment with current regulations/criteria, comparability between awarding organisations (AOs) and validity. We also wanted to learn about the broader context in which these qualifications operate. This would provide us with an understanding of the effectiveness of our current regulatory requirements for these qualifications in ensuring their optimal functioning and validity in relation to their purposes. This evidence base will contribute to policy development and decisions about future regulatory arrangements for SfL qualifications.

More specifically, we wanted to develop a clearer understanding of:

The research was carried out in several strands:

Study 1: Assessment strategy and design review (focus on Entry 1 and Entry 3)

Study 2: Interviews with key members of AO staff

Study 3: Curriculum alignment review (focus on Entry 1, Entry 3 and Level 1)

Study 4: Comparability study involving

CEFR content mapping (focus on Entry 1 and Entry 3)

CEFR linking of assessment standards of reading comprehension tests (focus on Entry 3)

CEFR linking of writing performance samples (focus on Entry 1, Entry 3 and Level 1)

The research involved a sample of 4 AOs that participated across all of the studies, with 2 additional AOs included for the interviews. In addition, due to resource and time limitations, we focused on particular levels as exemplars, on the assumption that practices would be broadly similar across all levels within one AO.

It is important to note that, although this research identifies qualification properties and practices related to specific, regulated qualifications, it is not concerned with issues of compliance. The findings are examined through the general lens of validity and comparability, and in the context of broader issues in language assessment for migration and integration, rather than in relation to specific regulatory requirements. The research is intended to explore general practices and approaches used for development and delivery of SfL qualifications so that we could learn from this and ensure that improvements are made overall where issues are observed, and that our regulatory approach is such that it can ensure optimal functioning, appropriate standards, and validity of these qualifications.

The report is structured as follows. After presenting some background information about the SfL qualifications, we report on the details of each of the 4 studies in turn. The sections for each study present the relevant methodology and results. This is followed by a general discussion and recommendations section.

SfL qualifications: background information

In this section, we briefly outline some information about the SfL qualifications' purposes and design, cohort, funding, progression patterns and apparent issues with recognition of these qualifications by different stakeholders. We also discuss the implications of the apparent lack of clarity in SfL qualification purposes. This provides a backdrop and context to the findings presented in the rest of the report.

Qualification purposes and design

The SfL qualifications are government-funded qualifications that are currently used to foster and improve linguistic integration of different categories of migrants to the UK. They were previously used to evidence language skills for immigration purposes (for visa, residency and citizenship requirements), but this is no longer the case as this role has been taken over by the SELTs.

In one sense, the SfL qualifications and tests can be considered relatively low stakes, as their results are not used for decisions that directly affect learners' legal rights (for instance, their right to acquire citizenship). On the other hand, they are still intended and funded to serve an important function in relation to integration and promoting the chances of migrants to achieve their full potential within the UK society by removing or reducing language and literacy barriers (DfES, 2001; QCA, 2009). They help migrants already in this country to acquire language skills that might subsequently enable them to achieve the SELTs, perhaps for citizenship requirements, or enable them to access employment or education courses. In this sense, they can still be seen as reasonably high-stakes qualifications, at least for some of their users.

The SfL qualifications were last accredited by Ofqual in 2014, against the 2011 ESOL criteria and the rules of the Qualifications and Credit Framework (QCF). Although the QCF was withdrawn in 2015 and the qualifications now have to abide by Ofqual General Conditions of Recognition these qualifications still currently largely follow, with some exceptions, the original QCF design rules.

The qualifications involve a full certificate in ESOL SfL and separate awards in reading, writing, and speaking and listening. Most AOs use the same QCF units, with 270 GLH (guided learning hours) across 3 units per level, while some have somewhat different units and GLH. The qualifications are offered at RQF levels Entry 1, Entry 2, Entry 3, Level 1 and Level 2. Individual awards can be taken at different levels, depending on candidates' skills profile and needs.The certificate is awarded where all units are achieved at the level of the qualification. The qualifications from different AOs should be comparable in standards and are intended to be used interchangeably by learners within and between levels.

The key regulatory requirement for SfL qualifications is that they should meet the requirements of the NSAL and demonstrate a clear relationship to the core curriculum. Unlike ESOL International qualifications, the SfL qualifications are not required to be mapped to the CEFR.

The core curriculum itself is based on the NSAL. According to DfES (2001), the publication of the core curriculum followed national consultation with teachers and managers of ESOL programmes and relevant national bodies. It drew on a wide range of existing curricula from a variety of contexts, including the core curriculum for adult literacy, the National Literacy Strategy in schools and curricula for English as a Foreign Language (EFL). It is intended as a framework for English language learning and defines the skills, knowledge and understanding that speakers of other languages need in order to demonstrate achievement of the national standards in English, providing a reference tool for teachers of ESOL.

Within these parameters, awarding organisations have flexibility in:

mode of assessment, including use of internal or external assessment in all skills areas and at all levels

assessment approaches (such as task type, nature of mark schemes and approaches to grading)

availability of assessments

approach to resits

whether to allow the use of dictionaries for reading assessments

Cohort properties

This level of flexibility in qualification design appears to relate to some extent to the nature of the SfL cohort, which is very diverse, with potentially a wide range of needs and goals in relation to English language learning. According to DfES (ibid., cf. DfES, 2000), the broad categories of learners who SfL qualifications are aimed at are:

settled communities, including communities from the Asian sub-continent and Hong Kong. Some of these learners work long and irregular hours and cannot attend classes regularly

refugees, who sub-divide into:

asylum seekers, most of whom are very keen to learn despite the challenges of resettlement and the trauma resulting from their recent experiences

settled refugees, many of whom have had professional jobs in the past, though some may have suffered a disrupted education due to war and unrest

migrant workers, mostly from Europe, who are here to work and settle for most or all of their lives

partners and spouses of migrants from all parts of the world, who are settled for a number of years and need to participate in the local community but are prevented by family responsibilities or low income from attending intensive EFL courses

Alongside their lack of familiarity with the cultural context of the UK, these learners have 'ESOL needs' in that they are learning English as an additional language, and their English needs further development, particularly with respect to writing, grammar and vocabulary (Roden & Osmaston, 2021) to enable their effective linguistic integration into the society. In this sense, SfL qualifications, as well as other qualifications for learners who are learning English as an additional language (for instance, ESOL International), have the broad purpose of enabling these learners to develop their English skills and meet their ESOL needs. This fundamentally differentiates them from qualifications such as Functional Skills English (FSE) or GCSE English, which are designed for learners whose first language is English.

Beyond the ESOL needs, DfES (2001; cf. also QCA, 2009; ALTE, 2016) emphasises that within the above‑mentioned learner groups the specific needs of learners vary considerably depending on their aspirations, educational background, language and literacy background and aptitude for learning languages. Learners' educational and employment backgrounds are often highly diverse, sometimes even within one teaching group. They may span a wide continuum, ranging from people with no previous education or employment at one end to highly educated professionals, such as doctors or university lecturers, at the other. Many learners already speak and write several languages and can use this knowledge to support their learning of English. Unlike many adult literacy learners for whom English is the first language, they often perceive themselves as successful learners and are very keen to learn. Some ESOL learners have left their countries unwillingly. Some are suffering from culture shock and are experiencing practical difficulties in specific areas of their lives, or racist attitudes and behaviour. Learners are typically settling in the UK and want to learn about the country's systems, such as education, how to apply for jobs, and rules concerning immigration.

More recently, Higton et al. (2019) found the following profile of learners in their ESOL provider survey sample: job seekers, people already in work, refugees, women not seeking work, recent migrants and asylum seekers and people of retirement age. Their survey findings showed that refugees were more likely to access pre-entry and Entry level provision and learners who were already in employment or receiving Jobseeker's Allowance were more likely to access learning at Level 1. They also noted that the profiles/demographics of those accessing English language learning provision changed during the past 10 to 15 years, resulting in the offer of more pre-‑entry and Entry level courses, to meet the increasing demand from learners. The changes include more asylum seekers and refugees wanting to learn English. Some of the providers they surveyed said they noticed a decrease of economic migrants from Eastern European countries while learners from Italy and Spain had increased. In general, however, it appears that the groups described in DfES (2001), for whom the curriculum and the SfL qualifications had been intended, are largely the same nowadays despite some shifts in the relative proportion of learners from different groups (see also Kings & Casey (2014) and WMCA (2019) for similar findings).

Funding, progression, and recognition issues

The SfL qualifications are publicly funded through a range of funding streams. They were fully funded for all learners until 2007; since then, the funding has gradually decreased (Roden & Osmaston, ibid.; Kings & Casey, ibid.; Foster & Bolton, 2018). Currently, learners part pay for the course and assessment unless they are 16 to 19 years old, unemployed or on certain benefits, in which case the qualifications are fully funded. There is no funding for ESOL provided in the workplace (Foster & Bolton, ibid.).

Most certificates are issued at Entry level. Furthermore, many of those learners who want to study at Level 1 appear to progress to other qualifications such as Level 1 FSE, either because higher-level ESOL provision is not available, or because they wish to study a mainstream qualification. This trend is confirmed by Learning and Work Institute (2020) and Roden & Osmaston (ibid.). Roden & Osmaston observe that, even though the majority of the providers they surveyed indicated that most of their learners do progress to ESOL SfL Level 1 from Entry 3, a substantial proportion of learners progress to FSE and GCSE English qualifications. They also observed that this was more common for young people aged 16 to 19. The authors note that this may disadvantage these learners compared to the adults because the FSE/GCSE provision may not be suitable for ESOL learner needs as they are designed for learners whose first language is English and who are familiar with the cultural context of the UK.

Roden & Osmaston (ibid.) suggest the following interrelated reasons for some ESOL learners' progression onto FSE/GCSE English from Entry 3 SfL qualifications despite the former likely having too few guided learning hours for learners with ESOL needs, or being otherwise unsuitable for ESOL learners:

In study after study reviewed, spanning more than a decade (QCA, 2009; Kings & Casey, ibid.; the Casey review, 2016; Bell, Plumb & Marangozov, 2017; Foster & Bolton, ibid.; WMCA, ibid.; Higton et al., ibid.; Roden & Osmaston, ibid.; Rolfe & Stevenson, 2021), there are testimonies from qualification providers and other stakeholders about the high demand and need for specialised ESOL provision, as well as calls to improve the funding for all ESOL levels including pre-entry and to increase the recognition of the SfL qualifications for progression to employment or further study. There are also calls to introduce funded Level 3 ESOL qualifications (or fund existing ESOL International qualifications such as IELTS) to enable learner progression to more specialised, high value‑added jobs or higher education.

In this sense, ESOL SfL qualifications appear to stand in contrast with a number of ESOL International qualifications, which are also designed for learners with ESOL needs, and are recognised for direct progression to higher education and employment, as well as for immigration purposes. These qualifications describe their levels both in terms of the RQF and the CEFR. The CEFR levels are used as a proxy for describing the required language skills for progression or immigration. For example, B2 level across all of reading, writing, speaking and listening skills is typically required of international students for entry to UK universities. For a sportsperson visa, A1 level in speaking and listening is required, while a Minister of Religion visa requires B2 level across all four skills. Skilled worker or health and care worker visas require B1 level across all skills, as do applicants for permanent residency or citizenship in the UK. What this also suggests is that English language skills of speakers of other languages at B1 or B2, demonstrated via some ESOL International qualifications, are deemed to be at an acceptable standard to enable them to engage in skilled work and academic study, respectively. They are not required to provide further or alternative evidence of their knowledge of English, after being admitted to this country, by taking additional qualifications that are otherwise designed for speakers whose first language is English, such as FSE or GCSE English.

Back to qualification purposes

Using Newton's (2017) perspectives framework for thinking about assessment purposes, the fact that the SfL qualifications should help with linguistic integration of migrants, and removing language and literacy barriers to accessing employment and education, would suggest purposes that could be seen from an engagement perspective, helping to ensure that learning actually occurs. From this perspective, the qualification needs to exist because the amount, direction or quality of engagement that it stimulates and shapes is personally, institutionally and/or socially valuable.

Given their required alignment with the core curriculum, it appears that some of the purposes relate to the expertise perspective, providing evidence of a level of mastery or the acquisition of expertise of the specified content domain, as well as boosting confidence and motivation to build on skills and expand the expertise within the domain. The acquired expertise can be seen as personally, institutionally and/or socially valuable in terms of enabling individuals to assume particular roles or responsibilities in the relevant communities of practice or perhaps more broadly in the society.

What is less clear from the literature is the extent to which these qualifications serve purposes related to the information perspective, and which, if any, specific purposes within that. The information perspective can be seen as focusing on the use of assessment results to make decisions, where assessment results provide information that may be personally, institutionally and/or socially valuable as the basis for making decisions. As noted above, these qualifications are no longer used for high-stakes decisions that directly affect learners' legal rights. However, given apparent issues with recognition of these qualifications by employers or education institutions, it is less clear whether these qualifications provide any information that is valuable to stakeholders beyond learners themselves or qualification providers as the basis for making any decisions.

The lack of recognition of SfL qualifications by certain key stakeholders such as employers or further education institutions suggests that the SfL qualifications may not have or may not fulfil some of the information-related purposes that they should or could have. This, in turn, may be driving their own or their qualification providers' choices regarding the most appropriate language qualifications for them. Directing potential SfL learners to qualifications that might have a more well recognised and socially valuable information purpose (for instance, Functional Skills English) while not serving the engagement or expertise purpose in terms of providing expertise of the most appropriate content domain (ESOL rather than English as the first language) may be detrimental for both ESOL learner engagement and their ultimate expertise and potential for progression.

This could also be seen as a situation where a qualification justifiably has multiple (competing) purposes (cf. Newton, ibid.), but where there is not enough clarity about their relative importance to different stakeholders, and no simple answer to the question of why these qualifications are actually needed. This situation may indirectly impact on overall quality and appropriateness of provision and reduce the extent to which any of the intended purposes are truly fulfilled, potentially reducing the usefulness of these qualifications to users.

It would be important to seek clarity about the intended purposes of these qualifications and to clearly acknowledge whether there may be multiple purposes and their relative importance to stakeholders, as well as what the nature of interaction is between these multiple purposes. This would help considerations regarding the most appropriate assessment models, as well as approaches to regulation, and any trade-offs that might need to be acknowledged.

Study 1: Assessment review

Introduction

The focus of this study was to review high-level curriculum coverage, assessment models and assessment instruments from the 4 AOs in our sample in order to understand the range of approaches currently in use, and to consider whether there are any potential threats to comparability, reliability and validity in current approaches.

Method

The review was conducted as a desk-based exercise by an Ofqual senior manager with expertise in qualification and assessment design supported by an Ofqual researcher. Initially, a review of assessment approaches was carried out based on qualification handbooks and specification documents. Following this, a detailed analysis of 2 publicly available sample assessment instruments per AO and component at Entry 1 and Entry 3 was conducted. This was subsequently cross referenced with an analysis of 2 live assessment instruments per AO and component to confirm that these were not substantially different from publicly available sample materials. Where there were some differences, the characteristics of the live assessments were considered in the analysis.

Results

Summary of findings

SfL qualifications consist of 3 components (certified as either individual awards, or overarching certificate if all components are achieved at the same level):

Reading comprehension is typically assessed via objectively marked written papers. The papers tend to involve 3 to 4 tasks which consist of a stimulus text and a number of selected or short‑response items which probe the comprehension of each text.

Writing is typically assessed via written papers typically involving 3 tasks. One of these tasks is always a form that needs to be filled in, while the other 2 tasks typically involve longer open written responses to designated topics (often an email or a descriptive written piece).

Speaking assessment involves performance assessment usually through 3 to 4 tasks. Usually, one of these tasks involves a candidate speaking uninterrupted about a particular topic or might involve exchanging personal information with the assessor. This is then typically followed by a role play on a given topic, which might involve the assessor and the candidate, or 2 or more candidates. At Entry 3, there is usually a task that involves some form of discussion about a given topic, often with the assessor, and sometimes with other candidates.

Listening comprehension of the interlocutor in face-to-face communication is assessed as part of the speaking assessment. Most AOs also have additional listening comprehension tasks involving listening to audio recordings and accompanying written or oral questions probing candidates' listening comprehension ability.

The assessments for these qualifications are summative, typically delivered 'on demand' or with several assessment windows during an academic year, and with opportunities for re-sits. The candidates are normally entered for assessments when deemed ready by their course provider.

The written exams (including reading comprehension and writing, and in some cases listening comprehension) are delivered in paper-based format. The speaking exam is typically delivered face-to-face in qualification providers' premises, though, more recently, it has been increasingly delivered online, via video link.

All exam papers and assessment tasks are externally set by the AOs (although some allow centres to modify some of the tasks). All AOs in our sample implement a competence-based approach (CBA) to assessment, in that they specify detailed learning outcomes (LOs) and assessment criteria (ACs) for each component/unit. This approach is consistent with the requirements of the QCF framework, which was in operation when these qualifications were accredited (for more details and discussion on CBA see Newton, 2018; Newton & Lockyer, (2022).

The ACs specified in this approach represent the backbone of assessment specifications. The ACs are linked to the curriculum content statements, typically sampling the curriculum fairly exhaustively. In terms of the coverage of specific curriculum statements, the AOs appear to be highly similar, according to our initial curriculum mapping based on their published specifications. There are a small number of differences where certain AOs choose not to specifically assess certain curriculum statements.

Beyond these similarities, which themselves are not without exceptions, our review demonstrates that there is a range of different assessment strategies and approaches implemented across the AOs in our sample, and that there are a variety of 'hybrid' approaches, with some elements inherited from the QCF era, and some perhaps influenced by the traditions of ESOL International/EFL approaches or approaches to assessment of other subjects within specific AOs.

Some of these approaches may be more or less appropriate or defensible in terms of validity, while some may be remnants of historical (regulatory) approaches and frameworks that may not entirely suit the construct of assessment. In addition, there are potential threats to comparability between SfL qualifications springing from the sheer range of approaches. These differences can be grouped into the following broader categories:

Overall approach
use of internal (centre-marked) vs. external (AO-marked) assessment

Grading

use of mastery vs. compensatory model of aggregation

requirement to pass each task vs no requirement

requirement to pass hurdle questions vs no requirement

types of ACs and/or marking criteria, for example, binary (met/not met) vs. graded (best-fit)

component percentage pass mark differences

Assessment design

task and response strategy complexity

use of role play and/or discussion between candidates vs. between candidate and assessor only

use of role play/discussion between pairs of candidates vs. 3 or more candidates

use of written vs. oral questions to assess listening comprehension

varying number of questions in reading and listening comprehension assessments

predominant item types in reading comprehension tests (selected vs. open‑response)

the extent of the guessing factor in selected response items in reading and listening comprehension assessments, both in relation to the overall pass and individual criteria

language demands (of stimulus materials and questions)

amount of language production required (in writing and speaking)

Assessment delivery

whether centres can amend the tasks or not

whether assessment has to be completed in one sitting or not

time allowed per task and component

Quality of some approaches, assessment instruments, etc.:

varying quality of questions in reading and listening comprehension assessments (implausible distractors, ambiguities, demand targeting)

clarity of mark schemes/marking criteria for writing and speaking assessments

extent of task administration guidance for speaking and listening assessments

With respect to curriculum coverage, there was evidence of the content areas included in the curriculum being differentially interpreted in terms of how much weight/emphasis is placed on certain aspects, how they are contextualised or how they are addressed with specific assessment approaches. A detailed curriculum alignment review is presented in Study 3 results section.

The range of approaches is perhaps unsurprising given the lack of clarity with respect to SfL qualification purposes, as well as given that there are no regulatory requirements for the AOs to implement any specific approaches, as long as these comply with Ofqual GCR. The GCR do not specify design requirements for qualifications in detail, thus permitting variation within the bounds of appropriate validity considerations for individual qualifications (Newton, ibid.).

It should also be noted that a degree of flexibility in assessment approaches may be beneficial to learners with such diverse needs and backgrounds as SfL learners and may contribute to overall validity of their assessment results even where it might pose some threats to comparability. This was also highlighted in the AO interviews (Study 2) and has been acknowledged in previous research (Lockyer & Cadwallader, 2020). Flexibility in assessment models may also benefit centres, depending on their resources and expertise (for example, centres might choose an AO offering an external assessment model because internal assessment might take up too much of their resources), although, arguably, such choices may contradict what some learners might find more suitable.

Some careful consideration of learner needs, qualification purposes and their relative weight as well as centre motivations, is necessary before suggesting 'one size fits all' approaches, even where certain threats to comparability may remain. Nevertheless, the different assessment models should be sufficiently justified in terms of validity and best assessment practice, as well as appropriate for the stated qualification purposes.

We discuss some of the more prominent discrepancies observed between AO approaches below, grouped into sections according to the broader categories outlined above.

Overall approach: Internal vs. external assessment

Some AOs implement internal (centre marked) and some external (AO marked) summative assessment models across all their components and levels, while some have different models for different levels, and sometimes for different component. The internal model involves a greater degree of administration flexibility compared to external, with assessments scheduled when candidates are ready and both candidates and assessors are available, rather than during specified assessment windows, which are more common in external assessment models.

Within the internal model, there are differences in the details of implementation between the AOs in our sample (for instance, how much flexibility in the choice of topic there is, whether assessment has to be completed in one sitting or not, whether, if completing in multiple sittings the candidates can see all parts of the assessment in the first sitting or not, whether the same task may be continued on different occasions).

The practice of internal administration of summative assessment as discrete tasks in multiple sittings has been observed in other research into internal assessment within the CBA tradition in the VTQ sector. Lockyer & Cadwallader (ibid.) cite studies finding evidence of an assessment-driven approach with BTEC (Carter & Bathmaker, 2017) and GNVQ assessors (Garland, 1998; Ecclestone, 2002; Wahlberg & Gleeson, 2003). In this approach, the assessors were focused on "getting students through" (Carter & Bathmaker, ibid.) through extensive formative feedback and the use of numerous discrete and accessible tasks. Some authors suggest this can create a scenario where candidates are coached to comply rather than learn (Garland, ibid.; Ecclestone, ibid.). Lockyer & Cadwallader did not replicate these findings in their own study, and even found that in some vocational sectors there was an emphasis on delivering learning above and beyond the course requirements to best prepare learners for employment. However, they did find evidence of a variation in the level of support and feedback provided to learners when completing internal summative assessments.

While our research did not specifically focus on this aspect nor investigate centre practices, the assessment approaches available to centres for some SfL qualifications may lead to some of the above-mentioned practices. This situation might be of more concern where the same qualification, such as SfL, can be assessed using either external or internal assessment. In such a situation it would be important to have sufficient confidence in the controls around each assessment approach in order to reduce disparities in candidate experience and potential threats to comparability of standards.

Grading

There are different approaches taken by the AOs to establishing pass grade cut scores/standards on SfL assessments. Consistent with the CBA tradition, most AOs implement a mastery measurement model, where overall competence is defined as competence across all of the specified ACs, and where competence in each AC has to be demonstrated a minimum number of times and to the required (minimum) degree. However, some AOs implement a compensatory model, without a requirement for each AC to be met.

In addition, among those that implement a mastery model, some define the pass standard purely in terms of each AC being met once across the assessment. Some define pass thresholds at individual task level, alongside a requirement for some hurdle questions to be answered correctly in the reading assessment. Some use a semi‑compensatory model where, in addition to the requirement for all ACs to be met once, a numerical pass mark that is compensatory must also be achieved in some components.

There are potential validity issues with some of the above-mentioned approaches. Sometimes the models where the pass threshold is defined by each AC being met once only can result in cut scores that might be considered too low, for instance, in some reading comprehension tests (see more on this in Study 4). In other cases, the use of hurdle questions without clear construct justification may create inappropriate barriers for achievement of the pass grade. If mastery measurement models were deemed appropriate given the construct of assessment, then it would be necessary to implement them in ways that reflect best practice in using such models, ensuring valid assessment results. It is questionable whether mastery models with very low pass grade standards genuinely represent reliable evidence of mastery.

This kind of variability in measurement models has been observed in previous research into grading in vocational and technical qualifications, which are all originally based on the CBA (Newton, 2018). Arguably, this variability (assuming that individual models were implemented validly) may be more justified across qualifications with different subject matter, which constituted Newton's sample. It is less obviously justified in the context of qualifications which are based on a specific shared curriculum and the same construct, such as SfL. Different approaches to defining the pass standard could interact in important ways with the construct of assessment, potentially unnecessarily leading to disparities in standards or disparities with respect to interpretation of results.

Another question that this situation raises is whether the compensatory or the mastery model are more appropriate for measuring language acquisition more generally. This is a high-level issue crucially related to how the construct of assessment is defined more broadly and is also potentially related to assessment purposes. As Newton (ibid.: 77) notes, CBA and its mastery model of aggregation may be an inappropriate template for those qualifications whose domains are not characterised by large sets of critical (emphasis added) micro‑competencies, or for qualifications that would be better served by a measurement model more akin to compensation or configuration. It should be possible to determine which of these measurement models is best suited to the nature of the SfL construct and content domain as well as the purposes of these qualifications rather than perpetuating certain models for historical or other inappropriate reasons.

Assessment design

Across all components, but particularly in writing and speaking, there is evidence that task demands between assessment versions within one AO and component may not be sufficiently comparable in terms of topic demand, vocabulary complexity, and other aspects. There are also differences in reading stimulus text complexity between AOs. For instance, some texts include noticeably fewer complete sentences at Entry 1 than others, or fewer paragraphs than others at Entry 3.

There is also disparity between the AOs in the number of questions in the reading comprehension assessments, with some including noticeably fewer questions than others. This could have an effect on reliability and classification consistency (especially in interaction with the high guessing factor of some items or implausible distractors).

Duration of writing and speaking assessments also varies, sometimes considerably. Furthermore, there are differences in the guidance regarding the number of words and/or sentences that the candidates are required to write. At Entry 3, for example, the amount of writing required varies by more than 100 words between some AOs.

Speaking assessment approaches also vary in several respects. There are differences in the amount of speech that is required from candidates at the same level between AOs. In addition, some AOs require candidates to produce a monologue at each level, while some do not. There are further differences in the duration of the required monologue between AOs that do require it.

There are important differences in the use of role play and discussion in the speaking assessment too. Some AOs require candidates to only interact with the assessor/tutor across all tasks, while others require them to interact with other candidates too in some tasks. Within that, some only require interaction between 2 candidates, while others require 3 or more candidates to interact. There are advantages and disadvantages to each of these approaches in speaking assessment, but the demands that they are likely to place on candidates and the nature of performance that is elicited by each task type, as well as the demands on assessor consistency, are likely to be different.

Listening comprehension is assessed alongside speaking, and some of the listening comprehension ACs are assessed via the speaking assessment (for instance, understanding the interlocutor). However, at both levels reviewed, some AOs also include a separate audio comprehension task, while others do not. This is discussed further in Study 3 in relation to construct validity and curriculum alignment of these approaches.

Most AOs assess across different topic areas within each assessment component, meaning that a range of topics is sampled in each component and across components within one level (for instance, family-related issues, shopping, course application, news articles on different topics, advertisements, tourist guides). In contrast, some AOs pre-define a range of topics and each component is assessed within each of those topic areas only rather than across topic areas. For instance, a whole reading comprehension paper would relate to the context of a single topic. This is intended to allow centres to choose the appropriate and relevant topics for the candidates to be assessed on, though with the requirement that at different levels a candidate is assessed on different topics. While there might be good reasons for this given how diverse the SfL cohort of learners is, it might limit the curriculum in terms of the range of vocabulary covered and evidenced at each level.

Assessment delivery

Several differences in assessment delivery were noted, mostly with the AOs that implement the internal model of assessment. Those AOs give some flexibility to their centres to amend the reading comprehension tests in particular, but also offer more flexibility in relation to the topics of speaking and writing assessments, compared to the AOs implementing the external model. In relation to potential amendments of the reading tests, it was noted in our interviews (see Study 2) that this opportunity is rarely taken by the centres, however, as the work involved to implement changes is usually prohibitive.

Among other flexibilities, the option to complete an assessment in multiple sittings is offered by some AOs but not others. Potential issues with such flexibilities were discussed earlier in the context of internal assessment approaches more generally.

Some variation was observed in relation to the time allowed by different AOs for different tasks or components, which could have some implications for task demand and comparability of candidate experience. For example, for Entry 3 reading tests, AOs varied between 2 and 4 minutes per mark. In Entry 1 writing assessments, the time allowance across the 4 AOs varied between 10 minutes and 20 minutes per task, with some allowing around 15 minutes per task. There were similar discrepancies in the speaking component.

Quality issues

A range of quality issues were noted with the questions in reading comprehension assessments, and, to some extent in the listening comprehension assessments where these were designed as written tests. It should be noted that our reviews did not involve statistical analyses of item and test functioning as we did not have access to AO's data for this research. Our qualitative review, however, strongly suggests a need to obtain evidence from the AOs that their written tests function appropriately.

One of the most common issues noted with some tests was the potential for getting the selected response items right by guessing. The guessing factor was variously increased by a high proportion of true/false items in some tests, or implausibility or cues in distractors of multiple‑choice questions in others. On the other hand, some tests were designed effectively to reduce the potential for guessing. Instances of items with implausible distractors, ambiguities, and apparently too low or too high demand for the level of the qualification were noted across most reading and listening comprehension tests to varying extent.

Our review also found that there was a range of approaches to and level of detail in assessor guidance in relation to the administration of speaking assessments. There were also varying approaches in relation to how assessment criteria were elaborated for both writing and speaking assessments. Some external assessments appear to provide more guidance than some internal assessment. This is of concern as internal assessors are likely to depend more on this guidance compared to external assessors. This finding echoes previous research which identified potential vulnerabilities in the delivery of internal assessments related to assessor guidance and training, and their potentially differing interpretation of AO-set standards (Lockyer & Cadwallader, ibid.).

Issues with a lack of detail and clarity of mark schemes and criteria were also noted with some external assessments. While it is possible that further clarity is provided to assessors during standardisation and training, arguably it would be preferable to provide clearer, publicly available criteria as far as possible. Furthermore, variability in assessor standardisation itself may pose threats to validity and comparability, particularly in those approaches that involve the mastery model for specifying grade standards (see Newton, 2018.: 69).

Study 2: AO interviews

Introduction

This study sought to explore the perceptions and views of 6 AOs who offer SfL qualifications. The overarching research questions for these interviews, outlined below, are deliberately broad, allowing us to explore the perceived purposes of the qualifications and how they are developed and delivered to meet those purposes.

What do AOs perceive as the purposes of SfL qualifications and who, therefore, are the learners that AOs seek to support?

How do AOs who offer SfL qualifications develop their specifications and assessment materials with regard to their purposes, the core curriculum and any other external frameworks or guidance?

Ofqual regulates such that qualifications are valid with respect to their established or known purposes and reliably indicate the knowledge, skills and understanding students have demonstrated. Understanding the purpose(s) of a qualification provides the foundation from which to understand the validity of the assessment (and, as part of that, its reliability). The above research questions therefore complement the wider research report by providing important context for understanding ESOL SfL qualifications.

Method

Given the open and explorative brief, a qualitative approach using semi-structured interviews was selected. Group interviews were conducted with small groups of employees from 6 different case study AOs.

Interviews typically involved 2 or 3 Ofqual colleagues and between 2 and 5 employees of the AO. The AO selected their own panel of interviewees, seeking to provide expertise across all areas of the development and delivery of their SfL qualifications. Interviews were conducted using a flexible interview schedule and took place via video conferencing software, typically lasting for 2 hours. Each interview was recorded (video and audio) and the audio elements were transcribed verbatim for analysis. The interview schedule was developed to explore the AOs' perceptions of the purpose and constraints for SfL qualifications within the market, their broad processes for the development and delivery of qualifications within that context, and their broad processes for setting and maintaining standards.

A flexible thematic approach was taken to analysing the data, with the goal of identifying patterns across the 6 interviews and making comparisons between them in relation to the research questions (Braun & Clarke, 2006). We did not therefore conduct a full 'line by line' thematic analysis of the data and instead undertook a broader comparative analysis of our case studies. The lead researcher used NVivo qualitative analysis software to code the written transcript with patterns and themes. The researcher had access to the audio files while doing this, allowing them to clarify the tone of an extract, or otherwise verify the accuracy of the transcription, where necessary.

Some of the codes that were used within the process had already been broadly identified following discussion between the Ofqual staff who attended the meetings, while others emerged more organically through the analysis process. The researcher usually coded sections of the transcripts rather than individual sentences, the aim of which was to take account of the context of a comment and to avoid analysing statements in isolation. The researcher revisited transcripts they had initially coded to ensure that the coding scheme had been applied thoroughly and consistently, focusing on transcripts that had been coded earlier in the analysis process at a point where the coding scheme may have been partially incomplete. At the end of this process, a second researcher scrutinised a sample of the transcripts to confirm that they agreed with the coding and provide challenge where they did not.

It is worth noting that this research does not capture the views of all AOs who offer ESOL SfL qualifications but rather it seeks to illuminate the key issues and how they may be influenced by the context of the AO. This will help us to gain a more nuanced understanding of the perceived purpose for these qualifications and how that influences how they are developed and delivered in practice.

Key themes emerging from interviews

Below, we outline the key themes that emerged from the analysis. We have selected quotations to illustrate these themes. We begin with a short summary of the overall findings.

Summary of findings

It is clear from the views expressed in our interviews with AO staff that the SfL qualifications are seen as having enormous value to learners and society as a whole. The AOs emphasised the breadth of purposes of SfL qualifications as well as a wide variety of learner background and needs.

There was a sense, however, from all AOs that these qualifications were undervalued in terms of funding. They also said the qualifications were not sufficiently recognised for their potential to enable learner progression, for instance to apprenticeships, or T Levels, particularly in comparison to Functional Skills English (FSE) qualifications. Practitioners told us that this affected stakeholder attitudes towards the qualifications, including centres, employers, higher education institutions, and learners themselves. They thought that this was also partly to do with a lack of understanding, on the part of some stakeholders, that ESOL qualifications have an enabling function in terms of developing broad language skills of learners with ESOL needs, from very basic to advanced, rather than just focusing on literacy skills.

They emphasised the need to recognise the unique value of these qualifications for addressing ESOL learner needs, while recognising their parity with similar qualifications such as FSE. The SfL qualifications were not seen as dissimilar in their enabling function to ESOL International qualifications, some of which are widely recognised at competence levels similar to Entry 3, Level 1 or Level 2 by reputable higher education institutions as well as for job recruitment.

The AOs discussed the flexibility of the SfL qualification structure, its modularity and the variety of assessment models that can be used as beneficial and justified by the needs of cohort and centres. They described how learners and centres required flexibility of choice and adaptability of assessments to accommodate the wide range of contexts in which these qualifications are delivered. All our interviewees referred to the flexibilities offered in the context of speaking assessments in particular. However, AO staff in our interviews suggested that the flexibilities offered to centres in terms of the possibility for written reading comprehension assessment adaptations are rarely taken up, although the AOs themselves offered some choice of context across different assessment versions in some models. Furthermore, the choices that centres make, for instance, in relation to using internal versus external assessment models, do not seem to always be tailored to the learners, and sometimes appear to be driven by centre needs (for instance, availability of assessors).

Our interviews with the AOs suggested that some of the assessment design choices, noted in our assessment and curriculum alignment reviews as potentially sub‑optimal or not entirely appropriate for the construct of assessment, were partly historical, and linked to the requirements of the QCF (the previous regulatory framework). These design choices were not always considered by our interviewees to be reflective of best practice in language assessment (for instance, atomistic assessment criteria, certain content aspects enshrined in the units, choice and implementation of measurement models, weighting of certain curriculum aspects). However, those we interviewed thought that the QCF has provided a helpful mechanism for ensuring a degree of consistency of approach, construct and standards across AOs, particularly because it allowed the units of qualifications to be interchangeable across AOs. This was seen as important for learners who often need to move and complete different units with different centres and AOs, though there was acknowledgment that other mechanisms may exist for achieving this.

The AOs were keen to emphasise that, although they saw the SfL qualifications, as well as the core curriculum on which they are based, as generally effective and appropriate, there was room for improvement and suggested aspects that may benefit from it. Within that, though, they strongly recommended retaining sufficient flexibility in qualification design and assessment models, which would ensure that there were no additional barriers to participation and learner progression.

Breadth of purpose

The AOs we interviewed were keen to explain that there is no such thing as a 'typical' ESOL SfL learner. Learners appear to choose SfL qualifications for a wide range of reasons.

The AOs also noted that ESOL learners are a highly diverse group in terms of their background, and that the cohort profile can fluctuate over time, partly depending on geopolitical factors.

The AOs noted that ESOL learners vary considerably in terms of their education and employment background. For example, while some learners may not be literate in their native language, others may be highly skilled professionals who wish to develop their English to improve their job performance and further their career. Learners also vary in terms of how much direct interaction they might have on a daily basis with native English speakers, with some not getting much interaction beyond their ESOL classes.

Our interviewees told us that ESOL SfL learners completed their courses in a wide variety of settings, from sixth form and FE colleges through to prisons and community groups. The latter of these, community groups, appeared to be particularly important for some AOs. The emphasis, from a teaching and learning perspective, is on providing a reassuring and supportive environment for entire community groups who were seeking to learn English, with ESOL provision being integrated within wider educational and social programmes.

I think the community provision seems to be the key for [us] and where our audience is, in that it's delivered in a community environment where the learners tend to be of a similar ethnicity within a class size and generally a similar age. AO6

...a lot of it is delivered within community groups where people feel more secure, where maybe the teachers share a first language with the learners, where there is a lot of support, not just language development, and often that helps to get across the barriers. [...] So, we have things like, in the voluntary sector there are things like cookery clubs where trained professional teachers will actually integrate the cookery with the language teaching and that way people feel safer and more secure to participate and achieve. AO3

Given the breadth of purpose for ESOL SfL qualifications, and the diversity of learners taking them, the AOs told us that they tended to form close and dynamic relationships with their centres to support them in meeting the specific, and often changing, needs of their learners.

This flexibility was considered particularly important and is discussed in further detail under a separate theme below. Overall, it was clear that those we interviewed felt that ESOL qualifications had a broad remit and could therefore meet the needs of a wide variety of learners.

Language as a key facilitator

The AOs were keen to stress the important role that language skills play in providing people with opportunity. Language skills were considered to be a bridge into society, allowing individuals and groups to integrate better with their communities and to access education and employment, helping them to fulfil their potential. Indeed, ESOL SfL qualifications were considered vital for unlocking the potential in learners commensurate with their capability in their native language, allowing them to use existing skills for employment in the UK where previously their lack of English language skills had been a barrier.

All RQF levels of the ESOL SfL qualification were considered to have a purpose for language learning. The Entry level qualifications were considered to be an important foundation for some learners, helping them to master the basics and gather confidence for further study. They were deemed justifiably divided into 3 sub-levels. On the other hand, access to specialist ESOL qualifications at levels 1 and 2 was also seen as important for unlocking higher-level skills as well as, consequently, employment and other progression opportunities.

As hinted in the quote above, AOs highlighted the need for ESOL provision at higher levels and emphasised that learners at higher levels do not benefit in the same way from FSE courses.

So sometimes we're seeing people who should be doing ESOL qualifications probably doing other qualifications, is what I mean to say. So, we often see people moving towards Functional Skills where probably application is more the purpose than language acquisition. So that causes some challenges I think for learners. AO4

[...] in my experience not only do ESOL learners not fit functional skills, it's square peg, round hole, [...] it just doesn't work, a lot of ESOL learners at Level 1 and Level 2 are actually at a higher level with their functional skills and quite often their reading and writing skills than native speakers doing Functional Skills. AO1

There were suggestions across all AOs that the current funding landscape, particularly for higher levels, may be driving some of the patterns of provision, where ESOL learners may be directed onto FSE courses for funding reasons. This was seen as limiting the potential for ESOL SfL qualifications to be as valuable and impactful as other qualifications at similar levels (for instance, FSE).

In my opinion there should be parity between functional skills and ESOL Level 1 and Level 2, because second language learners are disadvantaged by the lack of grammar and vocabulary content in the functional skills qualifications as they currently exist. Now, this also is a funding issue really, you know, because functional skills is part of the entitlement but ESOL is not and therefore employers don't necessarily recognise ESOL Level 1 and Level 2 as having parity with functional skills because it's not part of the entitlement. AO3

[...] providers didn't necessarily choose the qualification that was right for the learners, they chose the qualification that was more accessible for them in terms of drawing down funding and things. It's a lot easier to deliver a qualification where it's all being funded by the government than chasing learners for money. AO6

There were clearly some concerns about the nature of the qualification landscape, specifically around funding, and how this might affect take up and progression. There were also concerns around ensuring fitness for purpose of the qualifications that were made available for learners.

The core curriculum and the NSAL

The AOs were generally positive about the nature of the core curriculum and suggested that it was a helpful tool to inform assessment context and standards. The curriculum was also viewed as an important resource for teachers and learners for guiding and gauging their progress, as well as for teachers' skill development.

The benefits of the curriculum were perceived to extend to helping ensure a consistent approach was taken across qualifications and over time.

However, views on precisely how effective this was varied somewhat between AOs. Some comments suggested that differing interpretations of the curriculum were possible, something which has the potential to lead to a degree of inconsistency between AOs.

So, the learning objectives and the assessment criteria are directly based on the core curriculum. There was a lot of discussion about what they meant in relation to the core curriculum and the relationship, there was a lot of teasing out of what the writers of the core curriculum had intended. AO1

There of course are arguments for having an explicit assessment of listening and I would say that we decided for ESOL Skills for Life to prioritise two things: one is the interactive listening with someone who is in a slightly different social position from you, that relationship thing, and the other is the discussion among peers where (a) you have a decision-making responsibility, (b) those social layers are removed, but you have the overlay of having to manage perhaps accents that you're not as familiar with, varieties of English you're not as familiar with. So, we chose to prioritise those within the time constraints of the exam. [...] there are also aspects of that curriculum that justify our decision [...] So we felt that this is better or more easily operationalised through a live context instead of having the typical recording and listening to the contrived recording twice or three times or as many times as you feel. AO2

Those interviewed felt the curriculum was fundamentally fit for purpose, if in need of modernisation and refinement. The general view was that evolution over revolution would be recommended. The AOs thought that there was certainly room to improve the curriculum but that this could be achieved with careful revision rather than needing a fundamental reform.

Some AOs pointed out that the link between the core curriculum and the NSAL is a complex one. On the one hand, the nature of the ESOL cohort, with many learners not having basic literacy skills in either English or their first language justifies explicit emphasis on teaching and assessing literacy within ESOL SfL qualifications. On the other hand, the AOs were keen to point out that ESOL SfL qualifications go beyond teaching and assessing literacy, because the language learning needs of the ESOL cohort are much broader and very different from those for whom English is their first language. Some AOs drew our attention to the link between the core curriculum and the NSAL with regard to issues around the recognition and parity of esteem of ESOL SfL qualifications and other English qualifications.

Some AOs suggested that basic literacy, in terms of learning the Latin alphabet and letter formation, are skills that may benefit from being taught prior to Entry 1 level courses, with Entry 1 courses deemed too high level to easily accommodate learners who have no basic literacy skills whatsoever. On the other hand, it was pointed out that learning of these skills is sometimes complex and non-linear, and hence may need reinforcement and repeated assessment, particularly at lower entry levels.

So, I think it's just a really complicated area and we used to have the pre-entry curriculum, didn't we, years ago which took more of a stepping stone, milestone-type approach. [...] I think the other thing at this end, which is often overlooked, is the need to relearn. Learners will come and they will learn the alphabet, let's say, or they'll learn parts of it, or they'll learn to form letters or whatever it is, and it might be even speaking, listening or reading, but they won't necessarily remember it and they might have to redo it and redo it again even before they're comfortable moving forward. AO4

It's important that [the alphabet is] explicitly tested at lower levels, specifically at Entry 1, because obviously it is important to differentiate between the lower- and upper-case letters. I think it's implied that learners will have that skill as they move up the levels, although it's not necessarily always the case. AO1

Several AOs pointed out what they saw as an unfortunate interplay between the use of an atomistic QCF-based approach to specifying assessment criteria and a mastery model of assessment with the approaches to assessment of literacy in SfL qualifications. Generally, the AOs were of a view that knowledge of alphabet, spelling, punctuation or text purpose may need to be assessed more holistically and communicatively than they currently are.

It's a more holistic thing than saying I can understand alphabetical order, I can understand upper and lower case, I can understand the purpose of a text, it's more holistic than that, but that was the way the QCF we felt led us... AO3

We know that as part of the Adult ESOL Core Curriculum there is some attention paid to spelling conventions and punctuation. We also know that in real life spelling almost never interferes with the understanding of the text. And I'd love anyone on this call to be able to say they absolutely 100% know how to use punctuation. AO2

The interplay with the QCF is discussed in more detail below.

The QCF legacy

The SfL qualifications were accredited by Ofqual in 2014 against the rules of the QCF. Awarding organisations are no longer required to adhere to the (now defunct) QCF, but it remained prominent for many of those we interviewed. As is implied in some of the quotations in previous sections, our interviewees told us that, in some cases, the structure of current ESOL SfL qualifications is a legacy of the QCF, though also pointed out that the similarities between qualifications are also there due to them being based on the core curriculum and the NSAL.

The hangover of the QCF is still very much within our ESOL products and I think there could be a lot more efficiencies... AO4

[...] when the RQF was introduced in 2018, some of the AOs have changed their units, but fundamentally they're the same, because the content is still very coherent and there's a lot of similarities across them, because they're all based on the Adult ESOL Core Curriculum and the National Standards for Adult Literacy. [...] I think when we were talking about the QCF model, that is something that we could look at. It would be a significant change. If you're going to do it, you have to do it properly. At the moment centres are happy. AO1

Some AOs felt that timescales in which to revise qualifications, following reform or a change in requirements (such as the removal of the QCF), were insufficient.

I think that our reliance to the QCF is also lack of sufficient time to respond appropriately to the withdrawal of the QCF. So, the easiest solution was to carry some aspects of the QCF that guaranteed that comparability, for example. [...] May I also add that for me another barrier, not to good practice per se, because there's not one right way how to assess something, there are different ways, but the timeframe, the way that some changes are announced and the timeframes that AOs have to react to those changes. And the fact that I used instinctively the word react, that's probably not a good indication as well, we shouldn't be reacting, we should be part of it, and we should have the time to plan it carefully. AO2

I would just add to that, the timeframes that we get often to redevelop, design, get things out there literally up and running and working are so short and seem to get shorter and shorter. AO4

The reliance of the QCF has had a mixed impact. On the one hand, the AOs thought that the QCF provided a helpful mechanism for ensuring a degree of comparability across AOs, particularly because it allowed the units of qualifications to be interchangeable across AOs. This was seen as important for learners who often need to move and complete different units with different centres. AOs did, however, acknowledge that other mechanisms may exist for achieving this.

Yeah, so given that this is an entirely UK-based qualification, to remove a framework within which we all work, would inevitably create the need for a NARIC-type equivalent and I think that it would be counterproductive, personally. [...] And I think the AOs' reliance on the QCF, although we're not allowed to, I think that that speaks for itself for the need of something being in there to ensure the comparability. AO2

Learners often have to move. If they're being settled or housed by a local authority, they may be moved at short notice to somewhere else in the country and so we have to have a common framework within which to teach them so that they can take a qualification elsewhere and complete it somewhere differently. AO3

On the other hand, the QCF has proven somewhat restrictive at times, with certain content aspects enshrined in the units, the atomistic assessment criteria, and a strong emphasis on mastery, which most AOs recognised was not necessarily reflective of best practice in assessing language acquisition. QCF rules also partly determine the AO approaches to setting standards for individual assessments.

[...] the framework, it was very specific about, these are your assessment criteria, and you must evidence every single one, and in our design of our assessments, if we were to design it now, I don't necessarily think that those assessments would look like that. AO4

I think it was that, certainly when we first met and we met as all the AOs, there was an awful lot of discussion about how can you make a language fit into these very separate, small assessment criteria tasks; whereas a language is somehow more than that. AO3

[...] if you asked me what we've got that we've currently got that I wish we didn't have, I'd probably say I wish we didn't have in every single level a form-filling exercise, which is something we can do nothing about because it's in the units, it's a requirement. [...] I think there are some things we would do differently - sometimes I feel it's a bit of a straitjacket because of the QCF units and maybe without that it could give us more flexibility. AO5

The setting of the pass mark [...] I think it was a continuation of the QCF where it required two-thirds of the content you were examined to be correct so that you could pass. So, this is the continuation and the legacy, let's say, of the QCF. [...] May I also add that there's supporting skills and how they're combined into genuine communicative performance, and there's a tendency to concentrate at the level of the supporting skills. So, to try and define out at such a level of detail that that combinatory process is then forgotten, and it doesn't help anyone. AO2

Flexibility in design and delivery

Although all AOs build their assessment on the foundation of the core curriculum and the QCF, they considered the current flexibility that exists in the qualification structure and assessment models as essential to making the qualifications accessible to the full range of learners and allowing them to be as educationally valuable as possible. These flexibilities were often seen as tied in with the necessary flexibilities that were also required in course delivery.

I think that flexibility is probably the last thing that should go in terms of regulation. I think that's really important to our learners and really important to our centres who are often small, and community based. AO3

So, I think the cohorts are representative of people who do come in and out of learning, might be full-time, might be part-time, might even have to change that mode of attendance during the programme according to personal circumstances, so that flexibility has sort of enabled, the qualifications have stood up to that I would say, which I think is quite important. AO4

In particular, the AOs noted the importance of flexibility in terms of modular qualification structure, with individual units of assessment, and the ability to take individual units at different levels or move between different AOs.

So, we offer all 5 levels, and we offer reading as a separate unit, writing as a separate unit, speaking and listening as a separate unit, or all 3 combined as a certificate. [...] So that modularity and that flexibility is very much with the needs of the learners in mind. AO3

I think there are certain cohorts where [lack of modularity] would create a really big barrier, so prisons would be a really good example to that, just by the very nature that people move around, and I think that would be the biggest challenge. A lot of ESOL learners do move. [...] They can have a spiky profile, so that they can be higher at one skill, one mode than the other and I think that helps them on the way through. AO4

You just go for it, you can get a level 1 speaking and listening and do an entry 1 reading and writing, that's your level, just that's who you are. AO1

What we find is a lot of centres offer ESOL over 2 years rather than as a one‑year programme. You've got to look at the speaking and listening is 120 hours, which is more than a GCSE that's delivered over 2 years, the writing is 90 and I think the reading is 60. They are, more often than not, not delivered as full-time qualifications, especially with adults. They're a daytime part-time class or an evening class kind of thing and they just come in potentially for a couple of hours a week maybe once or twice. So, it does build up and it does take time for these learners. So, we often have learners that will start off on the speaking and listening in year one and then progress on to the reading and writing in year 2. AO6

The option to use internal assessment models with their strong formative approach was also seen as important for centres and learners, especially at entry levels, by some AOs. This model provides familiarity with the tutors and assessment surroundings, as well as flexible delivery (on-demand assessment with assessors available at short notice; options to deliver different parts of a question paper on different days; flexibilities around re-sits (of whole or parts of examination papers; scope for centres to amend or contextualise assessments as appropriate, in line with AO rules; etc.).

The AOs that deliver external assessment also noted that there was scope for flexibility within their own model, particularly within the speaking assessment, where assessors are trained to pick up on what the relevant topics might be for a specific learner. They also noted that they provide formative feedback to learners based on their external assessments. It was also acknowledged, however, that in some respects the external model which involves fixed exam dates was inevitably less flexible than internal assessment and may not work for some learners.

Some AOs saw the flexibility and choice in terms of high-level assessment models (for instance, in terms of internal vs. external assessment) as beneficial to both centres and learners, though it was not always clear how far centre choices were always tailored to their learner needs, rather than their own resources, expertise or other motivations. It was suggested that some learners prefer the formality of an external model, and perceive it to 'hold more weight', while other learners benefit from the flexibility of timing and familiarity of setting in an internal model, as mentioned above. How far learners themselves are able to choose centres depending on which AO and model they use to match their needs remained somewhat unclear.

Context and topic flexibility at task level were noted as a particularly valuable aspect of flexibility given the wide range of learner background, ages and interests, as well as the breadth of qualification purposes and uses. The AOs described their own approaches, where they offered a relatively broad range of topics and contexts to begin with and designed their assessments to be fairly 'open', providing options for centres to adapt contexts as appropriate. However, it was also recognised by AOs that topic and task amendment flexibility is mostly used in the context of speaking assessments (both in internal and external models), and that there is less scope or centre resource to adapt the written tests, despite, perhaps, this being in the interest of some learners, which is another reason why some AOs attempt to design assessments that are open. Some AOs allow centres to choose from different assessment versions, to suit individual candidate context.

Tensions around ensuring comparability between different contexts and topics at the same level, as well as across AOs and over time were also discussed, with AOs acknowledging the challenge but also expressing confidence that they were able to ensure comparability of standards. They also noted that this was mitigated by the clarity of a shared curriculum (as discussed above) which emphasises skills rather than specific context.

It is still possible because the mark schemes are the same mark schemes, the criteria are the same criteria. So, we can still get that consistency and the comparability across centres, even potentially if a centre may decide to tweak, for example, if they've got to talk about somebody at E1 and that person is slightly tweaked, the same language will come out in describing a person, it may just be a different person that has more relevance to them. [...] the issue is about having programmes that are flexible but still rigorous, can still be measured, because obviously what we don't want is something we can't make comparisons between one learner and another, but the way we've developed the qualifications and the assessment do enable them to actually tailor it to different needs. AO5

And I think in the UK system, which is different AOs each offer the same qualification, it does give that standardisation to say that they're all based on the same core curriculum. AO3

While the AOs emphasised the need for flexibility in order to address various learner needs, they also noted that some of the design aspects of these qualifications, or funding patterns, do not quite recognise or cater for certain needs. This was particularly noted in relation to the standard at Entry 1, which was seen by the AOs as not accessible to some learners within the funded guided learning hours, especially those with no basic literacy skills, with some AOs potentially unclear about existing funding arrangements for non-regulated provision.

In related points, some AOs thought that current ESOL SfL provision should also include Level 3, given that some learners need higher language skills for certain employment prospects or progression to further or higher education.

[...] everybody looks at ESOL, it's primarily entry level, but the reality is there are ESOL learners who need to be able to read extended documents and in fact when we developed the ESOL Level 3 it was for a very large FE college with a very large ESOL entry [...] and they said they wanted learners to be able to do Level 3, whether it was access to HE, whether it was [access to] Level 3 programmes [...] they needed more advanced reading skills and strategies to help them... AO5

So, levels, I think we think levels are really important, but we could probably go lower, and we could probably go higher, I guess, and maybe as we went higher, we could be more specific about the purpose. AO4

Varied assessment models

Perhaps as a direct result of the flexibility currently possible when developing ESOL SfL qualifications, AOs developed and deployed a range of assessment models. However, within that, they perceive the need for the qualifications to be interchangeable between AOs, and comparable in terms of construct and standards.

[...] we work within the framework that all of the other AOs have to work because, and it's really important actually, the principle that these qualifications at the module level are interchangeable is really important for us. [...] a learner can pick and mix and combine modules from different qualifications... AO2

I think the diversity between the AOs actually gives flexibility to centres, because, like we said before, the ESOL learners are so diverse that the different methodologies of assessment, it gives them the scope to meet the learners' needs, so that's quite important and at the end of the day everybody is all still focused, they're all being assessed against the core curriculum. So, it is standardised, it's just different methodologies of going about getting to the same end goal. AO1

As discussed in the previous section, most apparent was how AOs differed in their use of internal and external assessment, with some AOs using a mixture of both (depending on the skill being assessed and the level of the qualification). The strengths and weaknesses of these approaches were discussed in terms of flexibility, validity and perceived robustness in relation to qualification purposes and use, with an emphasis on the need to provide a balance between ensuring accessibility for all learners and robust controls around the assessments themselves.

Some AOs reflected on the benefits of reducing the amount of flexibility at higher levels to recognise potentially higher stakes of those qualifications. Some also suggested that learners at higher levels are more resilient and adaptable so might require less flexibility in assessment and be more able to deal with an unfamiliar assessor or the pressure of an external assessment.

So, E1 to E3, they could actually break the paper up and take it in sections, so if you've got an E1 learner who has never sat an exam before, they could actually do it in parts, as long as they did it in a secure way and they didn't give them the papers. So, they could also do it in class, which meant they didn't have to have separate exam sessions. [...] When we got to level 1, we said at level 1 you do the paper as is, in the time required, you sit there, you do it under supervision and then it gets marked, so just like sitting an exam. When it gets to level 2, we felt they needed the formal experience of a proper invigilation. So, it was almost like a gradation. [...] And the reason for it was every other level 2, whether it's GCSE English, whether it's functional skills, has always required external assessment and we thought that if people were then going to move on in their careers and were doing level 2, they needed the experience of sitting with invigilation. AO5

[...] so, for ESOL reading, we thought that was achievable and moved that component at level 1 and 2 onto an externally set and marked model. AO4

There were also suggestions that some of the assessment approaches reflected the ethos of the AOs, or their wider understanding of the construct of assessment in related areas, reflecting their 'brand' in that respect.

[...] the core curriculum and the other frameworks that have also informed the design of the test are important, but I think we've also referenced general English language assessment principles and [our] ethos, our assessment expertise and those things also inform the final shape of the qualification, so it's not just a paint by numbers exercise. AO2

So, based on the standards, linked to the Core Curriculum and I think we've always gone for as much internal assessment with internal quality assurance as possible, because that's always been more of a manageable approach for a lot of our centres really and our learners, with external quality assurance on top. AO4

Another prominent area where assessment models differed between the AOs was in relation to how they conceptualised their measurement models and pass grade standards. As already noted, some AOs implement a compensatory model, while the majority implement varied conceptions of mastery. Within the mastery models, there were differences between AOs with regard to how they conceptualised mastery and how 'hurdles' at AC, task or question level were used in their assessments. This partly reflects differing interpretation of the core curriculum and the QCF, as well as differences between AOs in their assessment philosophies and preferred strategies. As previously discussed, some AOs that employ mastery models acknowledged that this was a historical QCF-motivated approach and thought that this did not necessarily represent the best practice approach to summative assessment of language acquisition. Several AOs pointed out that defining measurement models and/or what represents sufficient evidence of attainment is a non‑trivial task even where there are attempts to come to decisions through discussion of a range of stakeholders.

Some AOs discussed the use of task- or AC-level hurdles in relation to a higher level of 'strictness' of SfL qualification subject-level criteria compared to qualifications such as GCSE which use compensatory models. Some AOs explained that task‑level hurdles were there to ensure that the full range of learning objectives is covered, while others explained the use of both AC-level hurdles and numerical pass mark in reading comprehension as intended to ensure wider coverage of the domain rather than evidence a minimum mastery level. There was also an indication from some AOs that hurdles may not always be an optimal approach but can be used to ensure coverage of required content areas.

[...] the qualification subject criteria is quite strict as well for ESOL [...] with our papers there's section 1, section 2, section 3, you've got to pass each section to pass the overall qualification, so, again, in that respect it's much tighter, it's much stricter than for GCSE or even functional skills, for example, where you've just got to, holistically you've got to pass the paper. There are hurdles within ESOL where you have to be able to demonstrate the skills to a certain level, even within the subskills. AO6

[...] for writing they have to pass every task and that was to ensure there was coverage of all the different learning outcomes and the learning outcomes and assessment criteria. AO5

[...] And I think that comes from when the qualification was being developed. It was felt that a numerical mark should be included on top of the assessment criteria requirement. Because particularly for reading it was felt that learners needed to be able to demonstrate that kind of understanding of breadth and range of lexis AO3

[...] the way the mark scheme was structured would enable people to pass [without getting many marks for spelling, punctuation and grammar], so we looked at it and we decided that the quickest way to do it was to actually say OK, and I'm not a great fan of hurdles, but there is a hurdle. AO5

Valued qualifications but room for change

AOs, perhaps unsurprisingly, saw these qualifications as having enormous value to learners and society as a whole. They also largely thought that they had appropriate processes in place for their development, and that their assessments were high quality and valid.

However, there was a sense from all AOs that these qualifications were undervalued in terms of funding, and not sufficiently recognised for their potential to enable learner progression, for instance to apprenticeships, or T Levels, particularly in comparison to Functional Skills English qualifications, and that this affected stakeholder attitudes towards them, including centres, employers, higher education institutions, and learners themselves.

This was seen as unjustified, and all AOs emphasised the need to recognise parity between these qualifications and similar qualifications such as Functional Skills or GCSE English, as well as some ESOL International qualifications. The SfL qualifications were not seen as dissimilar in their enabling function to ESOL International qualifications, some of which are widely recognised, at competence levels similar to L1 or L2, by reputable higher education institutions.

If ESOL ends up being a poor relation that is not recognised in the same way as functional skills then you're not serving the employers or the higher education and you're not serving the learners, because what you're doing is you're giving them less competence to do what they want to do by leaving out the ESOL elements and encouraging them, because of the funding regime, to take functional skills because it's funded and then they're not as skilled as they need to be to do what they want to be. AO3

I think funding also impacts, that's a barrier for learners for all levels, because they can't access the funding or centres can't access the funding and there are vast swathes of people that would benefit from ESOL Skills for Life that can't actually access it. AO1

The brand of ESOL... there's an awful lot of work's gone into the brand of functional skills that hasn't gone into the brand of ESOL and they do have very different purposes... [...] There has been a bit of a missed opportunity over time, because ESOL hasn't been accepted as evidence of language capability for apprenticeships and so there's a group of learners there who often end up doing functional skills because they can't access maybe GCSE in the timeframes that they've got. So, they opt for functional skills to meet the maths and English requirements for apprenticeships, but actually that ESOL qualification would be a really good piece of evidence. [...] There's huge potential for this qualification in things like T Levels. There's a transition year, there's an opportunity for people - there's a missed opportunity actually for ESOL learners potentially to make the transition to T Levels, because there's going to be an expectation for one or two named qualifications for eligibility. Why not ESOL? It's the most challenging qualification at Level 2 and [...] often learners come from educational backgrounds which are quite strong, just lack the language. AO4

If you want to go and study at what you would class as a decent university then you're not getting into that university with an ESOL qualification, you would have to do a Pearson Test of English or IELTS or get a GCSE, so you have to question the purpose of ESOL in respect to that and in respect to younger people. [...] I think the other key thing is then the marketing of these qualifications and the way that they're marketed in general to promote the benefits of these qualifications, firstly, to learners and, secondly, to employers and universities and stuff like that. [...] So, again, that is something that really needs to be addressed, because all we're doing really in the entirety of all this is saying that ESOL just isn't quite as good as GCSE or functional skills and if that is what we're saying, then why would anybody want to do ESOL when you get to the higher levels, why would you want to spend 270 hours doing a qualification that's just enough to make you get onto another qualification at the same level? [...] and I think that's the fundamental issue is how ESOL is positioned is, it's not the qualification itself, it's not the core curriculum, it's the simple fact is it is an afterthought in government policy. AO6

It is also important to note that the COVID-19 pandemic has had an effect on the way that ESOL SfL qualifications are currently being assessed and this may have a lasting legacy in terms of how certain assessment methods are delivered. Several AOs mentioned that they were thinking about, or in the process of developing, online assessments or ways to support centres with online delivery of speaking assessments.

I think that most of our centres feel that in-centre assessment is really important and the most supportive way to deliver these qualifications, because of the very specific nature of the candidates and the support that they need. However, they do see some value in online qualifications as well, online adaptations as well. AO3

I think there are some provisions also that are setting up with, because it's virtual they're actually recruiting learners from all over the place, so they actually haven't got a centre, they are now an online provider. So that's, the way forward is they need online assessments. AO1

AOs also highlighted a range of barriers to SfL qualification and assessment delivery during Covid that are related to the nature of the cohort, particularly technological barriers, and particularly at entry level. On the other hand, AOs mentioned resourcefulness and perseverance of ESOL teachers, who continued to come up with digital resources to support their learners as far as possible.

I think we've learnt an awful lot about who our ESOL learners are and the challenges they face, particularly at entry level I would say, even more so than level 1 and level 2, like the refugees and people like that that we've talked about earlier on, who have already faced so much adversity and then all they want is their certificate to show what they've done, and they can't get there because they've not got the technology to support it. [...] It was really fascinating and actually the ESOL teachers, very creative, finding ways of using apps and different things and there is a richness in EFL actually that a lot of them were drawing from, because there is a history there of quite a lot of remote teaching and learning. AO4

I think one of the other issues going on at the moment is how COVID has adversely affected the communities that are traditionally ESOL in their nature, probably more so than other communities as well and also looking at the delivery models within colleges and providers when we're talking about a community approach, a lot of these community places are not available at this moment in time. AO6

As previously discussed, there was an acknowledgement that there was room for refinement in the core curriculum and that further efforts could also be made to improve consistency across AOs as well as reconsidering some of the current, sometimes legacy, assessment strategies. However, AOs were keen that any reform or tightening of rules should not come at the cost of flexibility to the learners, a vital element of what makes ESOL SfL qualifications so valuable to those that take them.

Study 3: Curriculum alignment review

Introduction

Alongside Ofqual GCR, the only criterion for the AOs in relation to SfL qualifications is that they meet the requirements of the NSAL and demonstrate a clear relationship to ESOL core curriculum (which also reflects the NSAL). From our assessment review we could see that there were some differences in approaches to assessment and interpretations of the curriculum content statements between AOs, differences in weighting of assessment criteria related to specific content statements, differences in descriptions of pass grade standards, and so on. While different approaches to assessing certain constructs may be equally valid, a wide range of divergence between AOs might also lead to a divergence in standards at individual component as well as qualification level. We conducted this curriculum alignment review to enhance our understanding of the extent to which different AO approaches substantively align with the NSAL and the curriculum.

Method

Two SfL/ESOL subject experts were recruited to conduct this review. Both reviewers also participated in our study 4, which preceded the curriculum alignment review.

One of the reviewers had extensive expertise in teaching, teacher training, curriculum training and examining of SfL qualifications (as well as their predecessors and other ESOL qualifications) and was thus thoroughly familiar with the detail of the core curriculum. This reviewer also had extensive background and understanding of the history of the core curriculum and SfL qualifications development, as well as a detailed understanding of related policies and funding arrangements, and has held a number of senior positions in the ESOL area in several institutions within the further education sector. The other reviewer had a more extensive background in ESOL International and EFL qualifications as well as CEFR expertise primarily as teacher and assessor, and past experience of teaching MFL and adult literacy qualifications and of qualification development. This reviewer also carried out the CEFR content mapping for our Study 4 ahead of the curriculum alignment review, and therefore additionally gained familiarity with SfL specifications and the curriculum through that exercise. Both reviewers also had a research or lecturing background in linguistics as well as broader educational research, and experience of working as Ofsted inspectors in the further education and skills sector including ESOL SfL.

Neither reviewer had specific experience of operational assessment design from any work for awarding organisations though had clearly accumulated reasonable expertise through their teaching and examining as well as consultancy work developing MFL or teacher training qualifications. This is a potential limitation of our choice of reviewers. However, this study was primarily concerned with curriculum alignment, and, while some questions were asked about general assessment quality too, we interpreted subject expert responses to those questions alongside our own internal assessment review, which was carried out by an experienced qualification and assessment developer. We also sought views regarding assessment properties of the SfL qualifications from several participants from our Study 4, who had extensive assessment development expertise from working with a range of testing organisations delivering ESOL International or EFL qualifications.

Overall, given that it is sometimes challenging to identify suitably well rounded subject experts for these kinds of analyses, we believe that our reviewers had appropriate expertise and understanding of the key issues to carry out a thorough and authoritative curriculum review. While we cannot discount the possibility that their individual views were in some ways skewed or biased, this should have been to some extent reduced through discussion in order to agree on key points, which was part of the study design. Ideally, of course, views from a wider pool of experts might well be worth seeking were there to be a reform programme for these qualifications or the curriculum in the future.

The subject experts conducted the review against a series of questions that we asked in relation to each of the 4 AOs in our sample, and for each of Entry 1, Entry 3 and Level 1 SfL components[footnote 10]. For each AO and level, the reviewers considered qualification specification documents and any accompanying documents (such as mark schemes, assessor guidance) and 2 sets of live assessment materials for each component, provided to us by the AOs.

The questions (see Appendix 2) focused around 4 key areas. They asked whether each AO's SfL assessments:

Across these main areas, there were 12 questions that the subject experts answered for each component (reading, writing, speaking and listening) and 1 question that related just to writing and speaking. In addition to these AO-specific questions, the subject experts also answered questions about:

Subject expert responses were analysed by an Ofqual researcher. Most of the analysis was qualitative, and essentially summarises the expert views on our different questions, identifying the main themes that the experts raised. In addition, in order to get a sense of general patterns across different components and levels, the responses were coded in terms of whether the relevant AO/component was deemed fully (Y), mostly (M), partly (P) or not appropriate (N) in relation to each question. At each level, this resulted in 50 ratings for each of the 4 AOs (12 for each of reading and listening; 13 for each of writing and speaking).

Due to limited resource, no double coding was undertaken, which represents a potential limitation of this analysis. However, a discussion meeting was held with the subject experts to seek clarification for their responses where necessary (for instance, where their view was unclear and could not be coded using one of the codes above) and engage them in discussion to resolve any significant points of disagreement between the 2 experts in relation to specific questions. The codes were updated following the meeting where necessary.

The results section below summarises the main points made by the reviewers and findings based on the patterns of ratings across AOs, components and levels, highlighting key areas of concern and pointing out areas where curriculum alignment could be improved.

Results

Summary of findings

As can be seen from the tables below, summarising across the AOs and questions, there is a reasonable proportion of categories (Y and M) indicating that the assessments were seen as appropriate in relation to the questions we were asking and where the assessments are well aligned with the curriculum or reflect good assessment practice in relation to the curriculum. Speaking was generally seen as more appropriately aligned and assessed compared to the other components. However, a range of alignment and other issues were also raised, and the analysis in this section primarily focuses on those as potential areas for improvement.

Table 1 Proportions of ratings characterising assessments as fully (Y), mostly (M), partly (P) or not appropriate (N) - Entry 1

Table 2 Proportions of ratings characterising assessments as fully (Y), mostly (M), partly (P) or not appropriate (N) - Entry 3

Table 3 Proportions of ratings characterising assessments as fully (Y), mostly (M), partly (P) or not appropriate (N) - Level 1

It is notable that the overall appropriateness of curriculum alignment of the assessments seems to decline from Entry 1 to Level 1 for most of the AOs in our sample, based on the increasing proportion of categories characterising assessments as partly appropriate (P) or not appropriate (N). This is concerning given that the stakes of the qualifications are likely to increase at higher levels, as these are the levels that should allow progression to skilled employment or further education most directly.

With some exceptions, the demand of assessments and pass grade standards at Entry 1 was seen as broadly appropriate in relation to the standards implied in the curriculum. Where the demand at Entry 1 was not deemed appropriate, it was characterised as too high for that level. At Entry 3 and Level 1, the demand was considered less appropriate, and generally too low.

In terms of curriculum coverage, the review confirmed that the specifications reference most of the curriculum statements. However, this was sometimes deemed not to be at an appropriate level, with insufficient sampling across the range of contexts and functions specified in the curriculum, particularly at Entry 3 and Level 1. For instance, there is an emphasis on assessing form filling at every level in both reading and writing, even though they are just one example of many text types that could be required from candidates.

In some cases, the reviewers thought that the curriculum statements are assessed in the way that demonstrates adherence to the letter but not sufficiently the spirit of the curriculum. For instance, there seems to be insufficient focus on reading for meaning at Entry 3 or reading for detail at Level 1, with subskills being assessed in isolation rather than as components contributing to and enabling the global skill of reading comprehension. Similarly, certain enabling skills such as knowledge of alphabetical ordering at Entry 3 or declarative knowledge of text and grammar features at Entry 3 and partly Level 1 appear to be over-emphasised in the assessments, contrary to the apparent intention of the curriculum, which aims at developing the understanding of the meaning of these features rather than labelling them. These content statements are sometimes given 'hurdle' status in some assessments, where a pass grade could not be achieved without achieving those criteria/answering those questions correctly. As another example, at Level 1, the NSAL standard "trace and understand the main events of continuous descriptive, explanatory and persuasive texts" was deemed to have been taken too literally to mean chronological events and led to one long text per test often being chosen to have dates in it, which was deemed an unnecessary constraint and not in the spirit of the relevant standard, which also includes non-chronological examples.

In most cases it was not clear why such apparent misinterpretation or narrowing may have occurred, as the curriculum statements and examples were seen by the reviewers as sufficiently clear and indicative of the appropriate level and range of contexts and functions, which the curriculum lays out in terms of increasing complexity through the levels. It may be important, however, to confirm with other experts and relevant AOs whether there is indeed sufficient clarity about this in the curriculum and understand any additional constraints that may have driven some of the narrowing of the contexts and functions.

It was also highlighted, however, that there is sometimes a difficult balance to be struck between ensuring the breadth of coverage in assessments while not jeopardising depth of coverage in teaching. In addition, the atomistic nature of the ACs and the apparent requirement for exhaustive sampling of the curriculum in assessment in the mastery model were seen as partly to blame for some of the above-mentioned issues.

According to the reviewers, the assessments appear to offer some room for improvement, to varying degrees depending on level, AO and component, in relation to:

Comparability between AOs across reading components at each level in terms of stimulus text demands or pass grade standards was deemed insufficient. In relation to writing and speaking, in some cases it was difficult for the reviewers to make a judgement about comparability of demand and pass grade standards given the differences in assessment approaches, weighting of criteria and sometimes lack of clear marking criteria. However, the range of differences, as well as lack of clear marking criteria, particularly in internal assessment models, were in themselves suggested as involving threats to comparability.

The curriculum itself was deemed to be largely fit for purpose. The key issue seems to be finding a way to clarify curriculum messages and standards amongst users, including the AOs. As an example, the focus on more academic and professional reading and writing at higher levels, which is largely missing from the assessments, is present in the curriculum. It is not clear why it does not feature sufficiently in the assessments, given that some of the purposes of SfL qualifications, especially at Level 1 and above, could involve progression to further study and employment. The reviewers pointed out that there are resources in the curriculum itself, as well as other resources, which could help with pitching the assessments at the appropriate curriculum level.

The reviewers thought that the curriculum itself is largely consistent with the CEFR in terms of the language competencies it involves and should therefore allow alignment of the SfL qualifications to the CEFR. Overall, the reviewers considered it potentially beneficial to map these qualifications to the CEFR. This was deemed as potentially helpful for increasing recognition of SfL qualifications (for instance, for higher education progression or for comparison with ESOL International qualifications), as well as promoting and clarifying alignment between AOs (although appropriate mapping to the curriculum would also enable that).

Below we provide a more detailed analysis of the findings from the review, grouped into several broad themes.

Assessment demand and pass grade standards

At Entry 1, the demand of assessments was seen as broadly appropriate according to NSAL and the curriculum standards. However, there were some examples of standards that are potentially too high for the level in some reading and listening assessments, as well as placing too much weight on assessment criteria related to spelling and punctuation, which is not required by the curriculum at Entry 1. Pass grade standards, alongside the coverage of the content domain, were deemed to provide sufficient evidence of mastery for most AOs and components at this level, except for one AO's reading comprehension, and another's writing component.

At Entry 3 and Level 1, the demand was considered less appropriate, and generally too low. At these levels, only one AO's assessment was deemed entirely or mostly appropriate across most components (although potentially too high in some respects for writing and speaking, particularly in relation to the breadth of topics and length of assessment). In other cases, some component demands were deemed appropriate while others were not and were typically deemed as more appropriate in speaking than in reading or writing components, depending on the AO in question.

In terms of pass grade standards at Entry 3 and Level 1, again, only one AO was consistently deemed to evidence mastery of the curriculum at the relevant level. The others were deemed not to provide sufficient evidence of mastery, particularly in reading comprehension and writing, and in some cases speaking and listening comprehension. The reviewers pointed out that, in some cases, unclear marking criteria made it difficult to establish what the pass grade standard entails, and that, to establish this, it would be necessary to consider samples of writing and speaking at the pass grade boundary (as was done in our Study 4).

Across writing and speaking, the reviewers found the marking criteria often less than clear, which is particularly concerning where assessment is internal. It would be important for validity and transparency that the marking criteria are made clearer, particularly with respect to ranges of required language (for instance, vocabulary, grammar, functions) at each level, as well as perhaps allowing markers to distinguish qualitatively between performance levels rather than just determining whether certain ACs are met or not met.

Curriculum coverage

In terms of curriculum coverage, the review confirmed that the specifications reference most of the curriculum statements, although there was evidence of some narrowing of the curriculum, with insufficient sampling across the range of contexts and functions specified. In some cases, curriculum statements are assessed in the way that demonstrates adherence to the letter but not sufficiently the spirit of the curriculum according to the reviewers, with issues noted in relation to interpretation of certain aspects of the curriculum. This was observed mostly at Entry 3 and Level 1.

Particular issues were noted with the reading comprehension component, with not enough focus on reading for meaning at Entry 3 or reading for detail at Level 1, with subskills being assessed in isolation rather than as components contributing to and enabling the global skill of reading comprehension. At Level 1, it was noted that there is little focus on reading for inference, opinion and details that require a more in‑depth understanding. In addition, reviewers thought that there was not enough coverage of more formal aspects of language (for example, formal reports in reading at Level 1, more formal exchanges or more formal audio material at both Entry 3 and Level 1 in speaking and listening, more complex topics/types such as social problems, environment or reports in Level 1 writing) even though these are present and exemplified in the curriculum and would suit the apparent purposes of these qualifications, especially at Level 1 and above. There are also inconsistencies between AOs in the amount of interaction assessed through writing and speaking, with one AO assessing these aspects to a significantly lesser degree than the others.

The reviewers also noted instances where assessments appear to over-emphasise or over‑assess certain aspects of the curriculum. These are often enabling subskills, which, according to the reviewers, should not be assessed in isolation or for their own sake, and which are singled out in the curriculum for instruction rather than assessment purposes (for instance, assessing alphabet and alphabetical ordering at Entry 3 or assessing knowledge of names of grammatical and text features rather than how their function interacts with comprehension of text in both Entry 3 and Level 1 reading comprehension). Such approach to assessment of these aspects was deemed not to be in the spirit of the curriculum, which aims at developing the understanding of the meaning of these features rather than labelling them.

At Entry 1, too much weight in assessment seems to be placed on criteria related to spelling and punctuation, which is not required by the curriculum at this level. There also appears to be too much focus on assessing simple form filling at each level, both as part of reading comprehension and in writing. At Level 1 specifically, although forms are mentioned in the curriculum, this is only in connection with complex open responses, which are not the focus in the assessments, and they are one example of many text types that could be required from candidates.

Important inconsistencies were noted in relation to listening comprehension. At Entry 1, it was noted that the curriculum requires assessment of listening comprehension only in face-to-face communication, rather than listening to audio recordings or conversations between other people. Nevertheless, most AOs do include listening to audio recordings and answering specific comprehension questions in relation to those. This was deemed as potentially problematic, and reviewers thought that the recordings tend to be too information-heavy for Entry 1.

At higher levels, it was noted that the curriculum requires listening comprehension of audio recordings as well as in face-to-face communication. Most, though not all, AOs do include assessment of audio comprehension at higher levels. However, where this is included, the reviewers commented that the comprehension tasks are typically limited to very few, often fairly undemanding and sometimes poorly drafted questions, and the audio recordings are sometimes too slow for the level. At Level 1 in particular, NSAL focus on 'formal exchanges connected with education, training, work and social roles' or listening to authentic material on radio, TV. This type of audio or this level of topic complexity was not apparent in most of the reviewed listening assessment materials at Level 1, with most assessments under‑representing the domain in this respect.

Assessment approaches

Reading comprehension

With regard to reading comprehension, at Entry 1, the approaches were seen as broadly appropriate, although reading load of the rubrics and items and some text demand/lengths were deemed too high in some cases. Some of the questions require reading for detail, meta‑language skills such as naming text purposes, understanding of anaphoric pronoun reference and use of capitalisation rules that are too demanding for Entry 1. Some items were considered not entirely construct relevant, too guessable and sometimes involving implausible distractors.

At Entry 3, some form-filling tasks in reading assessments were deemed too low in demand, with most vocabulary at Entry 1 and the form features at Entry 2. There were instances of Entry 1 or Entry 2 level questions in relation to other tasks too. In some cases, the reading stimulus texts were considered to be too low level for Entry 3, and sometimes did not involve significant paragraphs, which are required at this level. Approaches of assessing within a single topic area were not considered good practice and were deemed likely to limit the range of vocabulary that the candidates should demonstrate at this level.

At Level 1, more formal course- or work-related texts are not given enough focus according to the reviewers. They noted that the curriculum is pushing towards more complex reading at this level and looking at how meaning is built up over several paragraphs, as preparation for vocational and academic courses. Some stimulus texts do not involve long enough paragraphs for the level and thus do not enable sufficient assessment of the above-mentioned comprehension skill. The approach of assessing within a single topic was considered even less appropriate at Level 1.

Across Entry 3 and Level 1, a number of reading comprehension items were deemed poor in terms of demand targeting to the level. Reviewers commented that there were too many true/false items in some assessments, which are guessable, as well as a number of items with implausible distractors or items giving answers to other items in the test.

Writing

Although most writing tasks were deemed appropriate across AOs and levels, some were deemed insufficiently authentic and sometimes not designed sufficiently well to elicit the required language from candidates. In some cases, this was because of insufficient prompts, which could affect learner performance. In other cases, especially at Entry 1, some tasks appear to require grammar that is too high for the level, for instance assessment of present continuous, or a word count that is too high for that level.

At Entry 3, some word counts were deemed insufficient for the candidates to demonstrate production of some important features of writing at Entry 3 such as development of ideas through paragraphing. Some writing tasks involve very simple form filling, which was considered to be below Entry 3. Some tasks were not deemed sufficiently communicative or suitable for assessing written production. On the other hand, while broadly compatible with the demands of the curriculum, some writing assessments were considered too demanding in the number of diverse pieces of writing required.

There are similar issues with the writing components at Level 1. In addition, according to the reviewers, the writing tasks sometimes do not provide enough range of audience and purpose for this level and have little focus on more complex topics/types such as social problems, environment, or reports.

Speaking and Listening

With regard to listening comprehension, as already noted, most AOs include audio comprehension tasks at each level. However, the reviewers characterised these as involving few sometimes undemanding and poorly drafted questions, while the audio recordings were sometimes deemed to be too slow for the level at Entry 3 and Level 1, and too information-heavy for Entry 1. The AOs which include audio recordings in their assessment also assess aspects of listening comprehension through their speaking assessments. Lack of use of audio recordings in comprehension tasks at higher levels was not considered appropriate.

Overall, in terms of approaches, speaking was deemed to be assessed best across most AOs, as well as being mostly appropriate in matching the spirit of the curriculum. Again, however, the reviewers thought that some AOs could improve their prompts to ensure that the required language is elicited from the candidates (particularly where assessment is internal) and include more interactive tasks.

Comparability of demand

There was evidence of lack of comparability within AOs (between different assessment versions within components), as well as between AOs. Comparability of demand between AOs across reading components at each level was seen as insufficient. Issues were observed in relation to comparability of pass grade requirements in particular, but also vocabulary demand in texts and questions, amount of reading time and extent of requirements for detailed comprehension, text complexity in terms of number and length of paragraphs, and item quality.

In relation to writing and speaking, in some cases it was difficult for the reviewers to make a judgement about comparability of demand and pass grade standards given the differences in weighting of criteria and assessment approaches as well as sometimes unclear marking criteria.

The range of differences, as well as unclear marking criteria, particularly in internal assessment models, were in themselves suggested as constituting threats to comparability. Some of the comparability issues noted in writing include varying word count ranges, variable time allowance, variable extent of requirement for extended writing at Entry 3 and Level 1, varying requirements and opportunities for use of complex sentences and discourse markers, and varying spelling and punctuation accuracy criteria. In speaking, there were comparability issues including varying requirements for length of speech, varying extent of use and nature of interactive tasks, varying emphasis on the requirements to display a range of grammar and functions in production, and varying amount of task prompts and administrative guidance.

Atomistic assessment criteria and the mastery model

The atomistic nature of ACs and apparent exhaustive sampling of the curriculum was deemed partly to blame for inordinate weight being given to some enabling subskills, which, according to the reviewers, should not be assessed in isolation or for their own sake, and which are singled out in the curriculum for instruction rather than assessment purposes. The reviewers suggested that the curriculum is sufficiently clear regarding this and implies teaching and assessment of increasingly complex or wide range of skills and contexts through the levels. It may be important to confirm with other experts or AOs whether there is indeed sufficient clarity about this in the curriculum.

This was also thought to lead to an inappropriately wide range of question demands in reading assessments, where questions on basic enabling skills which are taught at the start of Entry 1 for instance, sit alongside much more demanding questions targeting much more complex skills, or even skills that are beyond Entry 1:

The component skills range in level from the beginning of E1 ('understand that print carries meaning') to the end of E1, and AOs are not distinguishing between these 2 ends of the scale sufficiently, so some very basic tasks and other much harder ones are included. Recognising text type at this level is intended as a development activity to support understanding of a text, not as a skill in itself. (R2)

The mastery measurement model, where each criterion has to be met a minimum number of times or to a minimum degree, was not seen as particularly helpful or necessary to ensure appropriate performance standards. In the words of the reviewers:

There is a danger with a competence-based approach of "not seeing the wood for the trees"; language is much more than the sum of its parts. On occasion the mark schemes produce a pass because the candidate has ticked off each criterion at least once, without some kind of holistic overview of whether there has been genuine communication and task fulfilment. ... Where it works best is where there is a minimum mark as well as criteria coverage, and where [...] an overview is taken. (R1)

The criterion-based approach causes difficulties in assessing levels unless very clear guidance is given to markers. Unless the bar is set very low, compensatory approach may be fairer for candidates too, as writing covers a range of subskills and not everyone has the same strengths and difficulties but can still achieve communication. (R2)

Indeed, previous research in the context of internal assessment within the CBA tradition suggests that despite atomisation of assessment criteria and use of mastery model, there is evidence that assessors still make holistic, contextualised and compensatory assessment decisions (Torrance et al, 2005; Johnson, 2008; Carter & Bathmaker, 2017), in some cases disregarding assessment criteria when it may disadvantage a student they believe is 'worth' a certain grade (Garland, 1998). It may, therefore, be important to reconsider the suitability of the mastery model for assessing language competence as implemented in current SfL qualifications.

CEFR alignment

Overall, the reviewers considered it potentially beneficial to map these qualifications to the CEFR. This was deemed as potentially helpful for increasing their recognition, as well as promoting and clarifying alignment between AOs (although they thought that appropriate mapping to the curriculum should also enable that). The reviewers thought that the curriculum itself is largely consistent with the CEFR in terms of the language competencies it involves and should therefore allow alignment of the qualifications based on the curriculum to the CEFR.

Improving the understanding of the curriculum messages

The review suggests some scope for updating the curriculum in terms of adding more reference to some more up-to-date topics. However, this was seen as minor, and the curriculum was perceived to be largely fit for purpose. The key issue seems to be finding a way to clarify curriculum messages and standards to users, including the AOs:

I think the problem is not the curriculum but the way it is used by dividing into separate criteria for assessment. The 'component skills' are intended for teaching and not for assessing everyone, eg, no AOs assess phonics as this is too difficult in a test and anyway is something for teaching, as matching upper and lower case and recognising digits should be (whereas writing upper and lower case and digits is OK to assess). Recognising text type at this level [Entry 1] is intended as a development activity to support understanding of a text, not as a skill in itself. (R2)

The reviewers pointed out that there are resources in the curriculum itself that do not seem to be sufficiently used by the AOs, which could help with pitching the assessments to the appropriate curriculum level:

However, the useful list of the communicative functions expected at this level [Level 1] (at the end of the level in the Core Curriculum) gets insufficient prominence in the assessments and could ensure a more exacting assessment. [...] It includes describing, comparing, narrating in past, obligation (and absence of - "you must/don't have to"), reporting, requests, hypothesise, explaining and giving reasons, feelings, likes, dislikes, opinions, wishes and hopes, advice and suggestions, recommendations, persuading, complaining, warning. This provides a much clearer sense of what should be expected than the standards alone. (R1)

Some AOs have misinterpreted the level, so it seems there aren't enough examples of the level. But ESOL Exemplars[footnote 11] should help. They could also refer to SfL learner materials, which are quite accurate on levels too and still available on Excellence Gateway. There are comprehensive lists of grammar and functions in the curriculum and it seems that AOs are not referring to these. At the very least they should all reprint these in their guidance to assessors, or better, elaborate on them [as some AOs do], to encourage assessors to use them more. (R2)

Study 4: Comparability of standards and CEFR alignment

Introduction

The purpose of this study was twofold. One was to compare assessment and performance standards at the pass grade between the SfL qualifications in our sample to determine if there are any notable discrepancies. As demonstrated previously in this report, there appears to be a range of potential threats to comparability between different SfL qualifications arising from different assessment approaches or interpretations of the curriculum. A direct comparison of pass grade standards should provide a better indication of whether the threats to comparability are actually realised in practice. Notable differences in grading standards between similar qualifications at the same level might stand in the way of learner progression or their ability to use the qualifications from different AOs interchangeably, thus potentially reducing the flexibility in SfL provision.

The other purpose was to align aspects of these SfL qualifications to the CEFR. The CEFR is an internationally widely used framework for describing second language ability via a common 'can do' scale, allowing broad comparisons across languages and qualifications (see Appendix 1 for a figure showing the CEFR global scale descriptors). This purpose relates to some extent to broader issues of recognition of SfL qualifications. For instance, QCA evaluation of the SfL qualifications in 2009 (QCA, 2009) noted a lack of awareness among users and stakeholders of the ways in which the SfL qualifications map to other key known qualifications like GCSEs, EFL qualifications and IELTS as a contributing factor to their lack of external value/currency, the latter 2 being routinely mapped to the CEFR.

We should emphasise that the CEFR mapping carried out in this study can be considered exploratory and preliminary, rather than an 'official' linking, being limited in scope to a subset of qualifications and their components. This was a research exercise, carried out to facilitate our understanding of the extent of comparability of standards between different SfL qualifications, as well as to shed some light on how they compare to other related qualifications such as ESOL International in terms of content and standards at different levels (via the CEFR link).

Why aligning the SfL qualifications to the CEFR may be helpful in understanding their assessment and performance standards

As already noted, at the highest level, both SfL qualifications and ESOL International qualifications have a similar broad purpose of enabling learners whose first language is not English to develop their English language skills. In this way, they are both different from the FSE and GCSE English qualifications, which are intended for learners whose first language is English.

In the context of ESOL International qualifications, the English skills are described in terms of the CEFR levels. The CEFR levels are also commonly used for stating language requirements for different types of visas but are also sometimes used for people who are already in the UK to evidence their level of English, for instance, for entry into higher education. To enter the UK as a skilled worker or a health and care worker, CEFR level B1 is required, while B2 is typically required for entry to higher education institutions. Such uses indicate that there is a common understanding of the broad meaning of the CEFR levels in the context of immigration policy, among some employers and higher education institutions.

DfES (2003) provides a notional mapping between various qualifications and standards frameworks, including NQF, NSAL and the CEFR. According to that mapping, the NSAL levels Entry 1, Entry 2, Entry 3, Level 1 and Level 2 map onto CEFR levels A1, A2, B1, B2 and C1 respectively. While this mapping provides some indication of where the standards for the qualifications such as SfL, developed with reference to the NSAL, should be, this was not an empirical mapping, and did not involve looking at actual assessment and performance standards of individual qualifications developed within those frameworks. Therefore, it might be possible that, for instance, the learners who pass Entry 3 SfL qualifications actually do not exhibit B1 level of performance in terms of the CEFR, or indeed, that they exhibit a higher level of performance than B1 (cf. Curcin & Black (2019) for findings that show a mismatch of RQF and CEFR levels in several GCSE MFL qualifications).

The SfL qualifications are currently not explicitly mapped to the CEFR, but only to the RQF and the NSAL. In addition, as demonstrated in Study 3, the mapping to the NSAL and the core curriculum may not be entirely transparent, appropriate, or comparable between different SfL qualifications. It is certainly not entirely transparent to non-specialists what this mapping substantively means in terms of the skills that the learners at different NSAL/RQF levels have in relation to what might be required for different types of employment or further or higher education, and how those skills relate to the skills of ESOL International learners described in terms of the CEFR levels, who might be competing for those same jobs or academic courses upon entry to this country.

Given that the broadest purpose of the SfL qualifications is to impart English language skills to learners living in the UK whose first language is not English, it seemed a useful exercise to understand in more concrete terms how individual SfL qualifications align with the CEFR (against which Ofqual regulates ESOL International qualifications) and thus other CEFR-mapped qualifications with similar broad purposes. We wanted to find out if they do indeed impart the B1 level skills at Entry 3 and the B2 level skills at Level 1, for instance. If they did, then English skills at those levels should enable SfL learners to progress to employment or education that requires those skill levels in the same way as English skills at those levels enable international health and care workers (B1) or international students (B2) to access the relevant jobs or education in this country.

Potential tensions and discrepancies in the mapping

Even though the CEFR is intended to provide a metalanguage for description of second language competence across different languages and qualifications, it is not intended to be used indiscriminately and without regard to local context and local educational aims. Any language learning programme or specification of a communicative repertoire for assessment that uses the CEFR would necessarily select aspects of what is described in the CEFR that align with its purpose and the needs of the learners.

It should be noted that the CEFR was not developed with migrant populations in mind. It assumes a level of literacy, such as knowledge of the Latin alphabet, that certain refugees, for instance, may not possess in the target or their native language (ALTE, 2016). Little (2012) gives an example of elaboration of CEFR A1.1 level for migrant learners of French, which included many vocabulary items that are not high frequency (as in the default CEFR A1 descriptor) but are nevertheless essential for adult migrants because they refer to administrative and other procedures that migrants need to be able to cope with.

While the specific context of the SfL qualifications is different to that of ESOL International qualifications and other language qualifications typically mapped onto the CEFR, the fundamental aspects of language competence, which are encompassed in the CEFR and in the core curriculum, are substantively similar, and should be comparable, as confirmed by our subject experts in Study 3. Indeed, the basic communicative approach to teaching and learning language skills in the core curriculum is very much in line with the way language competence, as well as teaching and learning, are conceptualised in the CEFR.[footnote 12] Helpfully, the content of the SfL qualifications themselves is laid out in some detail in the curriculum as well as in individual AO specifications, which aids comparisons. Nevertheless, given the specifics of the SfL context, in our CEFR mapping for the SfL qualifications we acknowledge key points of alignment as well as discrepancies and tensions.

Method overview

The approach in this study followed the methodology used in Curcin & Black (2019) and was guided by the recommended methods and procedures in the manual for relating language examination to the CEFR (Council of Europe (2009), henceforth, the Manual), and the updated descriptors from the Companion Volume (Council of Europe, 2020). It was designed to provide empirical evidence for a link between performance and assessment standards of the 4 SfL qualifications in our sample to the CEFR.

Due to resource limitations, as well as availability of samples of student work, the study was carried out on a subset of qualification levels and components (reading comprehension and writing) across the 4 AOs. Therefore, it is only indicative for levels and components not included in this study.

Following the Manual, the study involved 5 stages:

familiarisation/training of participants

content mapping (relating the construct/content of the SfL qualifications to the CEFR) at Entry 1 and Entry

linking of assessment standards for reading comprehension at Entry 3

linking of performance standards for writing at Entry 1, Entry 3 and Level 1

empirical validation and evaluation of stages 3 and 4 in particular

A panel of 11 participants was recruited for the study. Of these, 10 had extensive CEFR experience, mostly in the context of EFL/ESOL International qualifications, while 3 also had expertise in SfL qualifications in the context of teacher training, curriculum development and Ofsted inspections. The participants included independent language assessment consultants, teachers, examiners and academics affiliated with higher education institutions. They had wide-ranging experience of working on test development or CEFR alignment projects across different language testing institutions in the UK and internationally, as well as teaching and examining in the context of ESOL International qualifications. All except one participant had significant experience of the CEFR in either teaching/examining or test/resource development, and some in both areas. One participant had some experience of it through teaching only. All except 2 participants had some or significant experience of developing reading comprehension tasks targeted at specific CEFR levels and had taken part in similar standard linking exercises before. None of the participants were permanently employed by any of the AOs involved in the study, although some of them worked for some of those AOs as consultants.

Following familiarisation and training (stage 1), 3 participants carried out the content mapping (stage 2) ahead of the standard linking activities. All 11 participants then took part in the linking of assessment standards for reading comprehension (stage 3) and in the linking of performance standards for writing (stage 4).

The SfL materials used in the study were provided to us by the relevant AOs. They included current specification documents, mark schemes and any accompanying marking guidance/criteria where appropriate, 2 live assessment instruments per component, and samples of writing from recent examinations.

The study also involved a range of assessment materials and performances from examinations that are already mapped to the CEFR. These materials were used during familiarisation as well as in the standard linking of writing components. We refer to these materials henceforth as the CEFR benchmarks. The CEFR benchmark reading and oral comprehension tasks, were obtained from the Council of Europe website. The CEFR benchmark writing performances were obtained from several sources:

the Council of Europe website

Cambridge Assessment teacher handbooks for A2 (Key), B1 (Preliminary) and B2 (First) exams (Cambridge Assessment English, 2016a, b; 2019a, b, c)

CEFTrain project

The British Council Aptis test

ESB ESOL International tests at B1 and B2

Trinity College London ESOL International tests at A2, B1 and B2

A small set of benchmarks from ESOL Exemplars (DfES, 2004) was also used in the mapping of the writing components.

We present more detailed methodology and results for each of stages 2, 3 and 4 of the study in turn below. Before that, we present some details about stage 1 of the study, which involved familiarisation and training of the participants.

Stage 1: Familiarisation and training

Prior to undertaking any of the linking activities in the study, the participants were provided with familiarisation and training to ensure reasonable individual and common understanding of the relevant aspects of the CEFR and of the relevant SfL assessments, as well as the standard linking methodology to be used. This aimed to ensure the integrity and quality of panellists' judgements.

Most familiarisation activities were 'self-study', conducted online in participants' own time over a designated period, using a survey tool set up with a range of activities. All the participants completed familiarisation activities related to global CEFR scales, reading comprehension and writing (one and a half days self-study, ahead of the content mapping or standard linking as appropriate). The participants who took part in content mapping also completed familiarisation activities in relation to speaking and oral comprehension, as well as more detailed familiarisation with SfL curriculum and specifications (2 days self‑study).

Individual self-study familiarisation included the following key aspects for all the participants:

The participants who took part in the content mapping ahead of standard linking also did the following activities:

familiarisation with oral comprehension and spoken production and interaction CEFR scales

consideration of examples of spoken performances with known CEFR levels and key features that distinguish between performances at different CEFR levels

rating speaking performances with known CEFR levels

consideration of the most salient aspects that distinguish between oral comprehension tasks targeted at different CEFR levels, using exemplar tasks with known CEFR levels

exercises in rating oral comprehension CEFR benchmark tasks

familiarisation with SfL specifications for speaking and listening and with the core curriculum

Following the self-study activities, ahead of standard linking for the reading comprehension components, a half-day discussion meeting was conducted online using MS Teams, where feedback on the activities related to standard linking for reading comprehension was provided and participants were able to discuss their responses. The meeting was attended by all the participants and it was recorded for quality assurance purposes. The discussion meeting included:

There was no separate face-to-face meeting ahead of the standard linking for writing because this was carried out using CJ methodology, which does not normally require face-to-face standardisation (cf. Black & Bramley, 2008; Curcin, Howard, Sully & Black, 2019; Jones, 2009). The CJ methodology does not require intimate knowledge of a rating scale and, by the very nature of the method, helps to cancel out severity or leniency effects in judgements, which is why it was chosen for use in this study.

While some aspects of the training were open book, the participants were encouraged to approach the tasks professionally and were given sufficient time to enable them to devote due attention to all activities. The participants' responses to our familiarisation evaluation questions suggest that they generally considered the activities effective in helping them use the CEFR and the relevant linking methodology confidently in the context of the current study.

Stage 2: Content mapping to the CEFR

This exercise involved relating the content, construct and intended demand of Entry 1 and Entry 3 SfL qualifications from the 4 AOs in our sample to the CEFR. It provides the initial, tentative link to the CEFR, based on profiling the content and construct of the qualifications in terms of the CEFR categories and levels. It also, therefore, provides an initial indication of the comparability between different qualifications in terms of their content and construct. The content mapping indicates the CEFR level that could be evidenced by learners given the content and nature of the tasks in a qualification.

As already mentioned, the CEFR assumes and advocates a communicative, action‑oriented approach to language learning, and this approach is reflected in the aspects of language competence that are described through its various descriptor scales. As we said, based on the findings of our Study 3, it was deemed broadly appropriate to consider the SfL qualifications in relation to the CEFR, given that the approach suggested in the curriculum for these qualifications is intended to be communicative and action-oriented too. However, the current content mapping study helped to understand in more detail the extent to which it is appropriate to link the assessment and performance standards of the SfL qualifications to the CEFR based on alignment of the 2 constructs. This is particularly important given that the SfL qualifications were not developed explicitly with reference to the CEFR.

Method

Content mapping was carried out by three experts, with each qualification reviewed by 2 experts independently of each other. The experts were asked to review the relevant specification documents and 2 samples of live assessment materials (including exam papers/tasks, mark schemes, audio materials for oral comprehension) per level (Entry 1 and Entry 3) and component (reading, writing, speaking and listening). They were provided with the relevant CEFR reference scales, tables and excerpts from the CEFR text to ensure thorough understanding of the relevant aspects to consider in content mapping. To capture their views in relation to the alignment of the relevant aspects of the content and construct of each qualification to the CEFR, they completed a series of forms adapted from the Manual (ibid.):

A1 - general examination description

A9 - oral comprehension

A10 - reading comprehension

A11 - spoken interaction

A12 - written interaction

A13 - spoken production

A14 - written production

A15 - integrated skills combinations

A17 and A18 - spoken and written mediation

A19 - communicative language competence in reception

A20-A22 - communicative competence in interaction and production

A24 - confirmed estimation by skill/component

The experts were also asked to:

Their completed forms were reviewed by an Ofqual researcher and then discussion meetings were held remotely via MS Teams with the experts to come to an agreement between each pair of experts on any discrepancies in individual views. There were separate discussions for each qualification. The final CEFR ratings were confirmed at the end of each discussion.

Results

The tables below present the results of the content mapping for each of the 4 AOs in our sample, for each of the components at Entry 1 and Entry 3[footnote 13]. In some cases, different tasks, task types or a significant proportion of questions within a single component were deemed to be at noticeably different levels (or sub‑levels), and these are represented by slashes in these tables. Each of these tables is followed by a table listing the CEFR activities and strategies scales that are relevant for describing each component across most AOs at each level. In the cases where the relevant scales do not apply to all 4 AOs, the number of AOs to which they do apply is noted in brackets.

General observations

As can be seen from the tables below, it was possible to map a range of aspects of language competence assessed in the SfL qualification to the CEFR. This is an important finding as it highlights the essential correspondence between the core curriculum, SfL specifications and the CEFR across the majority of the content.

The specifications and assessments were thought to be mostly communicative and functional in approach and were mostly deemed to involve useful and meaningful tasks, particularly in speaking but also in writing components. In most cases, the amount of language production/interaction required in the speaking and writing components was deemed appropriate for the relevant CEFR level implied. The reading comprehension assessments were deemed to be least in line with a communicative, functional approach.

Across all specifications and components reviewed, all 4 principal domains of language use were represented (namely, public, personal, occupational and educational). As can be seen from the tables below, there was a reasonable degree of correspondence between the AOs in relation to aspects of language competence assessed by each. However, some discrepancies between the AOs and components were observed in the nature of the tasks or amount and nature of language required, often consistent with the findings from our studies 1 and 3. These discrepancies are generally reflected in the CEFR level or sub-level assigned to each AO and component and can also be observed from the differences in applicability of different CEFR scales to different AOs and components.

All aspects of the CEFR communicative language competence scales were found to be relevant across all SfL components at both levels. In general, the experts noted that there was little in the way of integrated skills assessed explicitly. Therefore, these are not presented separately in the tables below. There was also very little assessment of mediation, apart from some limited aspects present in the speaking and listening component with some AOs.

As can be seen from the tables below, at Entry 1, most AOs and components map to A1 or A1+. However, speaking at Entry 1 seems to be pitched at a higher CEFR level (A1+/A2) than the other components (A1/A1+), and this is the case across the 4 AOs. This pattern for speaking seems to be in adherence with the level implied in the curriculum according to our Study 3. At Entry 3, most of the AOs and components map onto B1 or B1+ CEFR level.

Table 4 Content mapping at component level - Entry 1

Table 5 The CEFR scales represented in SfL components - Entry 1

Reading correspondence
Reading for orientation
Reading for information and argument
Identifying cues and inferring (more clearly represented in 2 of 4 AOs

Written interaction:
Correspondence
Notes, messages and forms
Written production:
Creative writing (3 AOs)

Oral comprehension:
(3 AOs)
Understanding conversation between other people
Understanding audio media and recordings
Understanding announcements and instructions (1 AO)
Oral interaction:
Asking for clarification
Understanding an interlocutor
Informal discussion (with friends) (3 AOs)
Conversation
Information exchange
Oral production:
Sustained monologue -describing experience/ putting a case (2 AOs)
Mediating a text:
Relaying specific information in writing (1 AO)

Table 6 Content mapping component levels - Entry 3

Table 7 The CEFR scales represented in SfL components - Entry 3

Reading correspondence
Reading for orientation
Reading for information and argument
Reading instructions
Identifying cues and inferring (3 AOs)
Explaining data (1 AO)

Written interaction:
Correspondence
Notes, messages and forms
Goal-oriented online transactions and collaboration (3 AOs)
Written production:
Creative writing (3 AOs)
Reports and Essays (3 AOs)

Oral comprehension:
(3 AOs)
Understanding conversation between other people
Understanding audio media and recordings
Announcements and instructions (1 AO)
Oral interaction:
Turn taking
Co-operating (2 AOs)
Goal-oriented co-operation
Asking for clarification
Informal discussion (with friends) (3 AOs)
Obtaining goods and services
Information exchange (3 AOs)
Understanding an interlocutor
Oral production:
Sustained monologue - describing experience/putting a case/giving information
Addressing audiences (2 AOs)
Oral mediation:
(2 AOs)
Facilitating collaborative interaction with peers
Collaborating to construct meaning

Some discrepancies between the AOs

At Entry 1, main differences between the AOs are apparent in writing, where 2 AOs appear to assess at a somewhat higher level in some of their tasks (A1+/A2) compared to the rest (A1).

At Entry 3, some discrepancies were observed across different components. In reading comprehension, AO1 was deemed to assess at a slightly lower level than the others (B1 vs. B1+). AO4's writing assessments appear to be somewhat lower than the others at B1, compared to up to B1+ with the others. In speaking, AO4 appears to assess at somewhat lower level than the others (B1), while AO2 appears to assess at a noticeably higher level (B1+/B2). In addition, one AO does not assess oral comprehension in its speaking and listening components at either Entry 1 or Entry 3 levels, which necessarily affects the nature of language competence evidenced in these assessments compared to the other 3 AOs in the sample.

Discrepancies with the CEFR

Where the specifications and assessments departed from the CEFR this was mostly in terms of 3 key aspects:

occasional use of assessment approaches and task types that do not elicit communicative language competence sufficiently well

approach to literacy assessment in relation to:

the assessment of Latin alphabet knowledge, which is assumed rather than described in the CEFR at the corresponding levels

the requirement of a degree of accuracy in spelling and punctuation, and an awareness of the writing process, not expected at corresponding CEFR levels

approach to assessment of oral comprehension

The first category of discrepancies involves, for instance:

limited use of sufficiently interactive writing and speaking tasks in some assessments

occasional assessment of knowledge of grammar or spelling through non‑functional non-communicative gap-filling or selected response tasks within the writing component, which may be more appropriate for instructional use

assessment of alphabetical ordering in a context‑free way rather than in relation to application in using reference materials

assessment of declarative knowledge of text features (comprehension of words such as: bullet points, diagrams, columns, titles, sub-headings) or grammatical features, rather than assessment of their functional use or how they contribute to text meaning

assessment within a single topic area across all components at the same level

These discrepancies represent a departure from what might be considered best practice in terms of action-oriented communicative approach typically assumed by the CEFR. While these discrepancies do not entirely preclude the possibility of mapping to the CEFR, they do limit the interpretation of the indicative levels in terms of how much evidence they elicit of learners' communicative ability in relation to some of the above-mentioned aspects. It should be noted, however, that such issues were only present in the minority of AOs reviewed, and usually only in a subset of their components (mostly in reading comprehension). Most of those issues were noted by the subject experts in our study 3 as well, as aspects that were at odds with the core curriculum.

The mismatches in categories 2 and 3 make the CEFR content mapping fuzzier, reflecting the specific context of the SfL qualifications. The discrepancies in the second category most notably include assessment of knowledge of Latin alphabet in Entry 1 reading comprehension components. This particular aspect cannot be mapped to the CEFR as the descriptive scales assume rather than elaborate on this knowledge. Furthermore, the fact that this is explicitly assessed through a subset of questions within the Entry 1 SfL reading comprehension tests, while at the same time assuming that the learners can engage with and read the other reading comprehension questions as well as the stimulus texts, makes it unclear why such explicit alphabet-related questions are at all necessary within the assessments at this level. Arguably, they may be more appropriate at pre-entry levels or only for instructional/formative purposes.

Furthermore, some Entry 1 assessments, which are broadly at A1/A1+ CEFR level, appear to require a degree of orthographic accuracy (such as spelling, use of capitalization) that do not appear until B1 level in the CEFR. In addition, in the CEFR, the process of writing is not addressed explicitly, whereas in Entry 3 SfL specifications for writing, the following are very clearly specified and required within some assessment criteria:

While the CEFR does include reference to some of the above-mentioned aspects in relation to pragmatic and sociolinguistic appropriateness, as well as in its orthographic control scale, or under coherence and cohesion, these are not always explicitly related to writing only, and are couched in terms of achieving communicative aims rather than being the focus of assessment.

In the context of the SfL qualifications, such requirements in the domain of literacy are related to their link to the NSAL, which are primarily targeted at learners for whom English is the first language. The results of our Study 3, as well as this CEFR content mapping, suggest that perhaps such requirements may not be entirely appropriate at all language ability levels in the context of learning English as a second language, and, in particular in the migration context. There are suggestions in the literature (ALTE, 2016) that in the migrant context, the focus on linguistic accuracy may not be as relevant as the ability to get the main idea across. Arguably, passing a test should be determined by how much the candidates have understood the task and how appropriately they completed it with more attention placed on pragmatic aspects, and less on grammatical ones. This may be particularly true of learners up to and including B1 level where language learning tends to be for general communicative purposes and provides a basis for the development of more advanced proficiency (B2-C2) that increasingly interacts with academic, vocational and professional use of the target language (Little, 2012).

The CEFR, and the ESOL core curriculum to an extent, appear to recognise this more clearly than the NSAL. This potentially raises questions about whether mapping English as a second language qualifications, such as SfL, to standards intended for English as the first language qualifications is entirely appropriate.

The content mapping also demonstrates that there is not a great deal of emphasis on oral comprehension in the SfL specifications and assessments, which we highlight as the third category of discrepancies. Oral comprehension is defined in the CEFR as one-way comprehension, for instance as a member of an audience or comprehension of audio media and recordings. The SfL specifications and assessments place more emphasis on assessing oral interaction. This, in itself, may not necessarily be seen as a limitation of the SfL specifications and assessments, as long as it is consistent with the agreed purposes and the curriculum. It was noted in the results of Study 3 that these approaches are not entirely aligned with the core curriculum either, as oral comprehension gets a reasonable amount of attention in the curriculum.

Implications of assessment quality issues for the CEFR mapping

In this content mapping exercise, the experts were asked to consider any particularly prominent issues in task or question design that might limit the interpretation of the mapping. Most issues that were noted relate to:

Such issues can also limit the interpretation of the CEFR linking, where assessment operationalisation rather than the scope of the content specification or intended demand, constrains the nature of the skills elicited by the assessments thus reducing overall validity of both the CEFR link and the assessment itself.

Furthermore, even though all of the AOs reviewed do have certain quality assurance processes in place to evaluate their assessment instruments and results, these may not always be in line with some of the best practice in assessment development (cf. AERA/APA/NCME, 2014; ALTE, 2016; 2020), for instance, absence of field testing of questions or tasks, or occasional failure to evaluate item functioning in tests. Furthermore, as already discussed, the nature of QA challenges and threats to validity are different in the context of internal vs. external assessment models. This further speaks to the need to improve some of the aspects of the SfL qualifications to ensure their optimal functioning in relation to their purposes and therefore the validity of their results interpretations as well as of the CEFR linking.

As already noted, a content mapping exercise can indicate the CEFR level that could potentially be evidenced by the candidates (based on the content, contexts, nature of tasks and assessment criteria), not the pass grade standard. For instance, the indicative CEFR level based on content mapping could in theory be higher than the pass grade standard would be. To establish the CEFR level implied by the standard at the pass grade, standard linking exercises are required.

To get an indication of how SfL reading comprehension components relate to the CEFR in terms of their grading standards, as well as how the (CEFR-linked) pass grade standards compare between different AOs, we linked 4 Entry 3 reading comprehension tests, one from each AO, to the CEFR. It is standard practice that the linking of assessments for receptive skills is done with reference to assessment demand rather than performance quality, due to these assessments typically involving selected response and short-answer questions, which may not support holistic judgements of performance quality. Following Brunfaut & Harding (2013), we used a modified version of a standard-setting procedure called the Basket method to conduct standard linking.

Method

The Basket method

The Basket method requires the participants in a standard-linking panel to rate each task[footnote 14] in a test in terms of the first CEFR level at which candidates could reasonably be expected to respond to tasks like that correctly, effectively placing each task into a CEFR level 'basket'. The Modified Basket method additionally requires panellists to select a CEFR sub-level for a task after deciding on the CEFR level first. The sub‑levels are based on the CEFR scales, with low, mid and high sub-categories within each level.

In order to rate each task in relation to the CEFR, each participant needed to answer the following key question:

Which is the first CEFR level describing learners who can answer this task correctly?

During their familiarisation, it was explained to the participants that, in general, global CEFR level descriptors summarise the threshold (criterion) abilities of learners that belong to a level (North & Schneider, 1998). Learners that do not have at least some level of the abilities listed in the descriptor for a particular CEFR level will be at the level below. Therefore, each task may be targeted at a threshold level, or somewhat higher but still within the same overall level. The 'low' sub‑level within each level was defined as the threshold level. The 'high' sub-levels were defined as the 'plus' levels, which are used in some CEFR scales. The plus levels describe learners at the top of the level. These learners will have a full range of the threshold skills and abilities and possibly some elements of the level above. The participants were familiar with these from the scales used in their familiarisation. The 'mid' sub-level was defined as somewhere in between the threshold and plus levels.

To answer the standard linking question for each task or mark point, each participant would need to form a judgement about the features that contribute to the demand of the task and consider how these features would interact with the performance features of learners at a particular CEFR level and sub-level to allow them to answer correctly or not. It was highlighted that this judgement did not imply that learners at a lower level could not give the correct response. It meant that (in the eyes of the panel member) a correct response should not reasonably be expected at lower CEFR levels.

It was emphasised that the focus of standard linking is the demand of each task based on its various features, effectively treating each task as an exemplar of other possible similar tasks that could appear in a test. This is because it is usually not possible to predict accurately how any task will be received by any specific sample of students in a specific test unless the questions are field tested appropriately. Therefore, it is usually conceptually simpler to focus on the key task features, apparent in the task itself, that are likely to affect demand rather than trying to estimate question difficulty (how this task may have performed on the actual test). Furthermore, statistical information about question difficulty was not available to us to consider alongside the panel ratings.

Procedure

All 11 participants took part in the standard linking after familiarisation. They reviewed and rated 4 live Entry 3 reading comprehension tests, provided to us by the AOs.

The standard linking activities were conducted online, in 2 rounds. First, the participants were given about a week to complete their independent ratings for each of the 4 tests and submit the ratings via an online survey tool. These constituted the initial round of ratings.

The initial ratings were then collated and the distribution charts of ratings for each task and mark point created to present to the participants for discussion. An online discussion panel meeting was then held over 2 half-days during which the initial ratings for each test in turn were discussed. The discussion was chaired and facilitated by an Ofqual researcher. During the discussion, the panellists were asked to justify their ratings, and to consider the rationale of other panellists for theirs. The panel sessions were video recorded with the consent of the participants.

After the discussion, the panellists were asked to consider their ratings again independently and to amend any ratings where they could see that their own rationale may not have been appropriate, based on panel discussion. It was emphasised that they did not need to amend their ratings if they could not see a good reason to do so. In other words, there was no imperative to agree with the others. This second round of ratings then constituted the final round. These ratings were used as the basis for calculation of the pass cut score on each test which would represent the relevant CEFR level, as described below.

Data analysis

In order to calculate the mean ratings across panellists (henceforth, raters) for each task and mark point, following Brunfaut and Harding (ibid.), the sub-level scale with low, mid and high categories was transformed into a continuous numerical scale shown in Table 9.

Table 9 Numerical rating scale categories - CEFR sub‑levels

Individual rater scores were transformed into the numerical scores from Table 9. These were then averaged across raters to get a mean score for each task and mark point. The mean scores were then transformed back into the CEFR sub‑level as well as the CEFR level ratings, using the ranges shown in Tables 10 and 11. The rationale for these ranges was that, as each CEFR level was conceptualised as containing low, mid and high compartments, the best way of classifying ratings at each compartment would be to divide the level into 3 equal parts and select the midpoint of each of these parts as the scale point. A similar logic was applied to the sub‑levels.

Table 10 Task score ranges - CEFR sub-levels

Table 11 Task score ranges - CEFR levels

The test cut scores were calculated based on sums of weighted frequencies of the mean CEFR level ratings. The approach taken for calculating the cut scores followed Brunfaut & Harding (ibid.) and de Jong (2009) in using weightings that imply a 'comfortable' achievement of a particular level. The weightings were based on the notion of each cut score representing the level at which a candidate can answer 50% of the tasks at that level correctly, 80% of the tasks at the level below, and 95% of the tasks at the levels below that. This is based on typical Item Response Theory probabilities, which was the method by which CEFR descriptors were scaled in development (Brunfaut and Harding, ibid.; de Jong, ibid.). This approach was taken to guard against a common criticism of the Basket method that it tends to result in cut scores that are too lenient (cf. the Manual, ibid., Kaftandjieva, 2010).[footnote 15]

For instance, based on the following frequencies of tasks rated at each CEFR level (Table 12), the cut score for A2 level would be calculated as 1x0.80+37x0.50=19.3 (rounded to 20). The rounding applied was to the next larger integer.

Table 12 Example frequency table

The mean task sub-level ratings were used to categorise the tasks within each test in relation to their content specification and to consider whether the demand implied by the CEFR sub-level of each task was sufficiently well targeted to ensure that demand distribution of the tasks was appropriate across those targeting specific ACs, as well as across the whole test. It was important to consider CEFR task demand targeting at AC level as well as at the whole-test level because, as described in the Assessment review section, some of the reading comprehension tests apply the mastery model, whereby the learners have to get a certain number of marks for each AC in order to get the pass grade rather than just a particular compensatory test-level score.

Results

Evaluation of judgement consistency

The analysis of the results of the standard linking for the reading comprehension tests involved checking that the outcomes are based on reasonably reliable and consistent rater judgements. This was established by calculating intra-class correlations (ICCs) (Hallgren, 2012) on individual rater scores.

We report both agreement (showing agreement level in absolute terms) and consistency (showing agreement in terms of rank order of values) inter-class correlations (ICCs). An ICC score between 0.6 and 0.74 is usually considered good, and over 0.75 excellent (Cicchetti, 1994).

To check agreement and consistency levels of raters before discussion, and thus get a sense of how well they understood and were able to use the CEFR scales independently following the training that was provided, we calculated ICCs for the initial ratings. It can be seen from Table 13 that even based on the initial ratings the ICCs were reasonable, and in some cases very high. This suggests that our participants were able to use the CEFR scales independently with a high level of agreement and consistency.

Table 13 ICCs based on initial ratings

We also calculated the ICCs for the final ratings, given the panel discussions - presented in the table below. It can be seen in Table 14 that agreement and consistency levels of the ratings improved after discussion. The cut score and linking analysis is based on these final ratings.

Table 14 ICCs based on final ratings

In this section, we present the outcomes of the linking exercise in relation to percentage scores and proportions of tasks/marks in each test belonging to different CEFR level or sub-level categories. Figure 1 shows the distribution of task sub‑level ratings, while Table 15 shows the mean CEFR sub-level rating, standard deviation and range for each test. It can be observed from Figure 1 that most of the tasks across the 4 tests are rated B1 low to B1 mid (cf. Table 11). However, in some cases there is more of a spread across the A2 and B1 levels, and in one case there are also tasks at A1 level alongside some B1 high tasks.

Figure 1 Box plots[footnote 16] showing task sub-level rating distribution by test

It can be seen from Table 15 that the AO2 and AO3 tests have the means within the B1 score range, while the AO1 and AO4 tests have the means at the top of A2 range, suggesting that these tests are somewhat less demanding overall.

Table 15 Descriptive statistics for task sub-level ratings by test

The indicative CEFR mapping is shown in Table 16 below. Column 2 shows the CEFR (compensatory) cut scores as the percentage of total marks on each test that would be required to demonstrate B1 level of performance. The CEFR cut scores were calculated based on sums of weighted frequencies of the mean task CEFR level ratings, as described in the Data analysis section[footnote 17]. These cut scores, alongside the task demand profile shown in Figure 1 above in terms of mean sub-level CEFR ratings, suggest that the 4 tests are broadly comparable in terms of their indicative CEFR level. However, these CEFR cut scores are derived in a compensatory way and cannot be directly interpreted as indicative of the CEFR level implied by the actual operational cut scores of these tests.

This is because the actual operational cut scores are not all compensatory and differ in important ways in how they are defined. Given different approaches to defining the pass grade cut scores, with some AOs having AC-level, section-level or item-level hurdles, sometimes with additional compensatory test‑level cut scores and sometimes without, it is necessary to consider the demand profile of items targeting individual hurdles and what this reveals about the indicative operational CEFR level.

To enable an approximate comparison of the indicative CEFR cut scores and the operational cut scores/hurdles, we cross-referenced the individual tasks with the relevant test specifications, where each task is typically assigned to a specific AC, test section or other hurdle where these are used. For each test, we then calculated the proportion of tasks/marks at each CEFR sub-level which would need to be answered correctly to pass each hurdle and achieve the operational cut score (however this cut score was defined). We assume that learners would be more likely to answer any items at a lower CEFR level correctly and less likely to answer higher‑level items (consistent with the weightings for calculating the CEFR cut scores). Recall that the method for calculation of the CEFR cut scores assumes that 50% of the B1 tasks contribute to the cut score in addition to any lower-level tasks (column 3 in Table 16).

Column 4 in Table 16 shows which proportion of the total number of B1 level tasks/marks in each test needs to be answered correctly to pass the relevant hurdles and achieve the operational cut-scores (the remaining tasks that need to be achieved are A2 or lower). It can be seen that, except for the AO1 test, over 50% of the B1 tasks in each test need to be achieved to achieve the cut score and/or different hurdles. However, in the AO1 test, only 20% of the B1 tasks in the test need to be answered correctly to achieve the hurdles.

Table 16 Indicative CEFR mapping[footnote 18]

The indicative operational CEFR levels for each test, based on taking into account the proportion of B1 tasks contributing to the operational cut scores, are shown in column 5. Considering the CEFR demand profile of the tasks required to achieve operational cut scores/hurdles, it can be concluded that the operational criteria for the pass grade translate into B1 CEFR level for 3 of the 4 tests, with the proportion of B1 items likely to contribute to achieving the relevant hurdles similar to the proportion contributing to the corresponding compensatory CEFR cut scores. Indeed, the CEFR-linked percentage cut score is lower than the percentage cut score at operational pass grade boundaries for these 3 tests (the operational percentage cut scores are not presented here for anonymity reasons). With the AO1 test, the minimum pass criteria do not translate into reliable evidence of B1 level, given that only a very small proportion of items at B1 level may be required to achieve the pass grade. This is despite the fact that the overall profile of this test and the implied compensatory CEFR cut score do align with B1 level, similarly to the other 3 tests. If the operational criteria for achieving the pass grade on the AO1 test were brought in line with the other tests, the standards on all 4 tests would be broadly comparable and at B1 level.

Some caveats

There are some caveats to these conclusions. It was noted in the content mapping results section that some of the assessment approaches in these tests may not elicit evidence of the construct of reading comprehension in line with the CEFR and its action-oriented communicative approach. This particularly relates to the following 2 aspects:

assessment of alphabetical ordering in a context‑free way rather than in relation to application in using reference materials

assessment of declarative knowledge of text features (words such as: bullet points, diagrams, columns, titles, sub-headings) or grammatical features, rather than assessment of their functional use in terms of how they contribute to text meaning

The former aspect is typically specified in terms of ACs that themselves suggest context-free non-functional approach to assessment (for instance, "Use first and second placed letters to order words in alphabetical order"). These tasks were rated as A2 low in all tests, though this mapping is not entirely appropriate and is there to indicate potential demand of such tasks were they to use an approach more in line with the CEFR, in relation to the "Reading for orientation" scale and its low A2 descriptor "Can locate specific information in lists and isolate the information required (for instance, use the Yellow Pages to find a service or tradesman)".

The latter aspect is specified in the ACs that suggest communicative approach (for instance, "Use language features to identify meaning in short straightforward text"), in line with the CEFR and its "Identifying cues and inferring" scale. Therefore, in the latter case, the misalignment appears to stem from assessment design choices rather than construct definition. Furthermore, the requirements for declarative knowledge sometimes raise the demand level of these tasks beyond what might be considered appropriate for B1 level, though not in relation to reading comprehension construct, but rather in relation to knowledge of linguistic vocabulary denoting text and grammar features.

While these discrepancies do not entirely invalidate the mapping to the CEFR, they do limit the interpretation of the indicative levels observed in terms of how much evidence they entail of learners' communicative ability in relation to the above‑mentioned aspects. Furthermore, these assessment approaches were noted by the subject experts in our study 3 as being at odds with the spirit and intention of the core curriculum too. In this sense, these discrepancies represent a departure from what might be considered best practice in terms of the CEFR action-oriented communicative approach as well as in terms of the curriculum. These assessment approaches should therefore be reconsidered to increase the overall validity of the way in which these tests elicit the evidence of the intended constructs. It should be noted that similar assessment approaches are used across all 4 tests, and therefore do not impact on the conclusions about comparability of standards between them.

In addition to these issues, it was already noted in our study 1 that a number of items across all tests sometimes had implausible distractors or guessability and other issues. Some of these issues were also flagged by the subject experts and the CEFR experts. The CEFR mapping additionally demonstrated that some of the items are poorly targeted to the intended demand at the cut score and do not contribute appropriately to the reliability of the pass grade, which is not intended to differentiate between multiple competence levels. More items closer to the intended demand at the pass grade threshold would help increase test reliability and classification consistency. Where tests are fairly short, this compounds other issues that could potentially reduce test reliability and classification consistency (for instance, guessability). While these are all general assessment design issues that could be improved, they do currently reduce the validity of test score interpretation, whether in relation to the CEFR linking or more generally.

The fourth stage of the linking study was to relate the pass grade performance standards of the writing components from the 4 SfL AOs to the CEFR and to each other. We used the rank ordering comparative judgement method to do this, as previously advocated by Jones (2009) and implemented in Curcin and Black (2019). The focus of this stage was on Entry 1, Entry 3 and Level 1 writing.

Jones (ibid.) showed that the method can be used to replicate the results of standard setting panel ratings using CEFR descriptor scales. In the current study, the method was used to create a common scale of quality across SfL performances with unknown CEFR levels and selected CEFR benchmark performances where the CEFR levels were already known. In this way it was possible to estimate the CEFR level of the SfL scripts from their position on the common scale relative to the CEFR benchmarks.

Method

Rank ordering

Rank ordering is a comparative judgement (CJ) technique for capturing judgements of quality or other subjective attributes of different kinds of artefacts. Previous research exercises have found that rank ordering, alongside other CJ methods, is a valid method for comparing performance standards in educational assessment, for example, between examinations from different years (Bramley, 2007; Black and Bramley, 2008; Curcin, Howard, Sully and Black, 2019; Benton, Leech and Hughes, 2020; Benton, Cunningham, Hughes & Leech, 2020).

The theory underlying the rank ordering method and other CJ techniques is Thurstone's law of comparative judgement (Thurstone, 1927). In essence, a function of relative frequency of occasions on which one artefact is judged to be better than another is an estimate of their separation on the scale (of quality) to be created by the analysis (cf. Bramley, 2007). In CJ methods effects such as differential severity of judges cancel out, while collecting judgement data on a large scale maximises the reliability of the quality scale that is created (Verhavert et al., 2019).

When using this method for comparing assessment performance standards, sets of candidate performances (scripts) from 2 or more examinations are rank ordered in terms of overall quality by multiple judges (such as examiners, subject experts). These rankings are then analysed using a latent trait model (for instance, Bradley-Terry or Plackett-Luce) to place each script onto a single scale of quality. The result of a rank ordering exercise is a common script quality scale across different artefacts (in this case, scripts from different examinations) that were included in the rank ordering exercise.

In the current study, scripts from SfL writing examinations across different AOs and levels were included alongside benchmark scripts from a range of other English examinations, with known CEFR levels (see more details on this below). Therefore, it was possible to see how the SfL scripts and CEFR benchmarks were distributed on this quality scale, and thus relate the performance standards of SfL scripts to the CEFR scale.

Given that latent trait models allow for non‑randomly missing data, it is possible to create judgment allocation designs that are sufficiently sparse to be feasibly implemented in practice, while being sufficiently large‑scale in terms of number of comparisons per script to result in reasonably reliable estimates of the script quality scale. Furthermore, the method involves comparing performances with one another, and putting them in a rank order, rather than using a rating scale, where detailed common understanding of the rating scale is required by all the participants. In the context of this study, while making holistic judgements of script quality does require familiarity with the basic notions and approach inherent in the CEFR and its way of describing writing competence, it does not require detailed (shared) knowledge of the CEFR writing scales by the participants.

Scripts

As already noted, a range of CEFR benchmarks from different sources was used in this study. The sample was opportunistic, obtained either from publicly available sources such as the Council of Europe website, or from certain testing organisations (for instance, the British Council). We sought to use several benchmarks per level in order to ensure that we could capture some of the variety of what different CEFR levels might look like in actual writing performances, acknowledging that no single performance will exhibit all different features of a CEFR level (Little, 2012).

We ensured that the benchmarks came from mostly general English exams, rather than any that are specifically academic or related to specific occupations, to help comparisons with the SfL samples, which are mostly demonstrating general English writing skills, albeit in a migration context. Some of the CEFR benchmarks represent "borderline" performances at particular levels, while some illustrate more secure performances within a level according to the standards of their respective exams. In this exercise, we were not interested in the comparability between the benchmarks from different institutions, and the sources are anonymised in the results section.

The CEFR benchmarked writing performances were obtained from:

the Council of Europe website

Cambridge English teacher handbooks for A2 (Key), B1 (Preliminary) and B2 (First) exams (Cambridge Assessment English, 2016a, b; 2019a, b, c)

CEFTrain project

The British Council Aptis test

ESB ESOL International tests at B1 and B2

Trinity College London ESOL International tests at A2, B1 and B2

The table below summarises the numbers of benchmark samples used for each CEFR level in the study.

Table 17 Number of CEFR benchmarks by level

A small set of benchmarks from ESOL Exemplars (DfES, 2004) was also used to illustrate standards of performance at different levels of the NSAL/core curriculum. ESOL Exemplars samples exemplifying secure performances at Entry 1 (N=2), Entry 2 (N=2), Entry 3 (N=3) and Level 1 (N=3) were used. All indications of the CEFR or RQF level were removed from both CEFR benchmarks and ESOL Exemplars samples.

The SfL samples were obtained from the respective AOs. They all came from live examinations. AO, centre and candidate information was removed from the scripts, as well as marks and annotations.

Ideally, for our SfL script sample, we wanted to have threshold (borderline) pass grade performances at each level from each AO. We also required the samples to have gone through the AO's quality assurance/moderation and to have had confirmed grades. However, it was not straightforward to ensure that all of the performances in our sample did indeed represent threshold performances. One reason for this was purely practical, and partly influenced by the extraordinary Covid situation at the time of data collection for this study. Some AOs did not have enough threshold scripts from recent live administrations that fitted our brief and were not able to access their archives due to Covid restrictions in order to get threshold performances from earlier administrations.

The other reason was related to how the pass grade standard is specified by the AOs. As discussed in Study 1 (Assessment review), some AOs employ traditional marking when evaluating writing performances, and therefore set a numerical cut score for the pass grade at each level. For these AOs, it was possible to request the performances at or near the numerical pass grade cut score. However, some AOs assess by means of descriptive, binary assessment criteria (met/not met) and do not involve numerical scores. For these AOs, it was not straightforward to define what represents a threshold performance. We therefore relied on those AOs to send us the samples that they deemed as representative of a borderline pass. In order to be confident that at least some of the scripts from these AOs did represent borderline passes, we also requested some borderline fail samples from each AO. The intention was to check how the borderline fail scripts in our sample relate to the borderline pass scripts, with an expectation that they should be relatively close in quality if they indeed represented borderline performances.

From each AO and for each of Entry 1, Entry 3 and Level 1, we requested 7 scripts at or near the pass grade borderline standard/cut score, and 3 borderline fail samples. For Entry 2, we requested 3 scripts at or near the pass grade borderline standard cut score. While Entry 2 was not the focus of our exercise, we wanted to include some Entry 2 scripts to ensure stronger linking across the entire quality scale from Entry 1 to Level 1.

The table below shows the number of samples per AO and level included in the rank ordering exercise. Note that we did not always receive the requested number of samples from all AOs[footnote 19], while receiving some extra samples that we did not specifically request (for instance, Entry 2 fail grade samples). AO4 in particular provided very few samples across the board, hence any conclusions about their results will necessarily be more tentative compared to the others.

Table 18 SfL script sample by level

Judging design

In total, there were 146 scripts to rank order, across the benchmarks (N=40) and the SfL samples (N=106). These were distributed across the judges into sets of 4, with each judge required to rank order 26 or 27 sets. All 11 participants took part in this exercise.

The judging allocation design was created such that it maximised the number of times each script was seen across judges while minimising the number of times each judge saw a particular script. Literature suggests that over 20 paired comparisons per artefact should be sufficient for a reasonable level of scale reliability (Verhavert et al., 2019). Therefore, the design ensured that each script was seen 8 times across different judges and sets, translating into 24 comparisons per script. Each judge saw most of the scripts once, and a maximum of 2 times. Each judge had a unique combination of scripts.

Traditionally, where the aim of a rank ordering exercise is to compare scripts from 2 different examination sessions, the sets might contain equal numbers of scripts from each session. In our study, because the number of scripts was not equal between the benchmarks and the SfL samples, and because it was equally important to ensure reliable judgements for both of these sets of scripts, we did not constrain the design in terms of script type in each set. Therefore, some sets contained 2 benchmarks and 2 SfL scripts, while some contained only 1 or none of one of the categories.

We constrained the design so that scripts that were more than 1 level apart (either on the CEFR or RQF scale) could not coincide in a set. There were no constraints regarding how many scripts at each level were in any set. Therefore, some sets contained all scripts at the same level, while some contained a combination of, for instance Entry 1 and Entry 2 or B1 and B2.

Table 19 Key features of the judging allocation design

Procedure

The participants were contracted for one and a half working days to carry out the rank ordering exercise. The exercise was conducted online, using the Cambridge Assessment Comparative Judgement Tool. This platform allows for electronic pdf script files to be uploaded and allocated to the judges according to a pre-specified judging design.

During their familiarisation activities, which preceded the rank ordering exercise, the participants were given the opportunity to access practice scripts and rank order them using the CJ platform. At the end of their familiarisation, they confirmed that rank ordering 4 electronic files feasible on screen, and that the CJ platform was straightforward to use.

The participants were provided with detailed instructions about how to complete the rank ordering task. They were asked to work independently of each other and not discuss their judgements. An Ofqual researcher and IT support were on hand in case of any issues with accessing the CJ tool.

As already noted, the participants did not know the CEFR levels of the benchmarks, nor did they know the marks or grades of the SfL scripts. They were asked to rank order the scripts in each set from best to worst in terms of overall quality. It was emphasised that they were to consider the important aspects of writing competence according to the CEFR approach and CEFR descriptor scales as the basis for their judgements of quality. They were not provided with any mark schemes for individual exams that the scripts came from.

The judges were asked to try as best as they could to focus on the qualities of performances and try to ignore the fact that these were based on different examinations, with different tasks and layouts. Furthermore, some scripts (typically the SfL scripts) were handwritten, while some were typed up (typically the benchmarks, though a few of these were also handwritten). The judges were alerted to this in their instructions and advised to try and not allow this to affect their judgement of script quality.

All SfL scripts and most benchmark scripts included at least 2 writing tasks. However, a small number of benchmark scripts included a single task. In such cases, quantity of writing in isolation should not have been used as a criterion when comparing with scripts based on 2 or more tasks. The judges were instructed to try and extrapolate as much as possible about the writing quality from the benchmark scripts based on a single task, focusing on key aspects of performance rather than the quantity of writing. We advised that, when considering such benchmark scripts, they might assume that the candidate who wrote a response to that task could write a piece of similar quality for another task or on another topic.

In general, the judges were asked to consider the whole work of each candidate. There were 2 exceptions to this. Firstly, most SfL scripts contained a form-filling task. In some cases, this was a basic form requiring candidates to enter some personal information. Given that this would reveal the identity of the candidates, we had to redact most or all of this information. Therefore, in most cases, this could not contribute to the overall judgment, and the judges were asked to focus on other, more substantive, writing tasks in those scripts. In some cases, the forms contained a section requiring an open response. The judges were asked to take those into account when judging.

Secondly, one awarding organisation's SfL writing papers contained a "use of English" task, with several questions focusing mostly on aspects of grammar or spelling. The judges were instructed to take the aspects of performance revealed in that task into account as far as this helped to form a judgement about a candidate's overall grammatical or spelling competence, with an expectation that their overall judgement would mostly be informed by the communicative writing tasks in those scripts.

Data analysis

The rank orders obtained from the judges in the rank ordering exercise were converted into paired comparisons and a single 'perceived quality' scale across the SfL and CEFR benchmark scripts was estimated using the Bradley-Terry model (Bradley & Terry, 1952)[footnote 20] implemented in the Sirt package in R (Robitzsch, 2019).

Each script is positioned on this scale in terms of its estimated measure (of quality), which is related to the probability of it being judged better than another script in a paired comparison. The unit of the script quality scale is known as a 'logit' or 'log‑odds unit'. The model can be stated as:

where P ij = the probability that script i beats script j in a paired comparison

B i = the measure for script i

B j = the measure for script j

After the initial analysis run, some data cleaning was undertaken to improve the fit of some judges or scripts where appropriate. In order to preserve most of the data, we removed misfitting observations (based on highest standardised residuals for individual paired comparisons) rather than all judgements from a judge or script that showed some misfit. Initially, observations with standardised residuals higher than 4 were removed from analyses, and analysis rerun. After that, in order to further improve the fit of a couple of judges and scripts, a few more observations with high residuals were removed and analysis rerun. In total, 34 observations out of 3506 were removed. In addition to this, 2 outlier benchmark scripts were removed from analysis. These were flagged as outliers in terms of the logit measure compared to the other scripts at the same CEFR level. The results based on the cleaned data were evaluated in terms of model fit, scale properties and mark-measure correlation, as is standard in rank ordering exercises, and as explained below.

One way of checking whether a rank ordering exercise has worked is to check the properties of the scale of perceived quality created from the judgements. This involves investigating scale separation reliability (SSR) and model fit, which are the usual checks conducted for any latent trait analysis (cf. Bond & Fox, 2007).

The SSR coefficient is analogous to the person separation reliability in Rasch modelling (Andrich, 1982) and to KR-20, Cronbach Alpha, and the Generalizability Coefficient. It is calculated as:

SSR = ((Observed SD) 2 - MSE)/(Observed SD) 2

where Observed SD is the standard deviation of the estimated measures, and MSE is the mean squared standard error of the estimated measures across all the scripts.[footnote 21] In this context, SSR means "reproducibility of relative measure location" (cf. Winsteps Manual). High reliability of the script measure scale would mean that there is a high probability that those scripts estimated with high measures actually do have higher measures (meaning that they are of better quality) than the scripts estimated with lower measures.

In general, the decision of whether the SSR can be considered satisfactory will depend on the purpose for which the scale is constructed, as well as on the context and type of the assessment under consideration. Verhavert et al. (2019) cite 0.7 as the level mentioned in the literature as appropriate for low-stakes or formative assessments, and 0.9 as the level often accepted as appropriate for high-stakes and summative assessments. In the rank ordering studies carried out to date, SSR of around 0.8 and higher has generally been judged as satisfactory and related to other aspects of the comparative judgement exercises being judged as satisfactory too.

A common way of checking overall model fit is to check the overall proportion of misfitting judgements. Usually, this should be at or below what would be expected by chance, or, less than about 5% of standardised residuals using the criterion of 2 for the absolute value of the standardised residual, and less than about 1% using the criterion of 3 (cf. Linacre, 2011). In addition to that, it is usually necessary to check the usual Rasch fit statistics for the scripts and judges (Wright & Linacre, 1994).[footnote 22] In particular, reasonable fit statistics of the judges would suggest the consistency of their judgements and a reasonable level of agreement on rank orders across all the judges.

In addition to these, checking the corelation between the test score scale and the measure scale is a way to establish whether the judges in a CJ exercise perceived a trait that is sufficiently similar to the one underlying the test scores. Previous rank ordering studies tended to consider correlations around and above 0.7 as satisfactory. In the current study, it was only possible to get score‑measure correlations on subsets of scripts for which a score or at least a score band was known.

Once the appropriate script quality measures are obtained, their logit scale can be plotted against the SfL levels scale/CEFR scale to observe their relative position and extrapolate the likely performance standards of the relevant SfL pass grade boundary scripts at each level in relation to the CEFR benchmark performance standards.

Results

Evaluation of scale properties and model fit

In order to have confidence in the results of a rank ordering exercise, it is necessary to evaluate the properties of the script quality scale and the model fit. The scale separation reliability (SSR) and separation were high, suggesting that the observed differences between scripts were not due to measurement error. This is shown in the table below.

It can be seen that the BT model overestimates the reliability to some extent. However, the PL estimates are also high and indicative of a reliable script quality scale.

Table 20 SSR and separation coefficients

As already noted in the data analysis section, we have based all of the analyses reported below on the estimates from the BT model. Appendix 5 presents the analysis based on the PL model for comparison, though there were no fundamental differences to the conclusions based on these analyses compared to those based on the BT model. The main difference is slightly less differentiation (more overlap) between the A2 and B1 script measures compared to the BT model, consistent with lower separation and shorter scale.

The overall model fit can be seen in the table below. Model fit was satisfactory, with less than 2% standardised residuals greater than absolute 2 and less than 1% standardised residuals greater than absolute 3.

Table 21 Overall model fit

Individual judge fit was also satisfactory, with all infit and outfit mean squares lower than 1.5, suggesting that the judges were consistent in their judgements. Script fit statistics were also largely lower than 1.5, consistent with the overall satisfactory model fit. There were several scripts that won or lost all of their comparisons, resulting in imputed measures.[footnote 23]

In order to get a sense of how well the rank order of scripts produced in this exercise matched the rank order based on the CEFR or RQF level, we calculated correlations between script measures and level, where levels were turned into numerical categories 1-4. The correlations were 0.89 for the CEFR benchmark scripts and 0.85 for the SfL scripts.

Where original scores or score bands (lower and higher A1), were known, which was for a subset of CEFR benchmarks and for the scripts from one SfL AO, we also ran score-measure correlations for those, separately for each benchmark group and SfL script group. With respect to the CEFR benchmark groups, while these correlations are based on a very small number of scripts in each group (min 2, max 4), they do suggest that the judges were able to discern script differences largely consistent with the original score or score band rank order.

Table 22 Original mark (band) correlation with script measure - benchmark scripts

With respect to the low correlation in the benchmark group 4 at A1 level, it comes from one of the higher band scripts having the lower measure than the band one scripts. With that script excluded, the correlation would be 0.96. At B1 level for group 4 benchmarks, the script with the lower score band had the higher measure. The low correlation at B2 level for group 2 benchmarks also represents the reversal of the script score compared to the measure. In this case, the scripts were only 1 score point apart, hence it is unsurprising that such reversal may have occurred.

Script scores were also available for one SfL AO. As can be seen from the table below, some correlations between measures and script scores were noticeably lower, though this is to some extent unsurprising given that the scripts within each RQF level represented a narrow range of marks around the cut score, and hence a relatively narrow range of quality. This is a familiar pattern, also observed in Curcin and Black (2019) with GCSE scripts on and around grade boundaries.

Table 23 Script score-measure correlations - SfL scripts

Comparability between CEFR benchmarks

The chart below presents the distribution of benchmark script measures by benchmark group and the CEFR level of the scripts. There were 2 outlier benchmarks at A2 level, which were removed from analysis.

It can be seen that there is some variability in the perceived quality of scripts within each CEFR level. This was to be expected given that the scripts representing each level varied in terms of original marks/mark bands (for instance, with some scripts at the borderline and some secure within a level) or came from different examinations that might have somewhat different standards compared to others. Nevertheless, the scripts in higher score bands within their organisations tended to get higher measures than the borderline score band scripts. Furthermore, it can be seen that there are clear increases in quality between A1, A2 and B1 levels in particular, though there is more overlap between B1 and B2 levels.

Figure 2 Comparability between CEFR benchmarks

This is further demonstrated in the table below, which shows median and mean logit measure of script quality by CEFR level. It can be seen that there is a clear distinction between the levels in terms of median and mean measure for all the scripts representing a level. In the remaining discussion, therefore, we grouped all the benchmark scripts representing a level into a single benchmark group, comparing these joint distributions by CEFR level with those of the SfL scripts.

Table 24 CEFR benchmark descriptive statistics by level

Borderline SfL script check

It was not always straightforward to ensure our SfL sample contained truly borderline pass grade scripts. This was particularly problematic where the pass grade was not defined in terms of numerical marks but rather in terms of binary criteria fulfilment. We therefore requested a small sample of borderline fail scripts from all AOs, to check that the pass grade scripts in each AO's sample were reasonably close in quality to borderline fail scripts. Charts 3 to 6 display the measures that each AO's fail and pass scripts obtained, plotted next to each other for each RQF level.

Figure 3 SfL sample borderline script check - AO1

Figure 4 SfL sample borderline script check - AO2

Figure 5 SfL sample borderline script check - AO3

Figure 6 SfL sample borderline script check - AO4

It can be seen that in the majority of cases, where both fail and pass scripts were available, they were either fairly close in terms of their measures, or there was a degree of overlap, with some fail scripts even judged to be superior to some of the pass scripts (for instance, AO3 at L1). In the case of AO4, the data submitted by the AO was not always clear as to whether each script was borderline or not. Given the difference between the fail and the pass script measure for this AO at Entry 2, it is likely that at least one of them was not borderline.

Overall, it can be concluded that at least some of the pass scripts that were submitted did represent borderline pass standard. For the analyses presented in the next 2 sections, we therefore only focus on the pass scripts.

Comparability between SfL AOs

One of the main questions in this phase of the study was to consider how comparable the performance standards of candidates at the pass grade threshold are between different AOs in our sample. Recall that AO4 provided a very small number of scripts, so their results are only included for tentative comparisons with the others.

Firstly, as can be seen from the Figure 7 and table 25, there is a fair bit of variability in perceived quality between the scripts from most AOs at each level. This partly just reflects the fact that not all the scripts that were submitted by each AO were borderline.

With respect to how the pass grade standards between different AOs relate to each other at each level, it can be observed that there is a fair bit of overlap between the script quality measures at the bottom end of each level, and particularly at Entry 2, Entry 3 and Level 1 (with the exception of AO4, for which we do not have sufficient data). Within that, AO2 does appear to involve a slightly higher performance standard at the pass grade at those levels. Overall, however, the amount of quality measure overlap at Entry 2, Entry 3 and Level 1 suggests that the standards at the pass grade borderline for the AOs in our sample are reasonably comparable.

There is somewhat more variability between the AOs at Entry 1, with AO1 and AO2 appearing to have markedly different performance standards at the pass grade threshold, with AO3 (and AO4) somewhere in between. Our curriculum review did flag some aspects of AO2 standards as potentially being too high in relation to the curriculum at Entry 1, followed by AO3, particularly with respect to high expected word counts for the level, and this could partly explain their apparently slightly higher standard compared to AO1.

Another notable finding from this exercise is that there is not a great deal of differentiation between Entry 2 and Entry 3 scripts in particular, irrespective of the AO, but also to some extent between Entry 3 and Level 1 across all of the AOs. This is discussed in more detail in the next section.

Figure 7 Comparability between SfL scripts by level

Table 25 Descriptive statistics for each AO script sample by level

In this section we present the results of the linking of the SfL performance standards to the CEFR. The CJ exercise also involved several ESOL Exemplars samples, and below we comment on the relationship between the SfL standards and those of ESOL Exemplars. The ESOL Exemplars show the work of learners who are mainly secure in a level rather than having a borderline performance.

The relationship between the quality measures (performance standards) of the SfL scripts (blue bars), the CEFR benchmarks (orange bar), and the ESOL Exemplars (yellow bar) is presented in the Figure 8[footnote 24]. The chart displays box and whisker plots of the script quality measures from the rank ordering exercise. On the chart, the borderline standard at each CEFR level is denoted by the dotted lines and the relevant CEFR level labels.

Figure 8 Box plots showing the SfL standards mapping onto the CEFR

At Entry 1, the borderline pass grade standard of SfL scripts is well within the A1 level range compared with the CEFR benchmarks, despite some differences between individual AOs (see previous section). In fact, several scripts appear to be of A2 level quality. Therefore, the pass grade standard at Entry 1 can be categorised as mid A1 to A1+. There is also good alignment with the standard of ESOL exemplars, which is also higher than threshold A1 CEFR level.

Similarly, at Entry 2, the pass grade standard seems to be well within the A2 CEFR range, at A2+ level, and somewhat higher than the standard of some ESOL Exemplars.

At Entry 3, it is notable that the SfL script quality is lower than threshold B1 CEFR level for a number of scripts in our sample from AO1 and AO3, as well for one script from AO2 and the only borderline script we had from AO4. In addition, nearly 70% of the SfL scripts in our sample across all the AOs were of lower quality than all except 2 CEFR benchmark scripts. There is also significant overlap between the quality measures of Entry 2 and Entry 3 SfL scripts from all AOs. This would suggest that at least some Entry 3 SfL borderline pass grade standards are more in line with A2+ across all AOs, with around 70% of the sample just in the low B1 level. At Entry 3, the ESOL Exemplars are of noticeably higher standard than the majority of SfL pass grade scripts. The ESOL exemplars straddle B1+/low B2 CEFR level.

At Level 1, the majority of the SfL scripts are at low B2 level, although several borderline scripts do not reach that level, and are more appropriately mapped to B1 mid or B1+ level. Again, the ESOL exemplars are higher in quality than the majority of the SfL scripts in our sample, and well within the B2 level.

Table 26 Relationship between SfL pass grade standard, ESOL exemplars and the CEFR

The findings from the CEFR mapping indicate that the SfL writing assessments from the 4 AOs in our sample are broadly comparable in terms of their CEFR level, despite some discrepancies at the pass grade borderline within A1 level in particular. The more significant issue than inter-AO comparability is the fact that there does not appear to be enough differentiation between different SfL levels at the pass grade threshold, particularly between Entry 2 and Entry 3, which can both be mapped onto A2+ or low B1 CEFR level for at least part of the sample across all AOs. Most of the SfL Entry 3 sample can be mapped to low B1 level at most.

This mapping also indicates that ESOL Exemplars, which exemplify secure performance within the RQF levels of the curriculum, map onto the mid to high A1, A2, B1 and B2 CEFR levels respectively. This suggests that the borderline performance manifestation should be somewhat lower within each CEFR level, but still map onto each level at about its threshold, rather than dropping below it, as seems to be the case with the SfL writing qualifications in our sample at Entry 3 and Level 1. The CEFR content mapping also indicated that the written assessments at Entry 1 and Entry 3 are at around A1/A1+ and B1/B1+ respectively, or rather, that they have the potential to provide performance evidence of those levels. It is another issue where the threshold standard is actually set on any specific assessment, as demonstrated by the Entry 3 pass grade threshold mapping onto A2+ rather than threshold B1 or higher for several borderline scripts.

The findings from this mapping largely confirm what was suggested in the curriculum review when the SfL qualification demands were considered against the curriculum and NSAL standards. Namely, that Entry 3 and Level 1 qualifications may not be sufficiently demanding given the demand implied in the curriculum and relative to the demand of the lower levels. The curriculum review also indicated that the demand of Entry 1 is mostly appropriate, if a little too high with some AOs. This is also supported by the indicative relationship found here between Entry 1 (and Entry 2) SfL samples and ESOL Exemplars.

Therefore, the issue of lack of differentiation between Entry 2 and Entry 3 in particular appears to stem from the standards of Entry 3 likely being too low, threatening the progression of learners onto higher levels. This apparent lack of stretch at Entry 3 may be contributing to the perceived "large jump" in standards at Level 1, which was noted by our curriculum reviewers, as well as some stakeholders.

It is also worth pointing out that even though the Entry 1 qualifications may be compatible with mid-high A1 level according to the curriculum, the question remains whether this is suitable for an initial Entry level qualification. Given that Entry 1 pass grade standard appears to be noticeably higher than threshold A1 CEFR level, this suggests a potential need for separate qualifications at somewhat lower level ('pre-‑entry'), as the current GLH may not enable some learners to achieve the equivalent of mid-high A1 level when starting from the point of no English in SfL Entry 1 qualifications.

Some caveats

As with the CEFR mapping of assessment standards for reading comprehension, there are some caveats that need to be borne in mind when interpreting the CEFR mapping of the writing pass grade standards. These caveats had already been identified in the content mapping phase, and relate to where the assessments or the constructs assessed depart from the CEFR in relation to:

use of some assessment approaches and task types that may not be optimal for eliciting communicative language competence

approach to literacy assessment in relation to the requirement of a degree of accuracy in spelling and punctuation and an awareness of the writing process which are not expected at corresponding CEFR levels

The first category of discrepancies involves, for instance, limited use of sufficiently interactive writing tasks in some AOs' assessments, occasional assessment of knowledge of grammar or spelling through non-functional gap-filling or selected response tasks within the writing component, which may be more appropriate for instructional use, and the approach of assessing within a single topic area across all components. These discrepancies represent a departure from what might be considered best practice in terms of action-oriented communicative approaches to some extent.

While these discrepancies do not entirely preclude the possibility of mapping to the CEFR, they do limit the interpretation of the indicative levels in terms of how much evidence they entail of learners' communicative ability in relation to some of the above-mentioned aspects. It should be noted, however, that such issues were only present in the minority of AOs reviewed. Most of those issues were noted by the subject experts in our study 3, as well as aspects that were at odds with the core curriculum.

The mismatches in category 2 make the CEFR content mapping fuzzier, reflecting the specific context of the SfL qualifications. For instance, some Entry 1 assessments, which are broadly at A1/A1+ CEFR level, appear to require a degree of orthographic accuracy (such as spelling or use of capitalisation) that do not appear until B1 level in the CEFR. In addition, in the CEFR, the process of writing is not addressed explicitly, whereas in Entry 3 and Level 1 SfL writing assessments, the following are very clearly specified and required within some assessment criteria (related to their link to the NSAL):

While the CEFR does include reference to some of the above-mentioned aspects in relation to pragmatic and sociolinguistic appropriateness, as well as in its orthographic control scale, and under coherence and cohesion, these are not always explicitly related to writing only, and are couched in terms of achieving communicative aims rather than being the focus of assessment. Therefore, even though these aspects may have been credit-worthy in achieving the pass grades in SfL assessments, they may not have contributed in the same way to our judges' views of the relative quality of different scripts that they considered in terms of the CEFR description of what constitutes effective writing at different CEFR levels.

Finally, as noted earlier, the form-filling task, which was present in each AO's writing assessment at each level, may have contributed only to a limited extent to judges' overall view of each script's quality. This is partly because some of the student personal details that achieve some credit operationally had to be removed. However, given that those details achieve most credit at lower levels, which in fact map quite highly within A1 and A2 CEFR levels respectively, this suggests that not considering those parts of the forms was not detrimental for the overall perception of those scripts in relation to the CEFR benchmarks or ESOL Exemplars.

Limitations

There are several potential limitations to this research in terms of generalisability, primarily springing from its limited scope. All the individual studies are based on a subset of 4 SfL AOs and a subset of levels. Therefore, even though there is some room for generalisation (the 4 AOs in the sample account for a large proportion of SfL certifications annually, the qualifications at other levels tend to have similar design and measurement approaches within each AO), it is possible that there are aspects that were not observed in the current study.

In relation to the curriculum review study, it should be noted that the views reflected in our findings come from only 2 reviewers, and therefore may be to some extent skewed. We attempted to guard against this by recruiting reviewers with slightly different backgrounds in terms of their closeness to either the SfL tradition or CEFR/EFL tradition, so that they could provide some challenge to each other's views and interpretations having done the reviews independently of each other. Nevertheless, for any higher-stakes purpose such as informing actual reforms to these qualifications or the curriculum itself, it would be advisable to consider a broader set of expert views and include experts in language assessment design more directly.

Similarly, with reference to the procedure for carrying out the CEFR linking for reading comprehension tests, as with other situations where human judgement is elicited from a group of experts, it is possible that various group effects may have been at play and somewhat affected the overall results. There is a possibility that aspects of group dynamics such as conformity (Asch, 1951; Deutsch and Gerard, 1955), polarisation (adoption of a more extreme position) (Moscovici and Zavalloni, 1969), and, to an extent, 'group think' (Baron, 2005) may have created more of a consensus in panel discussions than might have been the case in a different set-up. On the other hand, the purpose of panel discussions was indeed to try and achieve consensus regarding the relevant standards as far as this was appropriate given judges' individual views. In a high stakes linking study, or a standard-setting exercise, it would be preferable to use 2 different panels of experts for each specification and explore the extent to which their independent results agreed and supported each other. In addition, there is a range of other potential standard-linking methods that some might consider more appropriate for higher stakes uses, and which could be used sif item functioning information about the relevant tests were available (for instance, the Angoff or the Bookmark method).

With respect to the CEFR linking study more generally, it is important to note that it was conducted as a research exercise, to consider issues related to comparability and compatibility of content, rather than as an official linking study. Given its purpose, it was considered sufficient to focus the linking on a subset of SfL qualifications, components and levels at this point to serve as an example, but again, this limits the interpretation of the findings to those components and AOs. Having said this, we have made every effort to conduct this linking study according to best practice in the field, and in this sense, the results should be reasonably robust for those components on which the linking was performed.

The CEFR linking of the writing components was additionally somewhat limited by availability of ideally suitable candidate scripts, as well as by the insufficient number of samples from one AO. The latter issue did not really allow clear conclusions about this AO's pass grade standards, and additionally reduces overall generalisability of our findings, even though the limited data from that AO largely accords with the patterns across the other AOs.

With respect to using the rank ordering method to create a common quality scale across the CEFR benchmarks and SfL scripts, while the exercise appears to have worked reasonably well for our research purposes, with largely consistent judgements and high scale separation reliability, it might be advisable to replicate or confirm these results using a more traditional standard-linking methodology, which would not need to rely on using CEFR benchmarks and would just use the CEFR scales. This would avoid challenges with ensuring that the CEFR benchmarks are sufficiently representative and of comparable borderline standards. On the other hand, such an exercise would require that the judges were far more thoroughly standardised as a group in using the CEFR scale categories, and could also suffer from individual judge severity/leniency effects and group dynamics effects if involving panel discussions.

Discussion

The purpose of this research was to explore and evaluate a sample of SfL qualifications in terms of their design and assessment properties, alignment with the core curriculum, inter-AO comparability and validity. This provides us with an understanding of the effectiveness of our current regulatory requirements for these qualifications in ensuring their optimal functioning and validity in relation to their purposes. In this section, we bring together the findings from our 4 studies and suggest some implications. The discussion is organised according to some key themes that emerged from one or more of our studies or from our background research.

A challenging assessment context

The SfL qualifications were developed as part of a suite of basic skills qualifications to foster and improve linguistic integration of different categories of (adult or over-16) migrants who are living in the UK. The SfL qualifications are meant to align with the core curriculum and the NSAL, and it should arguably be a matter of indifference to learners which AO's qualification they are taking in terms of how the results are interpreted and used. This requirement calls for a degree of comparability of standards between these qualifications in the challenging context of language assessment for migration and integration.

The challenge is primarily due to the sheer variety among the target cohort, in terms of literacy levels, language and educational backgrounds, life histories including trauma and other vulnerabilities, aspirations and needs within the UK society, which often calls for flexibility in delivery and assessment approaches, in a non-threatening environment. In the context of language assessment for migration/integration, it has been argued that where qualifications are mainly used or needed to enhance linguistic integration, and do not primarily serve high stakes immigration or other high stakes purposes, the flexibility and quality of course provision can be seen as equally if not more important than ensuring assessment standardisation (Rocca, Hamnes Carlsen & Deygers, ibid.; cf. O'Leary, 2008).

On the other hand, insofar as the results of these qualifications may be needed or used for progression to employment or further education, it would be important that a reasonable degree of assessment quality and validity, as well as a degree of comparability given the shared curriculum, is ensured so that the results can be recognised as trustworthy. Finding the right balance between addressing individual learner needs, contexts and different qualification purposes, while satisfying certain best‑practice assessment principles, especially perhaps that of comparability, is likely to be a challenging task for both the AOs and the regulator. What is also apparent, and discussed in the following sections, is that this challenging context of varying learner needs and stakeholder requirements may also create challenges around defining the purposes of SfL qualifications in the first place.

Issues of recognition

It is clear from the views expressed in our interviews with AO staff, as well as from the literature, that the SfL qualifications are seen by practitioners as having enormous value to learners and society as a whole. Our interview participants felt, however, that the potential of SfL qualifications to enable learner progression was not sufficiently recognised. In our interviews, as well as in the relevant literature, there was also a sense that these qualifications are undervalued in terms of funding. The AOs told us that this affected stakeholder attitudes towards the qualifications, including centres, employers, higher education institutions, and learners themselves.

AOs saw this lack of recognition of SfL qualifications as unjustified (this view is also apparent in the relevant literature), and all AOs emphasised the need to recognise the unique value of these qualifications for addressing ESOL learner needs in terms of developing their broad language skills, including literacy skills, from basic to advanced, while recognising their parity with similar qualifications such as FSE. The SfL qualifications were not seen as dissimilar in their enabling function to ESOL International qualifications, some of which are widely recognised, at competence levels similar to Entry 3, Level 1 or Level 2, by reputable higher education institutions as well as for visa purposes and job recruitment.

Multiple qualification purposes

Using Newton's (2017) perspectives framework for thinking about assessment purposes, based on our background research, it seems fairly clear that inasmuch as these qualifications are seen as valuable for linguistic integration of migrants and for removing barriers to accessing employment or education, some of their purposes can be seen from an engagement perspective, helping to ensure that learning actually occurs. Given their required alignment with the core curriculum, it appears that some of the purposes relate to the expertise perspective, providing evidence of a level of mastery or the acquisition of expertise of the specified content domain, as well as boosting confidence and motivation to build on skills and expand the expertise within the domain. It is also fairly clear from the literature that the SfL qualifications are seen as both socially and personally valuable in relation to these 2 perspectives. What is less clear, based on the issues with recognition of these qualifications for progression to employment or education, is how far they have (or should have) purposes related to information perspective that are valuable to stakeholders beyond learners or qualification providers as the basis for making any decisions.

This could also be seen as a situation where a qualification justifiably has multiple (competing) purposes (cf. Newton, ibid.), but where there is not enough clarity about their relative importance to different stakeholders, and no simple answer to the question of why these qualifications are actually needed. This situation may indirectly impact on overall quality and appropriateness of provision and reduce the extent to which any of the intended purposes are truly fulfilled, potentially reducing the usefulness of these qualifications to users.

From the regulatory perspective, it would be important to seek clarity about the intended purposes of these qualifications and to clearly acknowledge and clarify whether there may be multiple purposes and their relative importance to stakeholders. This is necessary to enable coherent considerations regarding the most appropriate assessment models, as well as approaches to regulation, and any trade-offs that might need to be acknowledged or accepted, in order to ensure that the SfL qualifications are as fit for their purpose(s) as possible and regulated in the most appropriate way.

Assessment models

From our internal SfL assessment review we could see that a wide range of assessment models and approaches are used across the 4 AOs in our sample. Some of the differences in approaches appear to stem from a desire to offer flexibility of choice to learners and centres, so that different needs can be satisfied. The AOs in our interviews thought that the flexibility of the SfL qualification structure, its modularity and the variety of assessment models that can be used, were beneficial and justified by the needs and the wide range of contexts of learners and centres.

All our interviewees referred to the flexibilities offered in the context of speaking assessments in particular. Perhaps unsurprisingly, however, AO staff in our interviews suggested that the flexibilities offered to centres in terms of written task amendments are rarely taken up in the context of reading or writing assessments as they are resource intensive, although the AOs themselves offered some choice of context across different assessment versions in some models. In addition, the choices that centres make, for instance, in relation to internal vs. external assessment models, do not seem to always be tailored to the learners, and appear to be driven by centre needs (for example, availability of assessors), highlighting another area where there may be competing needs that may not always have solutions that satisfy them all.

Our assessment review also suggested that some of the individual AO approaches may not be entirely appropriate for the construct of assessment, while some may be remnants of historical approaches and regulatory frameworks such as the QCF, or may be poorly executed in practice. The interviews with the AOs suggested that some of the assessment design choices were indeed historical and not always reflective of best practice in assessment (for instance, choice and implementation of measurement models, weighting of certain curriculum aspects, atomistic assessment criteria), and AOs themselves were not always able to provide a clear rationale for them beyond citing original requirements that had not been updated. Therefore, it would be important to ensure that individual AO assessment models are of sufficient quality and sufficiently valid and justified for the purposes and construct of these qualifications.

Nature of curriculum alignment

The curriculum alignment review suggests that the SfL qualifications may not be sufficiently well aligned with the curriculum. This was primarily in terms of demand, aspects of coverage, and interpretation of the curriculum messages.

Based on the review, the demand of assessments and pass grade standards at Entry 1 were seen as broadly appropriate, if occasionally too high in relation to the curriculum standards. However, at Entry 3 and Level 1, the demand was deemed to be too low in most cases.

The overall quality and appropriateness of coverage and relative demand of the assessments seem to decline from Entry 1 to Level 1 for most of the AOs in our sample, with insufficient differentiation and stretch in the progress through the levels. This is a significant finding and highlights threats around appropriate candidate progression.

Furthermore, there was evidence of construct-irrelevant assessment approaches, and narrowing of the curriculum. For instance, it was noted that inordinate weight was sometimes given to content statements that involve enabling subskills (knowledge of alphabetical ordering, knowledge of names for text and grammar features), which are singled out in the curriculum for instruction rather than assessment purposes. In some cases, this was at the expense of assessing the global skills more thoroughly at the appropriate level (for instance, global and detailed understanding of text meaning). As another example, the focus on more academic and professional reading and writing, which is largely missing from the assessments, is present in the curriculum. It is not clear why it does not feature sufficiently in the assessments, given that some of the purposes of SfL qualifications, especially at Level 1 and above, are likely to involve progression to further study and employment.

In most cases it was not clear why such apparent misalignment may have occurred, as the curriculum statements and examples were deemed by the reviewers as sufficiently clear and indicative of the appropriate level, and the curriculum itself largely fit for purpose.

Comparability

A comparison of pass grade standards in Entry 3 reading comprehension assessments between the 4 AOs in our sample, carried out in relation to the CEFR, demonstrated that the pass grade standards are broadly comparable and at around threshold/low B1 level for 3 of the AOs. However, for one AO, the pass grade standard was mapped to A2+, and is thus lower than the rest. This is despite the overall profile and demand of the tests from the 4 AOs being reasonably aligned.

Similarly, some comparability issues were noted in our CEFR mapping study of pass grade standards in writing at Entry 1 level. However, the CEFR mapping also indicates that the SfL writing assessments from the 4 AOs are broadly comparable and all at the same CEFR level, despite some discrepancies at the pass grade borderline within A1 level. Therefore, these comparability issues are of somewhat less concern than those identified in reading comprehension.

In addition to these comparability issues that were directly identified between some AOs' grading standards, potential threats to comparability were identified that might stem from either inconsistency in assessment models (perhaps in particular the disparate measurement models and their implementations) or misalignment with the curriculum in terms of aspects and nature of coverage of the content domain. Given the issues around clarity of purposes of SfL qualifications, it would be important to clarify how much comparability of approaches and standards should be seen as a desirable given the relative nature and weight of qualification purposes.

CEFR alignment

The indicative CEFR content mapping produced in our studies suggests meaningful alignment of the SfL qualifications with the CEFR in many respects, and thus also with ESOL International qualifications.

The CEFR content mapping, which gives an indication of the extent of alignment between the content and construct of SfL qualifications and the CEFR and gives an indication of the most appropriate CEFR level that can be evidenced, placed Entry 1 components at A1/A1+ level, except for speaking, which was mapped to A1+/A2. The relatively high CEFR level for speaking may be unexpected for an Entry 1 qualification. Nevertheless, this was deemed appropriate for speaking in our curriculum mapping review, based on the standards implied in the curriculum. Furthermore, our CEFR standard linking of writing components indicated that Entry 1 writing pass grade standard is also relatively high, at about A1+ level.

The CEFR content mapping placed Entry 3 components at B1/B1+ level. However, the results of our standard linking study for reading comprehension suggests that actual pass grade standards at Entry 3 are lower, in some cases at A2+, and in all cases closer to B1 threshold level than B1+. Similarly, the standard linking for the writing components indicated that the pass grade performance standard at Entry 3 is at A2+/low B1 at most, while Level 1 was mapped to B1+/low B2.

Our CEFR linking exercise for writing components also indicates that ESOL Exemplars, which exemplify secure performance within the RQF levels of the curriculum, map onto the mid to high A1, A2, B1 and B2 CEFR levels respectively. This suggests that the borderline performance manifestation should perhaps be somewhat lower within each CEFR level, but still map onto each level at about its threshold at least, rather than dropping below it, as seems to be the case with some samples of the SfL writing qualifications in our study at Entry 3 and Level 1 and with some reading comprehension assessments at Entry 3.

Perhaps the most significant finding from the CEFR standard linking study for writing is the fact that there does not appear to be enough differentiation between higher RQF levels of the SfL qualifications at the pass grade threshold, particularly between Entry 2 and Entry 3, which can both be mapped onto A2+ CEFR level. This is also the case to some extent between Entry 3 and Level 1 writing qualifications.

The findings from the CEFR standard linking largely confirm what was suggested in the curriculum review, namely, that Entry 3 and Level 1 SfL qualifications may not be sufficiently demanding given the demand implied in the curriculum and relative to the demand of the lower levels. This apparent lack of stretch at Entry 3 in particular may be contributing to the perceived "large jump" in standards at Level 1, which was noted by our curriculum reviewers, as well as some stakeholders.

It is worth pointing out that even though some Entry 1 components may be compatible with A1+ level according to the curriculum, the question remains whether this level is suitable for an initial Entry level language qualification. Given that Entry 1 pass grade standard appears to be significantly higher than threshold A1 CEFR level at least in writing, this may support stakeholder calls for separate qualifications at somewhat lower level ('pre-entry'), as the current GLH may not enable some learners to achieve A1+ level when starting from the point of no English at Entry 1.

The CEFR content mapping highlighted the discrepancies with the CEFR in relation to assessment of literacy in SfL qualifications (such as, assessment of Latin alphabet knowledge, emphasis on orthographic accuracy at Entry 1, emphasis on awareness of the writing process at Entry 3). In the context of SfL qualifications, such requirements in the domain of literacy are related to their link to the NSAL, which are primarily targeted at learners for whom English is the first language.

The results of our Study 3, as well as the CEFR content mapping, suggest that perhaps such requirements may not be entirely appropriate at all language ability levels in the context of learning English as a second language, and particularly in the migration context. There are suggestions in the literature (ALTE, 2016) that in the migrant context, the focus on linguistic accuracy may not be as relevant as the ability to get the main idea across, at least at lower ability levels (up to B1). The CEFR, and the ESOL core curriculum to an extent, appear to recognise this more clearly than the NSAL. This potentially raises questions about whether mapping English as a second language qualifications, such as SfL, to standards intended for English as the first language qualifications such as NSAL is entirely appropriate.

The interpretation of the CEFR pass grade standard linking for reading comprehension tests is also limited to an extent by certain issues of item design, test design and potentially low reliability of some tests and use of some assessment approaches that do not optimally elicit evidence of reading comprehension competence. Similarly, there were issues in some writing assessments with approaches that are not sufficiently communicative or interactive. Such issues can also limit the interpretation of the CEFR linking, where assessment operationalisation rather than the scope of the specification or intended demand, constrains the nature of the skills elicited by the assessments and thus reduces overall validity of both the CEFR link and the assessment itself. Furthermore, the curriculum review has also indicated that the approaches that are not well aligned with the CEFR are also not sufficiently well aligned with the core curriculum, which is, like the CEFR, fundamentally communicative and functional in approach.

Potential improvements

In light of these findings, Ofqual (and other relevant stakeholders where appropriate, for instance, the DfE) should consider how the qualifications could be improved to address some of the issues identified. Some areas for improvement are listed below:

The range, relative weight, and interactions of SfL qualification purposes should be clarified. Given the complexity of SfL learner needs and the importance of engagement-related purposes of these qualifications, revisiting the trade-off between these and any information-related purposes would seem particularly important as a starting point. This should help to inform the choice of the most appropriate assessment models and any trade-offs between the extent of standardisation and flexibility in assessment design and delivery.

There are quality and validity issues in some individual approaches and assessment instruments, which would need to be addressed. In addition, appropriateness of approaches that may be historically motivated rather than meaningfully linked to qualification purposes and constructs should be reconsidered. Assessment approaches should be sufficiently justified in terms of validity and best assessment practice.

The way the curriculum is covered and interpreted in some assessments may need to be improved in order to reflect its intention and implied demand.

Insufficient comparability between Entry 3 reading comprehension pass grade standards should be addressed if comparability were deemed important for certain purposes of SfL qualifications. Wider checks on comparability (for instance, between other components, levels and AOs) may also be warranted depending on the extent of comparability required.

Differentiation and progression between SfL qualification levels should be improved to ensure optimal progression for learners.

Mapping SfL qualifications to the CEFR might clarify to stakeholders how the skills at each RQF level relate to the skills of ESOL International learners, who might be competing for the same jobs or education courses upon entry to this country and improve recognition of SfL qualifications for similar purposes.

References
AERA/APA/NCME (2014). Standards for Educational and Psychological Testing. ALTE (2016). Language tests for access, integration and citizenship: An outline for policy makers. Cambridge: Association of Language Testers in Europe. ALTE (2020). ALTE Principles of Good Practice. Andrich, D. (1982). An index of person separation in latent trait theory, the traditional KR-20 index, and the Guttman scale response pattern. Education Research & Perspectives, 9:1, 95-104. Asch, S.E. (1956). Studies of independence and conformity: a minority of one against a unanimous majority. Psychological Monographs, 70, 9, (whole no. 416), 1- 70. Baron, R. S. (2005). So right it's wrong: Groupthink and the ubiquitous nature of polarized group decision making. In M. P. Zanna (Ed.), Advances in experimental social psychology, Vol. 37, 219-253. San Diego, CA, US: Elsevier Academic Press. Bell, R., Plumb, N. & Marangozov, R. (2017). Integration not Demonisation. The final report of the All-Party Parliamentary Group on Social Integration's inquiry into the integration of immigrants. Benton, T., Leech, T. & Hughes, S. (2020a). Does comparative judgement of scripts provide an effective means of maintaining standards in mathematics? Cambridge Assessment Research Report. Cambridge, UK: Cambridge Assessment. Benton, T., Cunningham, E., Hughes, S. & Leech, T. (2020b). Comparing the simplified pairs method of standard maintaining to statistical equating. Cambridge Assessment Research Report. Cambridge, UK: Cambridge Assessment. Black, B., & Bramley, T. (2008). Investigating a judgemental rank‐ordering method for maintaining standards in UK examinations, Research Papers in Education, 23:3, 357-373. Black, B., He, Q. & Holmes, S. (2017). Vocational and Technical Qualifications: Assessment Functioning of external assessments. An overview of the functioning of assessments in 27 qualifications and 49 units, (Ofqual/17/6319). Coventry: Ofqual. Black, B., He, Q. & Morin, C. (2019). Vocational and Technical Qualifications: assessment functioning of examinations in performance table qualifications. An overview of 20 Level 3 assessments, (Ofqual/19/6501). Coventry: Ofqual.
Bond, T.G., & Fox, C.M. (2007). Applying the Rasch model: Fundamental measurement in the human sciences. 2nd ed. Mahwah, NJ: Lawrence Erlbaum.
Bradley R.A. & Terry M.E. (1952). Rank Analysis of Incomplete Block Designs I: The Method of Paired Comparisons. Biometrika, 39, 324-45. Bramley, T. (2007). Paired comparison methods. In P. Newton, J.-A. Baird, H. Goldstein, H. Patrick, & P. Tymms (Eds.), Techniques for monitoring the comparability of examination standards (pp. 246-300). London, UK: Qualifications and Curriculum Authority. Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3:2, 77-101. Brunfaut, T., & Harding, L. (2013). Linking the GEPT Listening Test to the Common European Framework of Reference. (LTTC-GEPT Research Report RG-05.)
Cambridge Assessment English (2016a). Cambridge English Key. Handbook for teachers for exams from 2016. Cambridge: Cambridge Assessment English.

Cambridge Assessment English (2016b). Cambridge English Preliminary. Handbook for teachers for exams from 2016. Cambridge: Cambridge Assessment English.

Cambridge Assessment English (2019a). Cambridge English Qualifications A2 Key. Handbook for teachers for exams from 2020. Cambridge: Cambridge Assessment English.

Cambridge Assessment English (2019b). Cambridge English Qualifications B1 Preliminary. Handbook for teachers for exams from 2020. Cambridge: Cambridge Assessment English.

Cambridge Assessment English (2019c). Cambridge English Qualifications B2 First. Handbook for teachers. Cambridge: Cambridge Assessment English.
Carter, B., & Bathmaker, A. (2017). Prioritising progression over proficiency: limitations of teacher-based assessment within technician-level vocational education, Journal of Further and Higher Education, 41:4, 460-474.
Casey, L. (2016). The Casey review. A review into opportunity and integration..

Christensen, K. B., Kreiner, S., & Mesbah, M. (2013). Rasch Models in Health. London, UK: Wiley & Sons.
Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6:4, 284-290. Council of Europe (2001). Common European Framework of Reference for Languages: Learning, teaching, assessment, Cambridge: Cambridge University Press. Council of Europe (2009). Relating Language Examinations to the Common European Framework of Reference for Languages: Learning, Teaching, Assessment (CEFR). A Manual. Council of Europe (2020). Common European Framework of Reference for Languages: Learning, teaching, assessment - Companion volume. Strasbourg: Council of Europe Publishing. Curcin, M. & Black, B. (2019). Investigating standards in GCSE French, German and Spanish through the lens of the CEFR, (Ofqual/19/6559/1). Coventry, UK: Ofqual. Curcin, M., Howard, E., Sully, K. & Black, B. (2019). Improving awarding: 2018/2019 pilots, (Ofqual/19/6575). Coventry, UK: Ofqual. De Jong, J. H. A. L. (2009). Unwarranted claims about CEF alignment of some international English language tests. Paper presented at EALTA, Turku, Finland. Department for Education and Skills (2000). Breaking the Language Barriers. London: DfES. Department for Education and Skills (2001). Adult ESOL Core Curriculum. London: DfES. Department for Education and Skills (2003). Pathways to Proficiency. The Alignment of Language Proficiency Scales for assessing competence in English language. London: DfES.
Department for Education and Skills (2004). ESOL Exemplars. London: DfES.
Deutsch, M., & Gerard, H.B. (1955). A study of normative and informational influence upon individual judgement. Journal of Abnormal and Social Psychology, 51, 629- 636.
Ecclestone, K. (2002). Leaner Autonomy in Post-16 Education: The Politics and Practice of Formative Assessment. London: Routledge Falmer.
Engelhard, G., Jr., Kobrin, J. L., & Wind, S. A. (2014). Exploring differential subgroup functioning on SAT writing items: What happens when English is not a test taker's best language? International Journal of Testing, 14, 339-359. Foster, D. & Bolton, P. (2018). Adult ESOL in England. Commons Library briefing paper 7905. Garland, P. (1998). Assessment in GNVQs: learning the hard way. Research in Post‑Compulsory Education, 3:3, 329-344. Hallgren, K. A. (2012). Computing Inter-Rater Reliability for Observational Data: An Overview and Tutorial. Tutorials in Quantitative Methods for Psychology, 8:1, 23-34. Higton, J., Sandhu, J., Stutz, A., Patel, R., Choudhoury, A. & Richards, S. (2019). English for speakers of other languages: Access and progression. HM Government (2019). Integrated Communities Strategy Green Paper. Summary of consultation responses and Government response. Johnson, M. (2008). Exploring assessor consistency in a Health and Social Care qualification using a sociocultural perspective, Journal of Vocational Education and Training, 60:2, 173-187. Jones, N. (2009). A comparative approach to constructing a multilingual proficiency framework: constraining the role of standard setting. Research Notes, 37, 6-9. Kaftandjieva, F. (2010). Methods for Setting Cut Scores in Criterion referenced Achievement Tests: A comparative analysis of six recent methods with an application to tests of reading in EFL. Arnhem, The Netherlands: Cito, ELTA.
Karabatsos, G. (2000). A critique of Rasch residual fit statistics. Journal of Applied Measurement, 1:2, 152-176.
Kings, P. & Casey, H. (2014). ESOL Qualifications and funding in 2014: Issues for consideration. NRDC report for The Association of Colleges. Learning and Work Institute (2020). ESOL Learners' progression to Functional Skills. Little, D. (2012). The linguistic integration of adult migrants and the Common European Framework of Reference for Languages (CEFR). Council of Europe: Language Policy Unit.
Linacre, J. M. (2011). A user's guide to FACETS Rasch-model computer programs. Program Manual 3.68.1. Chicago, IL: Winsteps.com.
Lockyer, C. & Cadwallader, S. (2020). Internal assessment in existing national technical and vocational qualifications, (Ofqual/20/6594). Coventry: Ofqual. Moscovici, S., & Zavalloni, M. (1969). The group as the polarizer of attitudes. Journal of Personality and Social Psychology, 12, 125-135. Newton, P. (2017). There Is More to Educational Measurement Than Measuring: The Importance of Embracing Purpose Pluralism. Educational Measurement: Issues and Practice, 36:2, 5-15. Newton, P. (2018). Grading Vocational & Technical Qualifications, (Ofqual/18/6441/3). Coventry: Ofqual. Newton, P. & Lockyer, C. (2022). How 'CASLO' Qualifications Work, (Ofqual/22/6895). Coventry: Ofqual. North, B., & Schneider, G. (1998). Scaling descriptors for language proficiency scales. Language Testing, 15:2, 217-263. Ofqual (2008). Regulatory arrangements for the Qualifications and Credit Framework. Coventry: Ofqual. Ofqual (2011). Criteria for English for Speakers of other Languages (ESOL) Qualifications. Coventry: Ofqual. Ofqual (2019). Corporate plan 2019-2022. Coventry: Ofqual. O'Leary, D. (2008). A Common Language: making English work for London. London: Demos. Plackett, R. L. (1975). The analysis of permutations. Journal of the Royal Statistical Society: Series C (Applied Statistics), 24:2, 193-202. QCA (2000). National standards for adult literacy. QCA (2009). Evaluation of ESOL Skills for Life Qualifications. Robitzsch, A. (2019). Sirt: Supplementary item response theory models. R package version 3.7-40. Rocca, L., Hamnes Carlsen, C. & Deygers, B. (2020). Linguistic integration of adult migrants: Requirements and learning opportunities. Report on the 2018 Council of Europe and ALTE survey on language and knowledge of society policies for migrants. Strasbourg: Council of Europe Publishing. Roden, J. & Osmaston, M. (2021). ESOL Learners and Functional Skills English. Report commissioned by Education & Training Foundation. Rolfe, H. & Stevenson, A. (2021). Migration and English Language Learning after Brexit. Leicester: National Learning and Work Institute.
Smith, R. M., Schumacker, R. E., & Bush, J. J. (1998). Examining replication effects in Raschfit statistics. In M. Wilson & G. Engelhard, Jr. (Eds.), Objective Measurement: Theory into practice vol. 5, (pp. 303-317). Stanford, CT: Ablex Publishing Corp.
Thurstone, L.L. (1927). A law of comparative judgment. Psychological Review 3, 273-86. Torrance, H., H. Colley, D. Garratt, J. Jarvis, H. Piper, K. Ecclestone, and D. James. (2005). The Impact of Different Modes of Assessment on Achievement and Progress in the Learning and Skills Sector (London, LSDA for the LSRC). London: The Learning and Skills Research Centre. Turner, H. L., van Etten, J., Firth, D. & Kosmidis, I. (2020). Modelling Rankings in R: The PlackettLuce Package. Computational Statistics, 35, 1027-1057. Verhavert, S., Bouwer, R., Donche, V. & De Maeyer, S. (2019). A meta-analysis on the reliability of comparative judgement. Assessment in Education: Principles, Policy & Practice, 26:5, 541-562. Wahlberg, M., & Gleeson, D. (2003). 'Doing the business': paradox and irony in vocational education - GNVQ business studies as a case in point. Journal of Vocational Education and Training, 55:4, 423-446. West Midlands Combined Authority (WMCA) (2019). Unlocking Potential - Making Sense of ESOL in the Region. Wright, B. D., & Linacre, J. M. (1994). Reasonable mean-square fit values. Rasch Measurement Transactions, 8:3, 370-371.
Appendix 1: The CEFR global scale

C2 - Can understand with ease virtually everything heard or read. Can summarise information from different spoken and written sources, reconstructing arguments and accounts in a coherent presentation. Can express him/herself spontaneously, very fluently and precisely, differentiating finer shades of meaning even in more complex situations.

C1 - Can understand a wide range of demanding, longer texts, and recognise implicit meaning. Can express him/herself fluently and spontaneously without much obvious searching for expressions. Can use language flexibly and effectively for social, academic and professional purposes. Can produce clear, well-structured, detailed text on complex subjects, showing controlled use of organisational patterns, connectors and cohesive devices.

B2 - Can understand the main ideas of complex text on both concrete and abstract topics, including technical discussions in his/her field of specialisation. Can interact with a degree of fluency and spontaneity that makes regular interaction with native speakers quite possible without strain for either party. Can produce clear, detailed text on a wide range of subjects and explain a viewpoint on a topical issue giving the advantages and disadvantages of various options.

B1 - Can understand the main points of clear standard input on familiar matters regularly encountered in work, school, leisure, etc. Can deal with most situations likely to arise whilst travelling in an area where the language is spoken. Can produce simple connected text on topics which are familiar or of personal interest. Can describe experiences and events, dreams, hopes and ambitions and briefly give reasons and explanations for opinions and plans.

A2 - Can understand sentences and frequently used expressions related to areas of most immediate relevance (e.g., very basic personal and family information, shopping, local geography, employment). Can communicate in simple and routine tasks requiring a simple and direct exchange of information on familiar and routine matters. Can describe in simple terms aspects of his/her background, immediate environment and matters in areas of immediate need.

A1 - Can understand and use familiar everyday expressions and very basic phrases aimed at the satisfaction of needs of a concrete type. Can introduce him/herself and others and can ask and answer questions about personal details such as where he/she lives, people he/she knows and things he/she has. Can interact in a simple way provided the other person talks slowly and clearly and is prepared to help.

Appendix 2: Curriculum alignment review questions

AO-specific questions

Does the demand of the assessments appear broadly appropriate according to the standards in the Adult ESOL Core Curriculum and the National Standards for Adult Literacy at this level?

Is the coverage of the content in the assessments, alongside the minimum pass grade criteria, sufficient to evidence mastery of the curriculum at this level?

Do the assessments assess an appropriate sample of the vocabulary (including spelling), types of words, grammatical forms and communicative functions indicated in the curriculum for this level?

Are there any prominent standards and/or curriculum statements that are under-assessed ( for example, that the assessments do not elicit enough evidence of from candidates)?

Is there too much emphasis on some aspects of the curriculum in the assessments?

Are the approaches used to assess the curriculum statements generally valid and appropriate for the nature of the content they are intended to assess?

Do the approaches used to assess the curriculum statements reflect the intention of the curriculum?

Are topics and contexts in the assessments appropriate and as intended by the curriculum at this level?

Are stimulus materials (for instance, reading texts, writing or speaking prompts) appropriate and as intended by the curriculum at this level?

Is the reading load and level of the language in the rubrics and questions appropriate at this level?

Do the questions/tasks test subject content in a way that is construct relevant?

In relation to the writing and speaking/listening components, are the assessment criteria/level descriptors clear and designed so they can be applied easily and consistently?

Are there any questions, tasks or stimulus materials that exhibit bias or may be inaccessible to some groups of learners such as learners with protected characteristics, or learners of particular ethnicity?

General questions

Is the way the curriculum statements are assessed across the 4 AOs broadly similar and, where different approaches are used, are these equally valid? If not, what are the most notable discrepancies between AOs and which curriculum statements do these relate to?

Is the standard implied by the coverage and the demand of assessments, as well as the pass grade criteria, broadly comparable across the 4 AOs?

Are there aspects of the curriculum at this level that might benefit from updating to make it more relevant to life in the UK in 2021 and beyond, or that you think should be changed or removed?

Is there anything that is missing from the current curriculum at this level that you think it would be helpful to add?

Are there aspects of the curriculum at this level that would benefit from more explicit alignment to the CEFR?

Appendix 3: Detailed CEFR content mapping tables

Entry 1

AO1 final estimation summary

Reception overall

Communicative language competence in reception

Production overall

Communicative language competence in production

Interaction overall

Communicative language competence in interaction

AO2 final estimation summary

Reception overall

Communicative language competence in reception

Production overall

Communicative language competence in production

Interaction overall

Communicative language competence in interaction

AO3 final estimation summary

Reception overall

Communicative language competence in reception

Production overall

Communicative language competence in production

Interaction overall

Communicative language competence in interaction

AO4 final estimation summary

Reception overall

Communicative language competence in reception

Production overall

Communicative language competence in production

Interaction overall

Communicative language competence in interaction

Entry 3

AO1 final estimation summary

Reception overall

Communicative language competence in reception

Production overall

Communicative language competence in production

Interaction overall

Communicative language competence in interaction

B1/B1+

AO2 final estimation summary

Reception overall

Communicative language competence in reception

Production overall

Communicative language competence in production

Interaction overall

Communicative language competence in interaction

AO3 final estimation summary

Reception overall

Communicative language competence in reception

Production overall

Communicative language competence in production

Interaction overall

Communicative language competence in interaction

AO4 final estimation summary

Reception overall

Communicative language competence in reception

Production overall

Communicative language competence in production

Interaction overall

Communicative language competence in interaction

Appendix 4: PL-based analysis

Comparability between CEFR benchmarks

Comparability between SfL AOs

Appendix 5: Judge and script status

Judge statistics

Script statistics

Appendix 6: Descriptive statistics for script quality measures by script group and level

E1/A1

E2/A2

E3/B1

L1/B2

# data[].attributes.attachment_link.title
Understanding ESOL Skills for Life qualifications: Assessment properties, comparability, core curriculum and CEFR alignment

# data[].attributes.attachment_link.description
We would like to thank the many people without whose input this study would not have been possible: all our participants, who devoted a lot of their time and enthusiasm to work on this study and share their expertise and opinions colleagues at Ofqual who have helped in different ways

# data[].attributes.headline
New home for Nexus/EG interactive learner resources (Oct 2023)

# data[].attributes.subject
New home for Nexus/EG interactive learner resources (Oct 2023)

# data[].attributes.attachment_link.description
ESOL Learners ETF online learning classes

# data[].attributes.headline
Towards an ESOL strategy for England (2016)

# data[].attributes.subject
Towards an ESOL strategy for England (2016)

# data[].attributes.headline
ESOL for Skills - Respublica Report Dec (2022)

# data[].attributes.subject
ESOL for Skills - Respublica Report Dec (2022)

# data[].attributes.headline
ESOL - Summary Policy Guidelines and Supports for V.E.C. Adult Literacy Schemes - Ireland

# data[].attributes.subject
ESOL - Summary Policy Guidelines and Supports for V.E.C. Adult Literacy Schemes - Ireland

# data[].attributes.headline
ESOL Policy Guidelines and Supports for V.E.C. Adult Literacy Schemes - Ireland

# data[].attributes.subject
ESOL Policy Guidelines and Supports for V.E.C. Adult Literacy Schemes - Ireland

# data[].attributes.body
www.nala.ie

# data[].attributes.body
N.B. this is aimed at school, but resources can be adapted or used as it.

# data[].attributes.attachment_link.description
Everything you need to provide quality Citizenship education. Expert advice, CPD, training, conferences, resources, journal.

# data[].attributes.body
Easy read version

# data[].attributes.body
N.B. NATECLA has been campaigning for several years for an ESOL Strategy for England. Contact this Padlet's author if you are interested in joining NATECLA's policy group.

# data[].attributes.attachment_link.readable_text
When we meet someone and ask "Where are you from?" or "What do you enjoy doing?", what we are really asking is "What's your story?"
When we start to share our story with others, we can begin to celebrate differences and recognise how much we all have in common.

What's Your Story? Series comprises 8 bookswritten and photographed by Jennie Cole with each book focusing on a personal story. This series is an ideal starting point to encourage conversation and invite others to share their story.

The 8 titles in the series are:

Developed for adults who are learning to read, or who are learning English as a second language, these delightful short stories, illustrated with colour photographs and presented in a comic book style, are a great way to build vocabulary around daily life.

Individually priced at £5.95, you can buy the full set of 8 books for only £45.00

Additional Information

Author : Jennie Cole

Reading Level : 2 - Easy Beginner Reader

Reading Level : 3 - Standard Beginner Reader

Series : What's Your Story?

Topic : Biographical

Topic : ESOL

Topic : Life

Topic : Literacy

Publisher : Gatehouse Books

ISBN : 9781842311974

# data[].attributes.attachment_link.description
What's Your Story? is a series of 8 books with each book focusing on a personal story. This series is an ideal starting point to encourage conversation and invite others to share their story. Developed for adults who are learning to read, or who are learning English as a second language.

# data[].attributes.body
With thanks to Elena K!

# data[].attributes.body
With thanks to Jonny Ingham 😊

# data[].attributes.attachment_link.readable_text
We have new courses starting this week at school so I thought I'd share my favourite getting to know you activity to use with a new class. The original idea of using an acrostics poem came from a session by Jeremy Harmer at the IH DoS Conference a couple of years ago and this inspired my colleague Amy Braid to adapt the idea for use with a new class.

The basic procedure

1. Show the students an example of an acrostics poem with your name. E.g.

2. Students write their name vertically down the left hand side of a page.

3. Students think of sentences beginning with the letters of their name to create their poem.

4. T monitors, helping students with ideas and language, encouraging them to play with sentence structure (cleft sentences, fronting, inversion etc. e.g Yoga is what I like doing, never have I written a poem).

5. When students are happy with their poems, they copy them onto a new piece of paper with coloured pens and pencils.

6. Put the poems on the walls around the classroom (like a gallery).

7. Students walk around the classroom individually with a notebook and a pen, they write one question for each person based on the information that they read, they will ask this question later (e.g. How long have you been doing yoga? Do you think you will ever write another poem?). T could play some music in the background while they are doing this.

8. Students then mingle and ask each other their questions. T monitors, taking notes of language to focus on in feedback (for tips on this, see my previous post on delayed error correction).

9. Students sit down and tell their partner everything they can remember about the other students.

10. Teacher conducts feedback, first on content, then language.

11. Leave poems on wall to help you remember the students' names in future lessons!

# data[].attributes.attachment_link.title
Getting to Know You with an Acrostics Poem

# data[].attributes.attachment_link.description
We have new courses starting this week at school so I thought I'd share my favourite getting to know you activity to use with a new class. The original idea of using an acrostics poem came from a session by Jeremy Harmer at the IH DoS Conference a couple of years ago and this inspired...

# data[].attributes.body
Do card matching activities - match words with pictures of vocab you taught last class or vocab that you are going to cover in your lesson to come (and that way also acts as formative assessment for you).

Do memory/vocabulary games such as: "In my kitchen I have a cooker". The next person has to add to it and say, "In my kitchen I have a cooker and a fridge" and so on.

Another fun memory game is to have random objects on a tray, learners name the object and memorise what is there. Then cover the tray with a tea towel, get the learners to look away and remove an item. They have to guess what has been removed. You can do this with vocabulary on separate small cards also. Why I like using random objects is so I have use extensions (talk about the object, colour, use etc...)

Spot the difference pictures: find ones that can focus on specific language such as prepositions for example.

Get learners moving by making them stand in order of something, Could be their age or the distance (farthest or closest) to the class or alphabetically according to their names (possibilities are endless!)

Incorporate reading by giving them a card with a word on and have the same number of words as students in the class and get them to stand in the correct word order to make a sentence. Or two students don't have a card and have to direct or check the meaning if the sentence after.

A game of 'I spy' or 'I can find' in the classroom.

Have some photos of famous people, stick them up around the classroom and start to describe their appearance etc. The first one to run to the photo gets a point. Great if you have recently studied appearance, adjectives etc.

A game of bingo (with words you have prepared from past lessons).

Find someone who - one of my favourites! Write a list of topics on a page, print and they must go round the class asking questions till they find someone who fits a question/statement (i.e. find someone who hates coffee). The person who completes their worksheet first wins. Again. you can link the topics to things you have covered in class.

Another warmer that includes movements is getting the learners to choose which side of the room they stand on (their preferences/likes dislikes) according to the words you shout out. (Sun or snow, dogs or cats, coffee or tea etc.). (but beware of cultural issues (i.e. avoid alcohol etc)

You might want to google ice breakers for the English class.

# data[].attributes.body
Here you can find some great ideas for icebreakers to use in your classes.

As much as possible, I like to use ice breakers that are connected to our learning aims, a revision of topics or to develop specific (personalised) skills areas that need improving.  Below are some ideas.

# data[].attributes.body
The Covid resources that were on this Padlet have now been 'retired' but you can find them on a new Padlet (click above image).

# data[].attributes.body
British Council Article

# data[].attributes.attachment_link.readable_text
How do you describe colours to the blind?

Are colours important to them? Are they fashion conscious?

Understanding degrees of blindness

Understanding the background

Setting up a readers service
Teaching tips
I have used talking books and taped dialogues for reading comprehension lessons.

I use real objects in my lessons. I brought in kites when the reading passage was about kites.

I took my students to the kitchen and we had a lesson on how to prepare sandwiches and how to make tea.

I took my senior students to a lower court to record the proceeding of court cases. I was able to do all these things because there were usually only eight to nine students in a class.

Reactions of other students

Is he or she totally blind?

Does he or she have some sight left that he or she may be able to read big print?

Eighty percent of learning is through sight.

How do you teach poetry to them?

How do they conceive concepts?

Can they learn science?

Do you know how to read Braille?

Of all the subjects teaching English to the blind is the easiest. Having said so there are one or two things teachers who are not trained to teach the blind may need to know before they can be an effective teacher to a blind student.

Whatever amount of residual vision that is left of a blind person should be utilised to the maximum. So students with partial vision should be encouraged to read big print books.

Textbooks could never be brailled in time for my blind students. I always had to cope with one chapter at a time. But from my experience I can say that setting up a readers service for the blind is never too difficult. Just spread the word that there is a need for a pool of volunteers to read books into tapes or to blind students and there will be many volunteers.

Blind students may be visually impaired but their other senses are intact, so teachers of the blind should utilise the other senses. Learning a language is very much tied up with culture, exposure and experiences. Blind students may not be able to acquire exposure and experiences the same way as sighted students. So teachers of the blind may have to do a little more than other teachers. Bring experiences and exposure to the blind students.

# data[].attributes.attachment_link.description
How do you describe colours to the blind?Are colours important to them? Are they fashion conscious?How do you teach poetry to them?How do they conceive concepts?Can they learn science?Do you know how to read Braille?Of all the subjects teaching English to the blind is the easiest.

# data[].attributes.body
Teaching English to Visually Impaired Learners  - English Club

# data[].attributes.attachment_link.title
Teaching English to Visually Impaired Learners | Learning Difficulties | EnglishClub

# data[].attributes.attachment_link.description
An introduction to visual impairment from low vision to blindness, how it affects language learners, with useful strategies for teachers of English and other languages. Author: Tara Benwell

# data[].attributes.body
Courtesy of Eliza Winnert: elizawinnert@gmail.com

# data[].attributes.body
By Learning Unlimited

# data[].attributes.body
https://www.eslprintables.com/vocabulary_worksheets/sports/golf/https://www.englishclub.com/vocabulary/sports-golf.php

https://www.youtube.com/watch?v=1tbxf8V2l3Y

https://www.learn-english-today.com/vocabulary/golf-vocabulary1.html

https://www.talkenglish.com/lessondetails.aspx?ALID=695

# data[].attributes.attachment_link.title
Golf - pictionary - ESL worksheet by oppilif

# data[].attributes.body
https://en.islcollective.com/english-esl-worksheets/search/martial+arts?page=1

https://en.islcollective.com/download/english-esl-worksheets/speaking-practice/warmer-filler-cooler/sports/guess-the-martial-art/95329

# data[].attributes.body
https://www.engworksheets.com/vocabulary-new/english/basketball-1.html?page=3

https://www.englishclub.com/vocabulary/sports-basketball.php

https://en.islcollective.com/english-esl-worksheets/search/basketball

https://myvocabulary.com/word-list/basketball-vocabulary/

# data[].attributes.attachment_link.title
Basketball Printable English ESL Vocabulary Worksheets - EngWorksheets

# data[].attributes.attachment_link.description
Want to learn Basketball words? Here are 40 different best printable English worksheets and exercises for Basketball vocabulary. Enter now and start learning easily

# data[].attributes.body
https://en.islcollective.com/english-esl-worksheets/search/swimming?page=1

https://www.eslprintables.com/vocabulary_worksheets/sports/swimming/

https://ellii.com/lessons/word-bank/2484-swimming

https://www.allthingstopics.com/swimming.html

# data[].attributes.attachment_link.title
35 Swimming English ESL worksheets pdf & doc

# data[].attributes.attachment_link.description
A selection of English ESL swimming printables

# data[].attributes.body
https://en.islcollective.com/english-esl-worksheets/search/skiing

https://oxfordhousebcn.com/en/essential-ski-vocabulary-infographic/

https://www.englishclub.com/vocabulary/sports-skiing.php

https://myvocabulary.com/word-list/skiing-vocabulary/

# data[].attributes.attachment_link.title
Essential Ski Vocabulary [Infographic] | Oxford House Barcelona

# data[].attributes.attachment_link.description
Are you a ski-fanatic that spends all week dreaming about white-capped peaks, fluffy snow and hearty mountain food? Here's an infographic with the essential ski vocabulary to help you learn the lingo before hitting the slopes!

# data[].attributes.body
https://www.englishclub.com/vocabulary/illustrated/bicycle.php

https://www.englishlessonviaskype.com/useful-english-vocabulary-for-cycling/

https://premierskillsenglish.britishcouncil.org/podcasts/fitness-challenge/learning-vocabulary-cycling

https://premierskillsenglish.britishcouncil.org/skills/listen/podcasts/learning-vocabulary-get-your-bike

# data[].attributes.attachment_link.title
Illustrated Bicycle Vocabulary | EnglishClub

# data[].attributes.attachment_link.description
Look and learn in English - saddle, brake, crossbar, and many other terms for parts of a bicycle, all pictured. Vocabulary for ESL learners and teachers.

# data[].attributes.headline
Learn English words for different types of sport (ESOL Courses)

# data[].attributes.subject
Learn English words for different types of sport (ESOL Courses)

# data[].attributes.body
https://www.esolcourses.com/content/topics/sport/english-words-for-sports.html

# data[].attributes.body
https://www.englishclub.com/vocabulary/sports-rugby.php

Quiz: https://www.englishclub.com/vocabulary/sports-rugby-quiz-vocab.php

https://www.eslprintables.com/vocabulary_worksheets/sports/rugby/

https://www.youtube.com/watch?v=AlxouBMsNQY

https://www.twinkl.co.th/search?q=tag+rugby+vocabulary

# data[].attributes.attachment_link.readable_text
This page covers the vocabulary and language of rugby by looking at its background, equipment and people. Check your understanding with the vocabulary quiz and comprehension quiz.

In 1845, three boys at Rugby School in England published the rules of a game that had been played at their school for at least 100 years. They called it "rugby football" and all the modern forms of rugby developed from their game, including rugby league, touch rugby, rugby sevens, wheelchair rugby and rugby union, as well as other games such as American and Canadian football. While all the forms of rugby are popular, rugby union is the form of the game we will be looking at here. It is by far the best known, and its premier tournament, the Rugby World Cup, is one of the biggest sporting events in the world. Only football's FIFA World Cup and the Olympic Games are watched by more people than the Rugby World Cup.

History

Many historians believe rugby developed from a game called "Cornish hurling" that was played in Cornwall, England, in the 17th and 18th centuries. In this game, the first team to get the ball to their "goal" was the winner, and there were very few rules and no referees. It was a wild and dangerous game, and rugby may have been created as a less-violent form of Cornish hurling, with rules and a referee, that upper-class boys in a school like Rugby could play without getting seriously hurt.

After the boys at Rugby school published the rules of their game in 1845 it spread throughout England, and in 1871 clubs from all over the country met in London to form the Rugby Football Union. The game continued to develop until 1895, when a major dispute over payments to players led to the formation of a rival organization, the Northern Rugby Football Union. These two organizations competed for control of the game, but neither could win total control. As a result, two forms of the game, or "codes", developed. One was run by the Rugby Football Union and called "rugby union", and players of this code couldn't earn money from the game (though rugby union players can earn money now). The other was run by the Northern Rugby Football Union and called "rugby league", and players of this code were allowed to earn money. These two codes have slightly different rules, with the most important differences being in the way play is continued after a player has been tackled, and in the number of players on each team; 15 for rugby union and 13 for rugby league. Both codes spread to many countries in the 20th century, and countries that now have strong rugby union teams include France, Australia, England, Ireland, New Zealand, Fiji, Canada, South Africa, Wales, Argentina, Italy, and Scotland.

Football is a gentleman's game played by ruffians, and rugby is a ruffian's game played by gentlemen.

This old British saying cleverly contrasts football (or soccer) with rugby. "Ruffian" is an old-fashioned word meaning a tough, violent, possibly criminal person. The saying shows the irony of the fact that a rough and dangerous game like rugby was played by polite, well-educated "gentlemen", while the much gentler and safer game of football was played by tough, lower-class men with a reputation for violence. Even today rugby players might seem to be very polite gentlemen when compared to many footballers, especially those seen swearing at referees and angrily abusing them when a decision goes against them. Some people might even say that this old British saying still applies today.

How the Game Works

Rugby union is usually played on a grass-covered pitch 100 metres long and 70 metres wide (see diagram below). Each team has 15 players and a game lasts for 80 minutes. The game is played with an oval-shaped ball that players can kick, run with, or pass to teammates in order to score tries or kick goals.

To score a try, worth five points, a player must carry the ball across the opposition's try line and ground it. His team's best kicker can then earn two more points for the conversion if he kicks the ball between the goalposts and over the crossbar. But scoring a try is not so easy. When an attacking team's players are running with the ball towards the try line, they are faced with a line of big, strong defenders who are experts at tackling. To get past these defenders, attacking players can pass the ball to each other until one of them finds a gap in the defence through which they can run to score a try. The ball can only be passed backwards or sideways, however; a forward pass is a foul that the referee should penalize.

The ball is usually passed underarm (see above) and it should be thrown ahead of a running player so he can catch it without slowing down. Defenders can only tackle a player who is carrying the ball (see below). They cannot tackle, or even block, any other players. Blocking or tripping is a foul called obstruction.

As soon as a player is tackled, he must pass the ball or let go of it. Any player from either team, including the player just tackled, can then pick up the ball and start a new attack by either passing it, kicking it, or running with it. Players can also form a pack over a loose ball, called a ruck, in order to push it towards the try line. A maul is similar to a ruck, but is formed around a standing player after he has been tackled but not taken to ground. When the ball comes out of a ruck or a maul, a new attack can begin. If the ball is within kicking distance, a player can score a field goal, also called a drop goal, worth 3 points by kicking the ball between the goalposts and over the crossbar with a drop kick.

Fouls and Penalties

A goal worth 3 points can also be scored when a team is awarded a penalty kick. In this case the kicker can use either a drop kick or a place kick. Not all fouls are punished with a penalty kick, however; only the more serious ones like obstruction and dangerous play. For minor offences like a forward pass or a knock-on, a scrum can be awarded (see above). In a scrum, a team's eight forwards link arms, bend over and push against a similar group from the opposing team. A player from the non-offending team then throws the ball into the scrum and players try to gain possession of it by kicking it backward toward their own side. When it comes out of the scrum, it can be picked up and then passed, kicked, or carried forward to begin a new attack. Penalties are awarded by the referee, who controls the game with the help of two other officials called the touch judges. If a player commits a serious offence, the referee can caution him (yellow card) and the player has to go off for ten minutes, or send him off (red card) and the player has to go off for the rest of the game.

If the ball goes out of the field of play (or into touch), a touch judge signals and play stops. The game restarts with a line-out (see right) in which two lines of players, one from each team, stand next to each other at right-angles to the touchline. The ball is then thrown in from the touchline, and the team that gets it can then begin a new attack. Unless the ball goes out of play, or a penalty is awarded, play continues without stopping until somebody scores. After a team scores, a player from the non-scoring team takes a drop kick from the middle of the halfway line to restart the game. Play continues until the referee signals the end of the match, with the team having scored the most points being the winner.

Tournaments

The sport's most important tournament is The Rugby World Cup. It's held every four years and is one of the world's biggest sporting events. The winners are awarded the William Webb Ellis Cup, named after one of the three boys at Rugby School that you read about above. Since the first World Cup was won by New Zealand in 1987, 24 nations have competed in the event, with Australia winning in 1991, South Africa in 1995, Australia again in 1999, England in 2003, and South Africa again in 2007.

Famous Player

Jonah Lomu

One of the greatest sights in sporting history has to have been that of the huge body of New Zealand giant Jonah Lomu, ball tucked like a child's toy under an arm, as he raced towards the try line at the speed of an Olympic sprinter, with would-be tacklers, themselves huge men, helpless to stop him as they lay in his wake after having been fended off easily with a single straight-arm shove to the chest. Jonah was born in New Zealand in 1975, and by the age of 20 he was 1.97 m tall (6 ft 6 in) and weighed 119 kg (260 lb/18.7 st). And he could run 100 metres in 10.89 seconds! Rugby fans were amazed when he first came onto the international stage during the 1995 World Cup in South Africa. He scored seven tries in five matches, including four in the semi-final against England. In the 1999 Rugby World Cup he scored eight more tries, making a record total of fifteen World Cup tries, more than any other player had scored at the time. Jonah Lomu was inducted into the International Rugby Hall of Fame in 2007 and he will long be remembered as one of the most exciting rugby players of all time.
Jonah Lomu, legendary All Black wing, in action:
Rugby Vocabulary List

word

example sentence

meaning

drop goal ( also field goal)

England's Jonny Wilkinson has kicked many great drop goals.

drop kick through the goalposts during normal play, worth three points

field goal ( also drop goal)

I'm not very good at drop kicks so I never try to kick field goals.

drop kick through the goalposts during normal play, worth three points

goal line ( or try line)

I nearly got a try, but I was tackled just a yard before the goal line.

line at each end of the pitch on which the goalposts are erected

try line

I've seen Lomu push his way past five big defenders to get to the try line.

line at each end of the pitch on which the goalposts are erected (see goal line)

Rugby Vocabulary Quiz

# data[].attributes.attachment_link.description
The vocabulary of rugby and the terms we use in talking about it. Background history and famous rugby players. With vocab, example sentences and quiz. Vocabulary for ESL learners and teachers.

# data[].attributes.body
https://www.esolcourses.com/topics/tennis.html

https://breakingnewsenglish.com/0412/06.spainDavisCup.html

https://learnenglish.britishcouncil.org/general-english/audio-zone/tennis-keeping-fit

https://www.talkenglish.com/lessondetails.aspx?ALID=677

# data[].attributes.attachment_link.readable_text
This page covers the vocabulary and language of tennis by looking at its background, equipment and people. Check your understanding with the vocabulary quiz and comprehension quiz.

Tennis is one of the world's most widely-played sports, enjoyed by players of all ages. It is also a popular spectator sport, with star players in glamorous tournaments watched on TV by millions of fans all over the world. Women's matches attract as much interest as the men's, and tennis is one of the few professional sports in which women can earn as much prize money as men. Singles matches are played between two players and doubles matches are between two teams of two players. Each player uses a racket (also spelled racquet) to hit a felt-covered ball over a net into their opponent's side of a court. A player wins a point when his or her opponent cannot return a shot, or the opponent's shot doesn't land in the court.

History

The modern form of tennis first appeared in the 19th century, but earlier forms of the game had been played in Europe for centuries. The most well-known was real tennis (or royal tennis), which had been popular with royal families and the rich since the thirteenth century. In 1571, French King Charles IX gave permission for the 'Corporation of Tennis Professionals', a type of 'pro tour', to be started, which shows how popular real tennis had become. The game became less popular around the time of the French Revolution, however, because of its links with royalty and the 'ruling classes' that people blamed for social injustices. In England in 1873, Major Walter Clopton Wingfield revived interest in the game by simplifying the rules and designing new courts that were easier and cheaper to build. He called his new version of the game 'lawn tennis' and kept most of the old scoring system, and many of the original French words used in real tennis, such as love and deuce.

Why does love mean zero?
Love, meaning zero (in tennis only), comes from the French expression l'oeuf, meaning the egg, and was used because of an egg's round shape, similar to zero (0).

How the Game Works

Tennis is played on a rectangular court, usually with a grass, clay, or hard court surface. The court is 78 feet (23.77 m) long and 27 feet (8.23 m) wide for singles matches and 36 feet (10.97 m) wide for doubles matches. The net is 3 feet 6 inches (1.07 m) high at the outer posts and 3 feet (.914 m) high in the middle. Lines divide the court into different areas, including four 'service boxes' in the centre of the court, in which serves must land. Before a match begins, one player is designated server in the first game, and play begins with this player serving the ball from one side of the back of the court into the service box diagonally opposite. If the serve is good, the receiver must hit it back into the server's side of the court. Play continues until one player cannot return a shot, or hits a shot out of the court, and loses the point. The server then serves again, and play continues until one player scores enough points to win the first game. Then the second game begins with the player who received in the first game now serving. Play continues until a player wins the first 'set' by being the first to win at least six games. Then the second set begins, and play continues until a player has won enough sets to win the match. In a 'best of three' match, the winner is the first to win two sets, and in a 'best of five' match, the winner is the first to win three sets.

Professional players learn to use many different shots, including the serve, forehand, backhand, volley, half-volley, overhead smash, drop shot, and lob. They also learn how to put spin on their shots to make them more difficult to return, and learn how to use tactics to win, such as by playing to their opponent's weaknesses.

Tournaments

Every year hundreds of tournaments are held for professional players. The biggest of these are the Grand Slam tournaments; the Australian Open, the French Open, The Championships at Wimbledon in England, and the US Open. The greatest challenge in tennis is to win all four Grand Slam tournaments in one year. In men's singles this has only ever been done by two players; Don Budge (1938) and Rod Laver (1962 and 1969). In women's singles it has been done by only three players; Maureen Connolly (1953), Margaret Smith Court (1970), and Steffi Graf (1988).

Famous Players

Rod Laver

Born in 1938 in Rockhampton, Australia, Rod Laver is said by many tennis historians to be the greatest player of all time. He is the only player to have won the men's singles Grand Slam two times. At 1.72 metres, Rod Laver was not as tall as many of today's players, but he had a strong arm and his shots were powerful. He was very fast around the court, and very accurate with his play. He was also a very clever player, and often won matches by outwitting his opponents tactically.

Steffi Graf

Steffi Graf was born in Mannheim, Germany in 1969, and is one of the greatest female tennis players of all time. She won 107 singles titles, including 22 Grand Slam singles titles. In 1988, Graf became the only player to have ever achieved a 'Golden Slam' by winning the Olympic gold medal in singles as well as all four Grand Slam singles titles that year. No other man or woman has ever done this. She also holds the record for the total number of weeks she was rated as the number one player in the world at 377 weeks. She is also the only player to have won all four Grand Slam singles tournaments at least four times each. Steffi married Andre Agassi, the former world number one in men's tennis.

Tennis Vocabulary List

word

example sentence

meaning

advantage

With the score at deuce, Roger returned serve with a winner and the umpire announced, 'Advantage Federer'.

a player who scores a point at 'deuce' has the advantage, meaning if they win the next point, they win the game

ballperson
ballboy/girl/kid

When I was young I was a 'ballboy', and my son does the same job now, but he's a 'ballperson'.

youngsters who collect balls from the court and give them to players as required

deuce

The word deuce was used in real tennis for hundreds of years before the modern game developed.

a score of 40-40, after which a player must win two consecutive points to win the game

set

Elena won the first set, but Maria took the next two sets to win the match.

in scoring, a player must win at least six
games to win a set

Tennis Vocabulary Quiz

# data[].attributes.attachment_link.description
The vocabulary of tennis and the terms we use in talking about it. Background history and famous tennis players. With vocab, example sentences and quiz. Vocabulary for ESL learners and teachers.

# data[].attributes.body
https://www.esolcourses.com/topics/football.html

https://en.islcollective.com/english-esl-worksheets/search/football

https://yourenglishpal.com/blog/esl-conversation-lesson-plan-football-soccer/

https://www.teachingenglish.org.uk/teaching-resources/teaching-secondary/lesson-plans/pre-interemediate-a2/football-and-motivation

https://www.onestopenglish.com/reading/reading-lesson-plans-football/146274.article

# data[].attributes.body
https://www.englishclub.com/vocabulary/sports-cricket.php

https://www.engvid.com/learn-about-sport-cricket/

# data[].attributes.attachment_link.readable_text
This page covers the vocabulary and language of cricket by looking at its background, equipment and people. Check your understanding with the vocabulary quiz and comprehension quiz.

Everybody agrees that the most popular sport in the world today is soccer. But which is the second most popular? Is it basketball? Maybe rugby? Tennis perhaps? No, as you've probably guessed by now, the answer seems to be cricket. The reason for this is that cricket is the number one sport in many countries with huge populations, such as India, Pakistan and Bangladesh. It is popular in many other countries as well, including the U.K., Australia, South Africa, Sri Lanka and New Zealand. Cricket is, like baseball, a "bat and ball" game in which bowlers "bowl" the ball and batsmen try to hit "shots" with a bat and score runs for their team. As in baseball, batsmen are "out" if their shot is caught, or if they don't get to a "safe haven" in time when they're making runs. What is very different, however, is the time taken to play the game. In cricket, a single game in the traditional "Test match" format can take five full days to complete! But thankfully there are shorter formats for people who love to play the game but also have to go to work sometimes.

History

Cricket has been a team sport in Britain for hundreds of years. Even though there is evidence that the game existed in England as early as the twelfth century, it wasn't until the eighteenth century that it became a well-organized sport. This resulted from cricket's growing popularity within traditional gentlemen's clubs, especially in London. These clubs were devoted to their wealthy members' enjoyment of eating, drinking, gambling and sports. Cricket was the perfect game for such gentlemen. It was not too demanding physically and could be played with dignity even by slightly older gentlemen, unlike more athletic games such as football and rugby. In 1787, the development of the game took a step forward with the formation of the Marylebone Cricket Club by a group of powerful and wealthy cricket lovers. It was based at Lord's Cricket Ground near St John's Wood in London, and MCC members took it upon themselves to write the "Laws of Cricket", upon which the game is still based today. The MCC also became the governing body of cricket in England, in which it still has a role to this day, and also governed international cricket until 1909 when the ICC (International Cricket Council) was formed.

One of the biggest changes to the game in the nineteenth century was the replacement of under-arm bowling with over-arm bowling. This meant one of the most exciting parts of the game, fast bowling, was now possible. International matches also began in the nineteenth century, with the first match being between the U.S.A. and Canada in 1844. While the game never became popular in these two countries, it became very popular in most Commonwealth countries and an international system of "Test matches" between these countries soon developed. The first official Test match was between England and Australia in March 1877 at Melbourne Cricket Ground, which the Australians won by 45 runs. Test cricket is the most demanding form of the game, with matches lasting up to five days. In the late twentieth century, shorter forms of the game developed, such as One Day Internationals and the Twenty20 format. These developed in order to satisfy demands for shorter, faster games that would make exciting viewing on television. An example of this development is the Cricket World Cup, a tournament in the one-day limited-over format, which is held every four years and has now become the world's third most viewed sporting event. Cricket, like most other sports, has now become very big business, and the media form a major source of income for teams, players and governing bodies alike.

How the Game Works

A cricket match is usually played between two teams of eleven players on an oval-shaped grass field. In the centre of the field is a strip of hardened turf 22 yards (20.12 m) long, called a cricket pitch. At each end a set of three tubular stakes called "stumps" are stuck upright in the earth with two small pieces of wood, called bails, balanced across the top of each set. These are the wickets, and a batsman stands in front of a wicket with his long wooden bat as he waits for a bowler to bowl the hard, fist-sized ball. The bowler bowls from just beside the wicket at the other end of the pitch. At the same time, the other members of the bowler's team stand in various positions around the field acting as fielders. If the batsman misses the ball, and the ball hits the wickets, he is out and his turn to bat is over. If he hits the ball into the air and a fielder catches it, he is out. If a ball which the umpire thinks is going to hit the wicket is blocked by batsman's legs, he is out lbw (leg before wicket). But if the batsman hits the ball, he and his batting partner, who is standing at the other end of the pitch, can run to the opposite ends of the pitch to score one run. They can score two runs by each running back again. Three runs are scored if each batsman runs three lengths of the pitch, and so on. But if a fielder gets to the ball and throws it at one of the wickets, and the ball hits the wicket before a running batsman crosses a line near the wicket called the crease, the batsman is "run out". In the Test match format, teams take turns to bat, and each team continues batting until ten players have gone "out", or until the team's captain "declares" and his team finishes its innings early. A captain can do this to ensure his team has a chance of winning the match, which they can only do if there's enough time left for both teams to complete two innings. If they do, the team with the highest run total wins. But if the five days allowed for the match have passed before both teams have completed two innings, the match is a draw. In the shorter forms of the game, each team has just one innings in which to bat, and the innings is sometimes limited to a fixed number of overs. The team scoring the most runs in its innings is the winner.

Most players specialize in batting (see right) or bowling, or they are "all-rounders", meaning they are good at both. Each team also has a wicketkeeper, who stands behind the wickets and catches any balls that aren't hit by the batsmen. The central contest in cricket is between a batsman and a bowler. A batsman's job is to hit shots and score runs, while a bowler tries to get batsmen out, or take wickets, while trying to prevent runs from being scored. A bowler bowls six overarm balls in one over, and then returns to his fielding duties while another bowler bowls another six-ball over from the other end of the pitch. There are several styles of bowlers, including fast bowlers, leg spinners, off spinners, swing bowlers and medium-paced bowlers. Most bowlers specialize in one type of delivery, and most deliveries are supposed to hit the pitch once and bounce up again before reaching the batsman. A ball which doesn't bounce, but reaches the batsman on the full, is a full-toss. Spin bowlers use the fingers or the wrist to put spin on the ball so that it doesn't come straight up off the pitch after bouncing, but bounces up at an unexpected angle. Swing bowlers try to get the ball to move unexpectedly through the air, while fast bowlers try to beat the batsmen with speed. If a batsman manages to hit the ball, he could hit one of several shots. These include drives, hooks, cuts and blocks. If a player hits a shot over the boundary line at the edge of the field on the full, he has "hit a six" and gets six runs. If the ball bounces or runs along the ground on the way to the boundary, he has "hit a four". One of the most important skills a batsman needs is the ability to place his shots between the fielders so that he won't be caught out, and so that he has time to score runs before a fielder can get to the ball and throw it back towards the wickets.

There are two umpires on the field during a game of cricket, and they are responsible for deciding whether a batsman is out or not, and for deciding whether a ball is legal or not, with illegal deliveries called no-balls. Umpires also keep count of the number of balls a bowler delivers and lets them know when an over has finished. They must also judge whether a batsman has been "run out" by judging whether he has reached the crease before the ball hits the wicket.

Most Famous Player

Don Bradman

Around the time of the First World War, a young boy in the small Australian town of Bowral spent hours every day playing alone with a cricket stump, a golf ball and a water tank. He'd use the stump to hit the ball against the tank's brickwork base, and the golf ball would bounce back quickly at unpredictable angles. But the boy became so fast that he could hit the ball with the stump again and again without letting it touch the ground. The boy went on to use the quick reflexes and sharp eye he developed beside the water tank to become cricket's greatest batsman of all time.

Sir Don Bradman (1908 - 2001) played in 52 Test matches for Australia from 1928 to 1948. His Test batting average of 99.94 is generally regarded as one of the most remarkable achievements in any sport. The next best batting average for a player who has played more than 20 Test match innings is just 60.97 by South Africa's Graeme Pollock. Before his last Test match, at The Oval in London, Bradman's Test batting average was 101.39. But in his last innings he was bowled by the second ball he faced. If he had scored just 4 runs, his average would have been over 100, but having scored no runs, he finished his Test career with an average of 99.94 runs per innings. For many years it was claimed that the magnificent ovation Bradman received as he walked out to the pitch for his last Test match had left him with tears in his eyes, and the tears had made him miss the ball and "go out for a duck". Bradman always claimed there was no truth to this story.

Cricket Vocabulary

word

example sentence

meaning

boundary

A fielder stands on the boundary to catch the ball and also to stop boundaries from being scored.

1 the perimeter of the ground
2 shot to the boundary for four or six runs

Cricket Vocabulary Quiz

# data[].attributes.attachment_link.description
The vocabulary of cricket and the words we use in talking about it. Background history and famous players. With vocab, example sentences and quiz. Vocabulary for ESL learners and teachers.

# data[].attributes.body
Simplified resource

# data[].attributes.attachment_link.description
The open register has lists of names and addresses of people who are registered to vote. This means your name and address will be on the list. Charities and businesses can buy these. If you do not want to be on these lists you need to click the square box to opt out.

# data[].attributes.headline
Refugees and migration teaching resources - British Red Cross

# data[].attributes.subject
Refugees and migration teaching resources - British Red Cross

# data[].attributes.attachment_link.title
Refugees & migration teaching resources | British Red Cross

# data[].attributes.attachment_link.description
Free refugee and migration teaching resources to help students learn about the challenges faced by refugees and migrants.

# data[].attributes.attachment_link.title
Activity - Seeking safety Amnesty International Primary 7 plus (1)

# data[].attributes.headline
ESOL Literacies National 2 Learning Support Material

# data[].attributes.subject
ESOL Literacies National 2 Learning Support Material

# data[].attributes.attachment_link.description
Find SQA ESOL Literacies National 2 learning support material here such as guides.

# data[].attributes.headline
Languages - according to 2021 Census for England and Wales

# data[].attributes.subject
Languages - according to 2021 Census for England and Wales

# data[].attributes.body
1.  Polish (612,000)
2. Romanian (472,000)
3. Panjabi (291,000)
4. Urdu (270,000)
5. Portuguese (225,000)
6. Spanish (215,000)
7. Arabic (204,000)
8. Bengali (199,000)
9. Gujarati (189,000)
10. Italian (160,000)
11. Tamil (160010)
12. French (120,000)
13. Lithuanian (119656)
14. All other Chinese (118,000)
15. Turkish (112,978)
16. Bulgarian (111,431)
17. Russian  (91255)
18. Persian or Farsi (87,713)
19. Hungarian (87356)
20. Greek (76,675)
21. Nepalese (74,368)
22. Kurdish (66,443)
23. Malayalam (65,008)
24. Somali (62,240)
25. Tagalog or Filipino (60899)
26. Cantonese Chinese (55,555)
27. Albanian (52,328)
28. Hindi (51,260)
29. Slovak (49,924)
30. Pashto (48,705)

Source : https://www.ons.gov.uk/peoplepopulationandcommunity/culturalidentity/language/bulletins/languageenglandandwales/census2021

# data[].attributes.attachment_link.readable_text
This page is also available in Welsh (Cymraeg) (PDF, 698KB)

In 2021, 91.1% (52.6 million) of usual residents, aged three years and over, had English (English or Welsh in Wales) as a main language (down from 92.3%, or 49.8 million, in 2011).

In 2021, a further 7.1% (4.1 million) of the overall population were proficient in English (English or Welsh in Wales) but did not speak it as their main language.

The most common main languages, other than English (English or Welsh in Wales), were: Polish (1.1%, 612,000), Romanian (0.8%, 472,000), Panjabi (0.5%, 291,000), and Urdu (0.5%, 270,000).

The largest increase was for people who specified Romanian as a main language, who accounted for over 0.8% of usual residents in 2021 (472,000 people), up from 0.1% (68,000) in 2011.

In 2021, 63.8% (15.8 million) of households consisted of members who all had the same main language; 6.0% (1.5 million) had different main languages within the household.

Tell us what you think about this publication by answering a few questions.
Back to table of contents

2. Main languages in England and Wales

English (English or Welsh in Wales) as a main language

In England and Wales, 91.1% of usual residents aged three years and over (52.6 million out of 57.7 million) had English (English or Welsh in Wales) as a main language. This is a percentage decrease compared with 2011, when 92.3% (49.8 million) had English (English or Welsh in Wales) as a main language.

In 2021, a further 7.1% (4.1 million) of the overall population were proficient in English, so they spoke English either "well" or "very well" (English or Welsh in Wales), but did not speak it as their main language. Additionally, 1.5% (880,000) could not speak English well, and a small percentage (0.3%, 161,000) of the overall population could not speak English at all.

Main languages, other than English (English or Welsh in Wales)

Polish remained the most common main language accounting for 1.1% (612,000) of usual residents, for those who do not have English (English or Welsh in Wales) as a main language.

In 2021, Romanian moved into the top 10 main languages, excluding English (English or Welsh in Wales). This was the largest increase over the decade. In total, 0.8% of people (472,000) listed Romanian as their main language, up from 0.1% (68,000) in 2011. This mirrors a similar increase in the number of people who listed Romania as their country of birth between 2011 and 2021 (an increase between censuses of 459,000, or a 0.8 percentage point change). It also mirrors an increase in the number who chose only a Romanian national identity (an increase between censuses of 405,000, or a 0.7 percentage point change).

Figure 1: The top ten main languages spoken in England and Wales, excluding English (English or Welsh in Wales)

Total usual resident population, aged three years and over, who speak each language as their main language, 2011 and 2021, England and Wales

Download the data
Back to table of contents

3. Main languages varied across England and Wales

English (English or Welsh in Wales) as a main language

The percentage of people who spoke English (English or Welsh in Wales) as their main language was higher in Wales (96.7%, 2.9 million out of 3.0 million) than it was in England (90.8%, 49.7 million out of 54.7 million).

In Wales, people were asked if their main language was anything other than English or Welsh. Therefore, it is not possible to determine how many people in Wales consider Welsh to be their main language. There was a separate question for people in Wales asking them about their Welsh language ability. On 6 December 2022, the Welsh Government will be publishing information on Welsh language skills in Wales.

The region of England with the highest percentage of people with English as a main language was the North East (96.5%, 2.5 million), whereas London had the lowest percentage (78.4%, 6.7 million).

Figure 2: The percentage of people with English (English or Welsh in Wales) as a main language varied across local authorities in England and Wales

Percentage of the usual resident population, aged three years and over, who reported English (English or Welsh in Wales) as a main language, 2021, local authorities in England and Wales

Source: Office for National Statistics - Census 2021

Download the data

Polish as a main language across England

In England, Polish (1.1%, 591,000) was the most common main language for those who did not speak English as a main language. At a regional level, the highest percentage of people who had Polish as a main language was in the East Midlands (1.5%, 71,000). The percentage of people with Polish as a main language varied across local authorities from 0.1% (100) in Castle Point to 5.7% (4,000) in Boston.

Romanian as a main language across England

The next most common main language in England was Romanian (0.9%, 466,000). The region with the highest percentage of people who reported Romanian as a main language was London (1.9%, 159,000). In particular, the local authority of Harrow had the highest percentage of its population reporting Romanian as a main language (7.5%, 19,000).

Panjabi and Urdu as main languages across England

The third and fourth most common languages, excluding English, both originate from South Asia: Panjabi (sometimes spelt Punjabi) and Urdu. They are both widely spoken languages in India and Pakistan, as well as elsewhere in South Asia. In England, the highest percentage of people that had Panjabi as a main language was in the West Midlands (1.4%, 83,000). Wolverhampton was the local authority with the largest percentage of people with Panjabi as a main language (6.5%, 17,000). The English region with the overall largest proportion of people who had Urdu as a main language was the North West (0.8%, 59,000). However, Slough in the South East was the local authority with the largest proportion (4.3%, 7,000).

The most common main languages, other than English or Welsh, in Wales

As in 2011, Polish was the second most common language after English or Welsh in Wales (0.7%, 21,000). The Welsh local authority with the highest proportion of people that had Polish as a main language remained as Wrexham (2.5%, 3,000).

Arabic was the next most common main language in Wales (0.3%, 9,000). The Welsh local authority with the highest proportion of people that had Arabic as a main language was Cardiff (1.4%, 5,000).

Figure 3: Main language spoken, excluding English (English or Welsh in Wales)

Percentage of total usual resident population, aged three years and over, who speak each language as their main language, 2021, local authorities in England and Wales

Download the data

British Sign Language in England and Wales

In addition to spoken languages, British Sign Language (BSL) was the main language of 22,000 (0.04%) usual residents aged three years and over across England and Wales. This is an increase of over 6,000 since 2011 (15,000, 0.03%).

Within England, BSL is the main language of just over 21,000 people (0.04%). This percentage is slightly higher than in Wales, where just over 900 people (0.03%) have BSL as a main language. Across the two nations, the area with the highest percentage of people with BSL as their main language was Derby (400, 0.2%), as it had been in 2011 (300, 0.1%). One reason for this could be that the Royal School for the Deaf is situated in Derby, so children have access to both BSL and English. For those who move for educational purposes, they may remain in Derby because of an active Deaf community.
Back to table of contents

4. English language proficiency

People who did not report English (English or Welsh in Wales) as a main language were asked to report how well they could speak English (8.9%, 5.1 million). Of those 5.1 million people, 43.9% (2.3 million) could speak English very well, 35.8% (1.8 million) could speak English well, 17.1% (880,000) could not speak English well, and 3.1% (161,000) could not speak English at all.

For households that did not have English as a main language, assistance in completing the census questionnaire was provided through interpretation services, and translation leaflets were made available in over 50 languages.

Figure 4: Most people who do not have English (English or Welsh in Wales) as a main language can speak English "Well" or "Very well"

Percentage of the usual resident population, aged three years and over, who do not identify English (English or Welsh in Wales) as their main language, 2011 and 2021, England and Wales

Download the data

The following interactive tool can be used to consider how English language proficiency, for those who do not speak English (English or Welsh in Wales), varies across England and Wales.

Figure 5: English language proficiency varies across local authorities of England and Wales

Percentage of the usual resident population, aged three years and over, who do not identify English (English or Welsh in Wales) as their main language, 2021, local authorities in England and Wales

Download the data
Back to table of contents

5. Main languages within households

There were 24.8 million households across England and Wales in 2021, up 6.1% from 23.4 million in 2011.

For the first time, we collected information on whether household members have the same or different main language. For the 24.8 million households, 69.8% were households with more than one person. Of these households:

all members have the same main language (15.8 million) in 63.8% of households

the main language differs between generations, but not within partnerships (480,000), in 1.9% of households

the main language differs within partnerships (534,000) in 2.2% of households

main languages differed between other household relationships, such as between friends (465,000), in 1.9% of households

Figure 6: Proportion of households where the main language differs, by household type, 2021, local authorities in England and Wales

Notes:

The following categories have been aggregated into "Multiple main languages within household": "Main language differs between generations, but not within partnerships", "Main language differs within partnerships", and "Any other combination of multiple main languages".

Download the data
Back to table of contents

On 6 December 2022, the Welsh Government will be publishing information on Welsh language skills in Wales, including an age breakdown.

More detailed data and analysis on language will be published in the coming months, alongside the release of multivariate data. Read more about our ethnic group, national identity, language and religion analysis plans and the release plans for Census 2021 more generally.
Back to table of contents

English language proficiency

How well people whose main language is not English (English or Welsh in Wales) speak English.

Main language

A person's first or preferred language.

Multiple main languages in household

Classifies households by whether members speak the same or a different main language. If multiple main languages are spoken, this identifies whether they differ between generations or partnerships within the household.

Usual resident

A usual resident is anyone who on Census Day, 21 March 2021, was in the UK and had stayed or intended to stay in the UK for a period of 12 months or more or had a permanent UK address and was outside the UK and intended to be outside the UK for less than 12 months.
Back to table of contents

Reference date

The census provides estimates of the characteristics of all people and households in England and Wales on Census Day, 21 March 2021. It is carried out every 10 years and gives us the most accurate estimate of all the people and households in England and Wales.

We are responsible for carrying out the census in England and Wales but will also release outputs for the UK in partnership with the Welsh Government, the National Records of Scotland (NRS) and the Northern Ireland Statistics and Research Agency (NISRA). The census in Northern Ireland was also conducted on 21 March 2021, whereas Scotland's census was moved to 20 March 2022. All UK census offices are working closely together to understand how this difference in reference dates will affect UK-wide population and housing statistics, in terms of both timing and scope.

Response rate

The person response rate is the number of usual residents for whom individual details were provided on a returned questionnaire, divided by the estimated usual resident population.

The person response rate for Census 2021 was 97% of the usual resident population of England and Wales, and over 88% in all local authorities. Most returns (89%) were received online. The response rate exceeded our target of 94% overall and 80% in all local authorities.

Further information on question-specific response rates will be published in a separate report later this year.
Back to table of contents

10. Strengths and limitations

Quality considerations, along with the strengths and limitations of Census 2021 more generally, can be found in the Quality and Methodology Information (QMI) for Census 2021. Read more about the specific quality considerations for Ethnic group, national identity, language, and religion quality information for Census 2021.

Further information on our quality assurance processes is provided in our Maximising the quality of Census 2021 population estimates report.
Back to table of contents

# data[].attributes.attachment_link.title
Language, England and Wales - Office for National Statistics

# data[].attributes.attachment_link.description
Main language, English language proficiency, and household language in England and Wales, Census 2021 data.