NAO: - Investigation into cheating in English tests

An extract from another interesting article from the team at "National Audit Office". Click on the link to find our more or register for their regular updates:

Investigation into the response to cheating in English language tests

This investigation examines the Home Office’s response to suspected cheating in the Test of English for International Communication (TOEIC).

Background to the report

In 2014 the BBC’s Panorama programme uncovered examples of organised fraud taking place during Secure English Language Tests (SELTs) required under immigration rules for students and other visas. The Home Office (the Department) responded by suspending the licences of test centres and revoking the visas of people accused of cheating. A number of people have protested their innocence. The Home Affairs Select Committee has carried out an inquiry, and the House of Commons has debated the matter, without achieving a clear resolution.

This investigation examines the Department’s response to suspected cheating in the Test of English for International Communication (TOEIC), a SELT administered by Educational Testing Service Global BV, a subsidiary of Educational Testing Service USA (ETS), a non-profit educational testing and assessment organisation. We consider how the Department assessed the number of people who may have cheated and what action it took.

  Content and scope of the report

We looked at the Department’s approach to identifying cheats and at its quality assurance work. In particular, we looked at its estimates of the number of people cheating, the steps it took to assess the evidence, and what happened to people subsequently. We examined the Department’s data and documents, reviewed published material and met with a number of stakeholders.

A number of factors make it challenging to provide a consistent picture over time about what happened to the individuals concerned. These factors include: the different kinds of monitoring the Department undertook between 2014 and the present, the evolution of the Department’s approach to legal cases and changes in individuals’ circumstances which have in some cases altered their immigration status.

  Concluding remarks

Evidence from ETS, Departmental analysis and successful prosecutions against those involved in administering test centres strongly suggests that there was widespread abuse of the Tier 4 visa system. The exact scale of that abuse is difficult to determine because experts drew different conclusions over the reliability of voice recognition checks. It may be right to say that the Department expert was more informed than the NUS expert, but not that he was sufficiently well informed to provide a definitive view. The Department nonetheless relied on his judgement to assume that collateral damage would be low. The data environment is weak and the Department has not been able to independently validate ETS judgements about cheating. It is reasonable based on the balance of probabilities to conclude there was cheating on a large scale because of the unusual distribution of marks, and high numbers of invalid tests in test centres successfully prosecuted for cheating.

The Department decided to react vigorously to the evidence of widespread cheating that was reported by Panorama, and which confirmed their pre-existing concerns. Widespread action to close colleges meant students who did not sit TOEIC exams have been affected, some of whom have struggled to find replacement sponsors. The courts found in 2016 that the Department’s evidence was sufficient to make the accusations it did. Nonetheless, the Department’s course of action against TOEIC students carried with it the possibility that a proportion of those affected might have been branded as cheats, lost their course fees, and been removed from the UK without being guilty of cheating or adequate opportunity to clear their names. To date the Home Office has considered its evidence sufficient to tell people that the number of people wrongly affected would be very small, and that they can appeal from outside the UK or lodge a Judicial Review.

Holly Gregory Associates provides HR Consultancy and Change Management Services. We are known as trusted advisers, and provide specialist HR management advice to help your business maximise the efficiency of your HR Operations. We will work with you to implement appropriate new policies and procedures in the best way possible. As experienced change agents wesupport you by delivering your HR changes by providing qualified HR Transformation and Restructuring Project Managers. Our Change Managers come with the knowledge of change methodologies. It may also be that you need is for our specialist HR Services e.g Outsourcing your HR, support writing or creating Policies and Procedures, HR support with redundancies due to a Merger and Acquisition and TUPE, or help to ensure compliance of your assessments with the up and coming IR35 Private Sector reforms.