English language test: There is nothing wrong with the integrity of the process by which Educational Testing Service (“ETS”) identifies its English language test results as “invalid” or “questionable”, the Upper Tribunal has said. As a result, where ETS provides evidence indicating that the test relied upon by an individual was taken by someone else, this is evidence that meets the required civil standard of a candidate’s dishonesty and calls for a credible explanation in reply.
This is so despite evidence that fraudulent administrators could fake test results using a “hidden room” in such a way that a genuine candidate would be wholly unaware of any fraud. The reason is that the risk of this having ever happened remains speculative.
This is the conclusion of President Mr Justice Dove and Upper Tribunal Judge Mandalia in Varkey & Joseph (ETS – Hidden rooms) v SSHD [2024] UKUT 00142 (IAC).
Background
The decision is the latest examining the fallout from the BBC Panorama programme broadcast on 10th February 2014, which exposed widespread cheating in the Test of English for International Communication (“TOEIC”). Candidates needed to pass this for immigration purposes. The tests were set centrally by ETS and administered through test centres throughout the United Kingdom.
Fraudulent administrators of English language tests set by ETS faked some test results by having proxies take the seat of candidates and faked others by having proxies take the test via separate computers in “hidden rooms.”
Thousands of tests were passed by fraud, to the extent that some test centres have been described by the Court of Appeal as “fraud factories.”
Mr Varkey was accused by the Secretary of State of using dishonesty in an entry clearance application in 2012. He supplied a TOEIC pass certificate issued by ETS and claimed to have taken the test in person at the London College of Social Studies (“LCSS”).
In 2017 ETS procedures identified Mr Varkey’s speaking test as having been passed by use of a proxy. Ms Joseph, Mr Varkey’s wife, was not implicated in any way in the allegations.
In April 2022 the First-tier Tribunal allowed Mr Varkey’s appeal against the finding that he had used deception. The Secretary of State appealed and the case came to the Upper Tribunal for remaking.
Mr Varkey sought to reopen the findings in DK and RK (ETS: SSHD evidence, proof) India [2022] UKUT 112 (IAC) (“Nothing wrong with official evidence of English language test cheating, Upper Tribunal holds”) which found that ETS evidence, while not conclusive, was overwhelmingly reliable such that it can be relied on by the Secretary of State to prove a case of dishonesty to the required legal standard, i.e. the balance of probabilities.
Mr Varkey relied on new expert evidence from Richard Shury, the Security and Compliance Specialist for ETS Global, and Christopher Stanbury, an expert in computing.
Mr Shury provided evidence that ETS Global knew as long ago as 2013 that “hidden rooms” were used for cheating. His evidence was not shared with the Tribunal in DK and RK.
Mr Stanbury provided evidence of four “hidden room” methods that could potentially be used by fraudulent test administrators. One of the methods involves fraudsters running the ETS test from an “open room” full of candidates, some of whom might be genuine, while a “hidden room” full of proxies would sit the test in parallel. Only the latter tests would be uploaded.
Mr Stanbury gave evidence as to why it would be easier and less risky for fraudsters to upload all faked tests, than to try to separate genuine candidates from those who were in on the fraud. This method would result in a risk of an innocent candidate being identified by the ETS as having used a proxy.
Mr Varkey argued that the reliability of ETS evidence as proof of dishonesty needed to be reopened in light of the above.
The judgment
The Upper Tribunal examined the six methods identified by the experts through which TOEIC fraud could be perpetrated, noting that there may be other methods not yet identified. The Upper Tribunal accepted that there was evidence that the less sophisticated of the six identified methods had definitely been used in practice, including where a proxy physically took a candidate’s seat for the test, or a proxy controlled a candidate’s screen from a hidden room using TeamViewer type software.
Only one of the six methods, referred to as the “parallel testing (cloned manager PC)” method, could operate without a candidate being aware of fraud. However, the Upper Tribunal found that it was “nothing more than speculation” that this method had ever been used.
Consequently, although the evidence of Mr Shury and Mr Stanbury was accepted, the Upper Tribunal found that it did not in any way undermine the findings in DK and RK that evidence from ETS is overwhelmingly reliable and will raise a case to answer.
The Upper Tribunal rejected emphatically a submission that the situation of people facing ETS evidence was analogous to that of sub-postmasters accused of dishonesty in the Horizon IT scandal. In contrast to the situation of the sub-postmasters there was no evidence of problems with the ETS software or manipulation by ETS. There was no proof of widespread dishonesty by sub-postmasters. In contrast, there was a wealth of evidence that test centres dishonestly faked TOEIC results, and that thousands of candidates were complicit.
The Upper Tribunal declined to set out any new prescriptive guidance for ETS cases, emphasising that the burden of proof was correctly described in DK and RK as being the balance of probabilities, and that there needs to be a fact-sensitive analysis of the evidence in each case.
Applying this analysis to Mr Varkey’s evidence, the Upper Tribunal found that he had not answered the case against him and allowed the appeal of the Secretary of State.
Comment
The decision confirms a difficult battle for any person attempting to discredit the ETS methodology for identifying suspect results. However, the Upper Tribunal reiterated that while ETS evidence is “overwhelmingly reliable” this does not mean that it is “conclusive.” The Upper Tribunal concluded in DK and RK that there may be a false positive rate in voice recognition of 1-3%, and appellants may persuade a judge through the combined weight of other evidence that they have answered the Secretary of State’s evidence against them.
