Proctoring: How to Ensure the Integrity of Online Assessment
Five Years’ Experience
HSE University is one of the first Russian universities to use proctoring software for its online exams. It all started in 2016, when students taking HSE courses on the National Open Education Platform began to take exams. At this time, the Proctoring Centre was created at the eLearning Office, and Examus, one of Russia’s leaders in proctoring software, became a tech partner of the University.
Every year at HSE University, several thousand students take proctored exams, so the procedure for the university is not new. 10-12,000 exams are administered with Examus proctoring platform.
Last year alone, HSE University administered:
4,500 proctored exams
for students from partner universities as part of network academic programmes
2,200 proctored entrance exams
for international applicants and new doctoral programme enrollees
More than 2,000 proctored exams
for students of HSE’s online courses on the National Open Education Platform
The Examus Proctoring System is used by most of Russia’s leading universities who offer courses on the National Open Education Platform, as well as corporate universities. Today, it is one of the most reliable systems.
Last year, online proctored entrance exams were conducted for the first time for the applicants of the ‘Master of Data Science’ programme offered fully online in partnership with Coursera.
Synchronous and Asynchronous
There are two types of proctoring: synchronous proctoring and automatic asynchronous proctoring which is then verified afterwards.
In the first case, the exam is monitored in real time by an online or external proctor, an Examus employee, or a specially trained HSE staff member. The activity of the test taker is recorded in two ways: the student is recorded via their web camera, and the student’s desktop (their computer screen) is recorded by the system. Before the exam, the proctor verifies the student’s identity (by checking their passport), and during the exam, the proctor can contact the student via the system chat window. One proctor monitors the exams of up to six students at a time.
In the second case, the computer programme completes the student identity verification process and monitors the entire exam process by monitoring the student via the student’s webcam. The programme flags suspicious behavior (such as the appearance of other individuals in the frame or talking) by placing tags on the video. These flagged instances are then checked by a human proctor when they view the recording.
The decision on whether a student has violated internal rules during an exam (plagiarized or cheated) is made by the student’s instructor. If the student considers the instructor’s decision to be unfair, they can file an appeal, which will be reviewed by their programme academic supervisor and the appeal commission. The advantage of using proctoring software is that the commission will have a documented video of the exam.
During the Pandemic
This spring, proctoring technology was widely used for the HSE Lyceum entrance exams, which students took remotely. According to Lyceum Director Dmitry Fishbeyn, the examination process generally ran smoothly, though the online format proved much more time-consuming for all involved.
Russia’s current self-isolation rules do not allow for taking exams in person, which is why most Russian universities are administering a significant share of their exams online. HSE is no exception: though the majority of student exams for the third and fourth modules will begin on June 10, some programmes began their exams in May. For example, exams began at the ICEF a little earlier due to requirements of its double degree programmes.
17% of the spring-summer session final exams will be administered with proctoring software. On exams administered to smaller groups of students, proctors will observe the students synchronously. Exams that are taken by hundreds or even thousands of students (about 2,000 people can take a foreign language exam at one time, for example) will be proctored asynchronously. In some cases, at the discretion of programme academic supervisors, proctoring may also be used on oral exams. In these cases, students will be monitored while preparing their oral responses on Zoom or MS Teams.
The number of proctored online exams this session has increased tenfold compared to last year. By the end of June, almost 23,000 students will have taken online proctored exams.
Rules of Conduct
The rules for taking and administering online exams with proctoring software are available here. The rules indicate the technical requirements that a user’s computer must meet as well as the necessary internet connection quality. The HSE Proctoring Centre estimates that 7-10% of students will encounter technical difficulties due to a poor internet connection. For example, if a student is taking their exam from a remote area in the country and has to use a cellular network connection, and their family members are also using the connection at the same time, the programme many not be able to support the two required video streams.
To ensure that the proctoring software runs smoothly, students should also pay attention to other aspects that may hinder it. For example, students should make sure their work area is quiet and that it is well lit so that they are visible on camera. In order to check these potential issues as well as your equipment, a trial exam is held the day before.
Almost all of the rules for taking online proctored exams are consistent with the requirements for any onsite exam. The only difference lies in the technical requirements, which are indicated in advance, and these are consistent with the general technical requirements of any distance learning course. If students fulfill these requirements and read the instructions, they will experience no problems during their exams.
Answers to Questions
Teachers who were not familiar with proctoring software had questions, such as whether the software really does ensure the integrity of the examination process and prevent cheating, plagiarism, and other forms of improper behavior.
‘Proctoring is a high-tech procedure. It’s not just a video recording of students and their desktops. It is based on a smart algorithm that identifies and flags changes in student behavior, which the teacher may not notice. Therefore, observing an online exam using proctoring is better and more effective than observing a regular onsite exam in person,’ colleagues from the HSE Proctoring Centre say.
Students, on the other hand, voiced concerns about the software. They worried about technical malfunctions and the additional stress it could cause. HSE Vice Rector Sergey Roshchin addressed students’ concerns at a recent HSE Student Council meeting.
But if some time ago many students had no idea how online proctoring works, now the situation is much different. Uncertainty is behind us, detailed answers to frequently asked questions are available, and you can read instructions, as well as opinions and reviews from students who have gone through the process. The first students did trial testing on June 5.
Possibilities
The coronavirus pandemic and the University’s subsequent transition to distance learning showed the importance of having a proctoring procedure in place. But even after in-person instruction is resumed, it is clear that proctoring technology will continue to play an important role in education at HSE, and that this role will only increase with every year.
Alexey Chukarin, Senior Director for Digital Transformation
The HSE IT Office intends to embed proctoring tools in the SmartLMS educational platform that is being created. But we won’t stop there. It will be necessary to develop proctoring tools so that we can improve video analytics algorithms, take into account behavioral patterns, and automatize the proctor’s work as much as possible. Our immediate agenda includes summarizing the experience we have gained so far, as well as that of our domestic and international partners, so that we can jointly develop this important tool.
Alexey Chukarin