Education · 20 Oct 2022 · 8 min

How a unit's exam papers get marked

Three hundred students, five markers, one question each — and the procedures that exist so the mark a student gets does not depend on who read the paper.

How fair is the marking process for exams? It is a fair question, and it is asked every semester. A unit with more than 300 students cannot be marked by instinct: it needs a written process, and this post describes the one I worked inside.

Two disclaimers first. These are my own opinions and I am not representing the University in any way; I am describing how it was done based on my experience. And this is about paper-based exams, which is what it was like before e-exams. E-exams have simplified most of this to the point where the human-error parts below barely exist.

I worked as a sessional lecturer and teaching associate from 2016, and across units in the faculty the process was almost always the same. What varies is the size of the class and the people doing it.

The marking process

Marking starts as soon as the teaching team has the papers — usually a few days to a week before the teaching period ends, not after. If the university has overseas campuses or distance learners, the papers are all marked in one location without exception. That is deliberate: it is the only way marking quality can be the same across campuses.

Take a unit with 300 students and five teaching associates. The paper has several questions, and the questions are divided among the team:

• Teaching Associate A marks questions 1 and 2
• Teaching Associate B marks questions 3 and 4
• Teaching Associate C marks questions 5 and 6
• and so on

One marker per question, never several. That is the rule that makes the rest of the process work. Markers are usually PhD students, sometimes people from industry working in the field, and the allocation matters: in a world where knowledge is a search away, a student's answer can legitimately be better than the model answer, and you want the person marking it to be the person who can recognise that.

Every question also has a marking rubric setting out what each answer earns. Building it is the hard part, because an answer and its justification can sit anywhere within a range of marks. The rubric is written by whoever set the paper together with the teaching team, and it can be updated during marking when the team accepts an answer that the rubric did not anticipate.

Which brings us to the sharp edge:

• Each question in the exam goes to a single marker, no matter how many papers there are — even at 500 students.
• The rubric written by the team is what the answers are measured against, and it can be amended when a legitimate answer was not anticipated.
• If the rubric turns out to be inconsistent or wrong, every paper has to be remarked to keep the process fair.

There is also a policy about pen colour: the first marking is done in one colour, so that a remark is visibly in another.

Is my paper anonymous?

Papers arrive in numbered bundles of ten or twenty, and each bundle has a list of student IDs and names on top. So: anonymous to a certain extent. We know the student ID, and the ID maps to a name if anyone wanted to look. In practice nobody does, because looking a student up is tedious and pointless — knowing the name changes nothing about the mark.

Don't spill coffee on my exam papers

Papers stay at the university. Members of the teaching team and the professor have lost bundles before, and a lost bundle means an exam resit for whoever was in it. Handling and storage are the university's problem, on purpose.

Then the marks move

A question is marked at the bottom of the question itself, and then the marks are transferred to the front page of the paper, into the boxes. That transfer is one of the places a mistake can creep in, so it is checked — by somebody else:

• Teaching Associate B checks questions 1 and 2, confirming the marks were added up and transferred to the front page correctly
• Teaching Associate A checks questions 2 and 3, and so on

A question usually has several sub-questions, so this counter-check exists because arithmetic and transcription errors are common enough to expect rather than exceptional. Marking a whole cohort usually takes under a week, and it is not finished when the last tick is made.

Once the front pages are confirmed, the numbers go into a spreadsheet — Excel or Google Sheets — by bundle number. Then a different person again transfers the front-page marks into the sheet, which is the last confirmation before the sheet becomes the student's exam mark.

Three separate people have now touched each mark: the marker, the counter-checker, and the person who typed it.

Reading the cohort, not just the student

With every question's marks in the sheet, the team looks at how the cohort performed question by question. This is where exam marking becomes interesting, because a question can be wrong.

Imagine every student scoring zero on one question. That is not a cohort of failures; that is a broken question. When that happens, the team and the chief examiner have three remedies:

• Remove it and keep the marks awarded. The paper's total can now be less than 100 — say 90 — while still allowing some students to score above 90 on the remaining questions.
• Make it a bonus question and give everyone full marks. Unpopular, and reasonably so: the students who spent time on it feel their effort was not recognised.
• Leave it and scale the whole paper at the end.

Which one gets used depends on what the chief examiner thinks the question was supposed to be doing. Each question is written to cover a learning objective in the unit guide, so a question that failed to do that is a question about the assessment, not about the students.

A question can also perform badly for a boring reason: the marker took a strict reading of the rubric. That is not a problem with the question, and it is still consistent, because every answer to that question was marked by that one person.

One quiet benefit of doing this in Sheets or Excel: there is an audit trail of every change, with who made it.

Remarking

Now the tedious part. The sheet shows the failures and the near passes. Depending on the faculty, P is a pass at 50 and above, and below that is NA.

The borderline papers are re-marked — typically by the lead teaching associate or the person who set the paper — against the enhanced rubric, in a different coloured pen so the remark is visible on the page.

What surprises people from outside is how often a remark changes nothing. It is very common. The reason is structural: marks are awarded against a rubric, so finding a student one extra mark to get them over the line would mean every answer to that question had been marked unfairly, and every paper would have to be remarked. It cannot just happen for one student.

The whole process

Summary

I think the process is fair and consistent, and the reason is structural rather than moral: it is built around one marker per question and a rubric, so the same answer earns the same mark regardless of who reads it. High achievers stand out clearly, because their marks are consistent across every question — several markers independently agreed the answers were good.

The same structure is why a near pass almost never turns into a pass on remark. One mark given outside the rubric would invalidate the marking of everybody else who answered that question, and no teaching team can justify that for one student.

So exam marking is treated seriously, with a lot of process, and whether it is actually fair comes down to whether the team followed the procedure that was laid out. When I have seen it done properly, it was.

And for anyone who was going to ask anyway: does the good-looking person always perform better? No. Strictly no — not when the process above is followed.