Avoid giving credit for marks beyond the last question in answer key

[gutow]

A common problem with student entries on bubble sheets is for them to mark beyond the end of the number of questions on the exam. This is either because they get offset when filling in the bubble or because they are aware that some OMR software can be fooled into giving them credit for marks beyond the number of questions.

This pull request gives no credit for bubbles beyond the last answer in the answer key and flags them by putting the bubbled in character at the end of the list of 0 and 1 used to record scores.

This also adds a test of this to the e2e tests.

[iansan5653]

Hello, I'm not sure I understand why this change is needed. The zip function will drop extra elements from the longer list:

Suppose, two iterables are passed to zip(); one iterable containing three and other containing five elements. Then, the returned iterator will contain three tuples. It's because the iterator stops when the shortest iterable is exhausted.

So if the answers list is longer than the keys list in this code, the extra answers will be ignored:

open-mcr/src/scoring.py

Line 91 in ce8cd14

for actual, correct in zip(exam[answers_start_index:], key)

Were you finding that the software was giving points for extra answers beyond the key length? If so, that would be a surprising bug.

[iansan5653]

Actually I can see from the examples that the output was incorrect sometimes. That's definitely a bug I'd like to fix! I'm not totally sure about flagging the answers by putting them at the end of the list, but I'll take a closer look at this and get it reviewed as soon as I can.

[gutow]

Listing the extra answers does not impact the scoring and makes it easy for an instructor reviewing the data to see that there is a problem with that scan sheet and see what the issue is. As an instructor handling spreadsheets with hundreds of these it is nice to see what answers ended up shifted beyond the end. I can then quickly judge if there is an offset or other problems.

An alternative is to flag the score in the column where you warn about missing answer keys.

A combination of the two may be the best.