Session Dokumentation / Documentation

Cool. Thank you, Christine.

Happy moodling! ;)

Thu 15:00 02A

Antonia Bonaccorso, Franziska Klarer | ETH Zürich (EN):

An advanced quiz report

Please also compare slides added to the forum post

Idea: Every teacher can easily check the quality of questions - no running code yet, these are the specs needed.

Userstory:

• Bea Basic, few exam, needs a quick and efficient look on the analysis
• Ann Advanced, many exams, subsets of questions, wants more information
• Eva Expert, many exams, in depth statistical Analysis
• The plugin only supports closed question types. Will be a new entry in the dropdown menu of the quiz reports "Analysis fo questions"

4 sections

the quiz report page "Analysis of questions" consists of the following four sections

• analysis settings - the filter
• download data - the export
• scatterplot of questions - facility vs. discrimination index
• analysis of questions and responses - the table

1. filter
You can create groups of your students for your analysis of the results.

New core feature: Grade item (chunk of questions) - available since 4.4; MDL-74610 - you may restrict the report to one grade item at a time

With the ruler "include attempts from---to" you can restrict the report to a certain range of student's grades. It is e.g. possible to exclude outliers

2. export

export of comments:
Temporary solution right now: ungraded essay question at the end of the quiz to receive feedback/ comments on the questions.

3. scatterplot
plots the facility index vs discrimination index. The closer the question is to the line, the higher the chance is that the question is bad. gives a general good overview of questions that need review

4. Analysis table:

Question numbers. name, Grade item, attempts, facility and discrimination indexes, Review progress (you can have more than one status on a question, "reviewed" and "in process"), First line is all the info to the questions, it is by default collapsed so if you need more details, you can expand it.

Model response (options of the questions): options are shown with Partial credit, Count, Frequency and there is a new menu item for the Frequency graph --> blue: correct answer, yellow: false answer, not answered: black.

State of the project (05.09.2024)
Specifications are ready
Waiting for quotes
Still open for suggestions
Planned Moodle version: 4.4 or 4.5
Planned start of development: october 24
Planned not as core, but as a Thirs party plugin

Ideas/ suggestions/ questions by the audience

include question mark icons with additional help text for "Grade item", "Facility index", "Discrimination index".
Provide a better wording for "Model response" (can be done via AMOS)
Be aware of accessibility guidelines when choosing colours.
Use icons (e.g. stars) instead of colours.
Results are based on one specific quiz activity in one course.
Suggestion: Check the custom reports tool for inspiration.
Usage of AI to disentangle the reports with pre-defined prompts may not be the best solution (yet?).
The scatterplot is very helpful.