Generate a custom quiz based on assignment submission

Generate a custom quiz based on assignment submission

di Marcus Green -
Numero di risposte: 5

I have been working on this idea which would use an external LLM to read the contents of a student assignment submission and generate a custom quiz using my AI Text question type. I have created some early code here

https://github.com/marcusgreen/moodle-local_jomot/blob/main/docs/userdocs.md

Would people be interested in developing the idea further?

In riposta a Marcus Green

Re: Generate a custom quiz based on assignment submission

di Michael Hughes -

Interesting.

From an org perspective, we might need/want an extra layer of active consent, to "read the students submission and submit to contents to an AI model", which we may want to ask either "once" (à la Turnitin) or "every time". 

In riposta a Marcus Green

Re: Generate a custom quiz based on assignment submission

di Julia Lee -
***Deutsch unten***

Hello Marcus,
I’m very interested in this, even though I’m not a programmer. However, I’ve looked closely at your qytype_aitext and developed ideas for its use in teaching. I’m therefore confident I can contribute the users’ perspectives effectively during development, particularly regarding use cases, usability and the legal framework for online exams in Germany.

I think it’s also worth looking into assessment principles, for example, whether it’s possible to train assessment systems for specific subjects and according to certain criteria such as objectivity, reliability and validity.
I could also take on a coordinating role, for example, in developing, planning and allocating work packages.
Just let me know if you need any support with the DEV Camp. But I’ll understand if you’d rather just get on with writing code first.

Best regards,

Julia

Hallo Marcus,
ich bin sehr interessiert daran, auch wenn ich keine Programmiererin bin. Ich habe mich aber eingehend mit deinem qytype_aitext beschäftigt und Ideen zum didaktischen Einsatz entwickelt. Daher kann ich sicher die Perspektiver der User*innen gut einbringen bei der Entwicklung in Bezug auf Use Cases sowie zur Usability und dem rechtlichen Rahmen für Online-Prüfungen in Deutschland.
Ich denke, es lohnt sich auch einen Blick auf Bewertungsgrundlagen zu werfen, z.B. ob es möglich ist für spezifische Fächer Bewertungen zu trainieren und nach bestimmten Kriterien wie Objektivität, Reliabilität und Validität.
Ich könnte auch moderative Funktionen übernehmen, z.B. beim Entwickeln, Planen und Verteilen von Arbeitspaketen.
Sag einfach Bescheid, wenn du Unterstützung beim DEV Camp brauchst. Ich habe aber auch Verständnis, wenn du vor allem erst Mal nur Code schreiben möchtest.

Herzliche Grüße
Julia
In riposta a Julia Lee

Re: Generate a custom quiz based on assignment submission

di Marcus Green -
"I think it’s also worth looking into assessment principles, for example, whether it’s possible to train assessment systems for specific subjects and according to certain criteria such as objectivity, reliability and validity."

I have been toying with the idea of having a bank of AI Text questions on various subjects and a way of submitting them to varying models to get some idea of those issues you mention. Despite my interest in this area I always advise great caution when using any AI/LLM in that the results should not be used to influence "big decisions".

I devoted quite a significant part of my presentation at MoodleMoot to these issues
https://www.youtube.com/watch?v=dlVEGQlOvW0
WIth
WIth AI Text it is possible to have a very custom prompt to focus on the small amount of text that is typically submitted. With Assignment size text it gets harder to have such a focus in the prompt.