https://doi.org/10.36866/pn.130.20

Dr Dawn Davies
Associate Professor, School of Physiology, Pharmacology and Neuroscience, University of Bristol, UK

The potential for artificial intelligence (AI)- generated answers submitted by students for assessment has created a lot of anxiety within the academic community. When ChatGPT was released at the end of 2022 we started to see many articles in the educational media stating that it was the end of assessment in higher education as we know it and that we must rethink the way that we assess students (Bagshaw, 2022).

The challenges of AI to student assessment

There is no doubt that AI, such as ChatGPT, poses a challenge to assessment for and of learning and that it will become more sophisticated with time. Currently, our own testing of ChatGPT suggests that it is good but not great at answering with the combination of breadth, depth and critical awareness that we look for in good answers to essay questions. But even as I write this, I know that there will be more iterations with greater ability to approach complexity by the time this is published.

Rethinking assessments

However, as the initial panic subsides, it is clear that AI offers opportunities to rethink assessments (Chowdhury and Wamba, 2023). But what are these opportunities and how do we turn them into sustainable activities for our students? One potential route is to use AI output to focus on reliability of sources and critical evaluation and thereby support the development of transferrable skills in our students. In a world of information overload, it is crucial to teach students at the earliest opportunity about reliable sources and in doing so help them to think critically about the information they are gathering (Wiley et al., 2009).

What is a reliable source?

Currently, we know that AI-generated content can be unreliable, containing significant inaccuracies and can create false references (Lo, 2023). This provides a topical and relevant platform for engaging students in the use of reliable sources and general fact checking. As an example, we are in the process of developing an activity aimed at first-year students where students assess the validity of a piece of writing that has been generated by AI.

In this case, we are focusing on two factors. Firstly, factual accuracy. Is the piece accurate overall and does it contain any specific factual inaccuracies? Here we engage students with the concept of reliable sources of information and how to fact check. Secondly, validity of any references cited. Can the students find the reference, is it the correct reference for the topic and can they find the information within the reference that is being stated in the piece? In this way we are encouraging students to engage with the primary literature in a structured way early in their programme. They are directed to find a specific paper and to look for specific content. This activity also highlights the limitations of AI. It strikes a note of caution for students regarding factual inaccuracies and irrelevant or even non-existent references that occur in AI-generated text, hopefully discouraging the wholesale outsourcing of their intellectual effort to AI.

Regardless of how much we are required to rethink our assessments to ensure that they remain relevant and robust in the age of AI, it is vital that we bear in mind that our graduates will be entering into a world of employment where these tools are widely used. It should be part of our role as educators to equip them to use AI effectively and to demonstrate the added value that they can deliver. It is important now that we engage with AI, assess how it can support education including skills development and disseminate what works and what doesn’t work within the education community.

*No AI was used in generating this article, it’s all my own work!

References

Bagshaw J (2022). What implications does ChatGPT have for assessment? Wonkhe. Available at: https://wonkhe.com/blogs/what-implications-does-chatgpt-have-forassessment/. [Accessed 08/05/23]

Chowdhury S, Wamba SF (2023). How can we teach and assess with ChatGPT? Times Higher Education. Available at: https://www.timeshighereducation.com/campus/ how-can-we-teach-and-assess-chatgpt. [Accessed 08/05/23].

Lo CK (2023). What is the impact of ChatGPT on education? A Rapid Review of the Literature. Education Sciences 13, 410. https://doi.org/10.3390/educsci13040410

Wiley J et al. (2009). Source evaluation, comprehension, and learning in internet science inquiry tasks. American Educational Research Journal 46(4), 1060–1106. https://doi.org/10.3102/0002831209333183