By Conor Fahy and Simon Coupland, De Montfort University

Understanding how students use generative AI

To better understand how novice programmers engage with generative AI, a pilot study was conducted with undergraduate students on the BSc Computer Games Programming programme at De Montfort University. Students were given a programming task, one hour, and unrestricted access to generative AI. The intention was to observe how learners incorporate an LLM into their problem-solving process. A pre-session discussion revealed that most students rarely used AI in their studies. The reasons were notable: concerns that AI diminished creativity, fears of deskilling, uncertainty surrounding academic integrity, and a broader question of identity. If a student’s identity is built around being a good programmer, what does it mean when an LLM is better at programming than they are?

We expected students to use the LLM in a similar way to how we use it: dialogically and iteratively with the LLM acting as a tutor or a knowledgeable peer. We anticipated learners would decompose the problem, request explanations, relate new concepts to existing knowledge, ask for minimal working examples, and gradually construct a solution through conversation. Instead, a very different pattern emerged. Many students initially waited for tutor guidance before engaging with the task. Once they turned to the LLM, the dominant behaviour was to copy-and-paste the entire task description and request a complete solution. The generated solution was not interrogated or adapted and often became a barrier to further learning, with students unsure how to proceed with code they neither understood nor had contributed to. When encouraged to break the problem into smaller parts, some students abandoned the LLM altogether and began to seek tutorial videos on the more familiar (and passive) YouTube.

Rethinking how we teach programming with reverse Bloom

In hindsight, our expectations reflected our own expertise rather than that of novice programmers. Experienced developers rarely ask an LLM to solve an entire problem. Instead, they use it to clarify concepts, challenge assumptions, explain unfamiliar ideas, and iteratively refine partial solutions. This interaction relies on metacognitive skills that experts have developed over years of practice.

This observation aligns with recent discussions around reversing Bloom’s Taxonomy (see Figure 1) for AI-supported programming education [1] (and in wider HE [2]).  Traditionally, learning progresses from remembering and understanding through application and analysis before culminating in the synthesis of a working program. LLMs disrupt this sequence by allowing students to begin with synthesis. The proposed alternative is to present students with a complete program, ask them to evaluate and reverse engineer it and improve it through prompting. As a result, learners understand and remember the underlying concepts.

Figure 1: The Inversion of Bloom’s Taxonomy (adapted from [1]).

However, the cognitive leap from generating a complete program to critically evaluating it may be substantially larger than the equivalent transition in traditional Bloom. Programming has long relied on productive cognitive struggle [3]: writing code that fails, identifying faulty internal logic, debugging, iterating, and gradually refining mental models. A completed program is evidence that this learning process has occurred. In reverse Bloom where does learning happen? Presumably, it occurs during the process of understanding, critiquing and modifying a program the learner did not write. Does that produce an equally good mental model? That remains an open question.

Supporting learning in the age of AI

Our observations also suggest that educators cannot assume students naturally know how to learn effectively with AI. Skills such as decomposing problems, recognising knowledge gaps, engaging in productive dialogue with an LLM, and critically evaluating generated solutions are themselves learnable and may require explicit instruction. Developing these skills is, in essence, developing learners’ metacognition [4].

For decades the argument was that students learn programming by writing programs. Now we are forced to consider if students can learn programming just as effectively by understanding programs they never wrote.

References

[1] Pesovski, I., Vorkel, D. and Trajkovik, V., 2024, November. AI-Powered Education: Rethinking the Way Programming is Taught Using AI Tools and Reversed Bloom’s Taxonomy. In 2024 21st International Conference on Information Technology Based Higher Education and Training (ITHET) (pp. 1-6). IEEE.

[2] Shelley, A.W., 2021. Reverse Bloom: A new hybrid approach to experiential learning for a new world. Journal of Education, Innovation and Communication, 2(2), pp.30-45.

[3] Young, J.R., Bevan, D. and Sanders, M., 2024. How Productive Is the Productive Struggle? Lessons Learned from a Scoping Review. International Journal of Education in Mathematics, Science and Technology, 12(2), pp.470-495.

[4] Rivas, S.F., Saiz, C. and Ossa, C., 2022. Metacognitive strategies and development of critical thinking in higher education. Frontiers in psychology, 13, p.913219.