Get with the program – AI-assisted coding is here to stay

Aalto’s computing education researchers are putting large language models to the test and the results are true.
AI-generated colourful images of robot-looking figures and humans programming.
Illustrations generated by using DALL-E 2 with the prompt "a colorful illustration of AI-assisted programming in hyperrealist style, digital art". Credit: Aalto University

Generative AI has hit the public imagination in full force during 2022. Perhaps the biggest splash was made by OpenAI’s launch of the text-to-image generator DALL-E 2 with its stunning illustrations.

Under the guise of generative AI art, code-completion programs are boosting developer productivity by automating repetitive and mundane programming tasks. The world’s largest source code host GitHub released its code-completion tool called Copilot in June 2022. It is trained on 45 terabytes of coding data from the GitHub code repository and runs on OpenAI’s Codex model. The company claims the tool can automate around 40 per cent of a developer’s tasks.

‘Copilot has a lot of potential, but also a risk of creating over-reliance on AI-generated code, if the user doesn't check its outputs properly,’ says Juho Leinonen, postdoctoral researcher at Aalto University’s Department of Computer Science, whose research is focused on educational technology and AI in education. ‘The risk is particularly high for students and beginners.’

GitHub made the tool freely available for students from the get-go, in contrast to the 10-euro monthly subscription price for developers. The decision raised questions about whether students could use the tool for cheating.

‘The question we should be asking is how do we rethink computing education to incorporate these tools?’ says Leinonen. ‘This is just the beginning.’

Leinonen and his colleagues took up the challenge and studied the question from a teacher’s perspective in their recent research article on Codex.

‘We examined how teachers can use Codex to automatically create new programming exercises and natural language explanations for code,’ says Arto Hellas, senior university lecturer at the Department of Computer Science. ‘Regardless of some small issues, our exploration with Codex showed remarkable results in creating novel exercises and code explanations. We were even able to generate exercises around a specific topic, like basketball.’

Among beginners and students, the need for programming exercises is huge. Being able to generate sufficiently accurate exercises and natural language explanations would be a major help for teachers. The research article received the best paper award at the ICER 2022 conference, which is the main research conference for the computing education community.

‘Although the generated exercises and explanations need to be verified by a human, the performance surpassed the research community’s expectations,’ says Leinonen. ‘The researchers and teachers in our field whom we’ve spoken with, are excited about the opportunities yet anxious about the challenges such as over-reliance.’

The results didn’t go unnoticed by GitHub either. The company decided to make Copilot free for teachers shortly after the ICER conference, referencing the team’s research in its announcement.

The impact of large language models on computing education is still an emerging topic and research can be hard to come by. Leinonen, Hellas and their colleague Sami Sarsa have been busy with answering all the collaboration requests that their award-winning article generated. Moreover, the trio is digging deeper into the possibilities that the new frontier may hold.

‘Due to the rapid development of the field, it is hard to predict exactly how large language models will reshape computing education,’ says Leinonen. ‘Two things are for certain – AI-assisted coding is here to stay and we’re here to figure it out.’

Education technology used in class room, symbolic illustration by Matti Ahlgren
Department of Computer Science
Illustration of a person holding virtual reality headset

Department of Computer Science

To foster future science and society.

Arto Hellas

Senior University Lecturer

Juho Leinonen

Sami Sarsa

Doctoral Researcher
  • Published:
  • Updated:
URL copied!

Read more news

ANI Aalto logo
Campus, Research & Art Published:

ANI Open House on 15th of June 2023 at 12:00-15:00

Welcome to ANI Open House to celebrate 10th anniversary of Aalto NeuroImaging Infrastructure. Doors are open 12:00-15:00 on 15th of June 2023.
A closeup of a woman in green light wearing goggles that reflect colourful pixels
Research & Art Published:

In September, 15 new Academy Researcher Fellows will start at Aalto

The Academy of Finland has granted Aalto University funding for 15 Academy Research Fellowships
On the background, white radiant lines over a black bacground and only hair and shoulder of a person passing by visible
Research & Art Published:

Nearly 20 million euros for research

A total of 45 researchers received Academy Research Fellow and Academy Project funding from the Academy of Finland
Outi Turpeinen standing on stage talking about Unfolding public art book to a seated audience facing her.
Campus, Research & Art, University Published:

Love and poetry - the artists were inspired by the passion conveyed by the university's research

Engineering Materials, an art collection for K1, K2, and K3 buildings, was published