Decoding ChatGPT - Students Weigh in on AI’s Role in Learning
Exploring student experiences using ChatGPT at the Creative Computing Institute, University of the Arts, London.
Recently, the academic sphere has been consumed by the implications of ChatGPT. The discourse frequently gravitates toward the potential for academic misconduct1, embodying an attitude that presupposes the power of scholars to dictate how students engage or disengage with these technological tools. Yet, within this dialogue lies an opportunity to examine student interactions with large language models and discern pedagogical approaches that foster collaboration with both their peers and with AI.
Though the ghost labour, racial, disability, gender, and climate injustices ingrained in AI are genuine and substantial concerns, we cannot deny that our future will be interwoven with big AI. It is our responsibility to comprehend the utility of large AI models and facilitate conversations rather than suppress them. At the Creative Computing Institute (CCI) at the University of the Arts London (UAL) this term, the MSc class on Computational Thinking delved into the ethical complexities of big tech while acknowledging the need for mindful responses to the rapid evolution of technology. Fluid engagement and exploration of new tools enhance student employability and equip students with the necessary skills and imagination to create more ethical alternatives.
Embracing paradoxical thinking enables us to develop multi-layered understandings and envision alternatives, often originating as science fiction and prototypes before materialising as science fact, careers, and healing forces within our culture. This term, our critical examination of broader social issues within AI went hand in hand with our engagement with OpenAI's ChatGPT to comprehend its workings and our connection to it. What advantages and disadvantages does it present in a learning context? The students have articulated their thoughts – it is time we lend our ears to their voices.
Methodology:
This study was run within the University of the Arts London Creative Computing Institute during the winter term of 2023. 17 participants filled out a questionnaire of five free-text questions designed to capture their experiences with the tool. The students, staff, and members of our community were recruited via slack and in class to document their ways of working with chatGPT during the term. The data was analyzed using Thematic Analysis, which served to cluster the results into four sections: creative practice, multilingual translation, computational education, and employability. The average usage frequency of ChatGPT among these users is approximately 3.79 times per week.
Questions:
1. Do you wish to use your name if I quote you or stay anonymous? Put your name you'd like me to use or anon here.
2. What are your pronouns?
3. How are you using chatGPT at CCI for your course work?
4. How often do you find yourself using it?
5. What are you finding its most surprising use?
Empowering Creativity: ChatGPT in Meaning-Making and Creative Pursuits
In the initial phase of creative inquiry, some of our students are employing ChatGPT to generate inspiration. By feeding concepts into the language model, they uncover connections, themes, and new perspectives that ignite their creative sparks. ChatGPT becomes a tool for exploration, as users sift through its output to find inspiration and expand their understanding of topics. Jenny said, “I take some inspiration from ChatGPT before I need to do anything creative.”
ChatGPT is also helping these students make sense of their writing and creative expressions. From assisting in the analysis of emotional themes in poetry to providing reflective insight on science fiction, the language model enables users to refine their ideas and articulate them more effectively. Zewen Xue said, “I wrote a poem two days ago, but I didn't understand why I wrote it that way. I asked GPT to help me explain my emotional state, and unexpectedly it answered very accurately.” .
Matt Jarvis, a post graduate certificate student and member of our technical team, for example, discovered the impact of seeing their own words reordered and organized by category by ChatGPT. The mere act of reorganization can lead to new understandings. Jen Sykes, an Lecturer in Diploma Creative Computing and PhD student, said, “Is still very beneficial to see how the same words can be re-ordered and immediately make much more sense.”
Mae Horak observes that the tool eliminates friction in her creative methods by allowing her to stay within the "notes" or "getting ideas down" stage. By generating lower-level sentences based on her rough notes, ChatGPT frees up time for her to focus on more intricate creative problem-solving and thinking.
Embracing Linguistic Diversity: How ChatGPT Supports My Multilingual Classroom
In the ever-evolving landscape of education, classrooms are becoming increasingly multilingual. This shift brings about unique challenges for both educators and students alike. Fortunately, ChatGPT is supporting multilingual learning environments, enabling me to create more inclusive and accessible resources for my students.
In class, I have used ChatGPT to create glossaries in situ for terms and vocabulary specific to software and coding tools. By refining the generated content, we can then translate the list into multiple languages and share it with our students. The feedback on these translations has been overwhelmingly positive, with one student remarking, "It is most of the time better than professional translations. The level of exploration of expertise is deep and wide."
Students are turning to it to understand their homework. In a coding unit, students are asked to do and create Python Challenges. Python Challenges are small coding puzzles students need to solve. Here Ziqing Xu uses ChatGPT to explain the idea first in English then again in Chinese. Seeing the explanation in multiple languages helps facilitate better comprehension for people who are multilingual speakers.
While code behaviour is often language-agnostic, the natural language involved in writing code (e.g., naming variables, functions, and classes) is predominantly in English. This can be a barrier
or students who are more comfortable with other languages. To address this issue, our students are using ChatGPT to translate code documentation (i.e., the non-code text within code files and projects that explains how code works) into their preferred languages, making it more accessible and easier to understand. Students report that seeing the translation improves their English computational literacy and teaches them new vocabulary. By facilitating translations of code documentation into any language with superior results to Google Translate, ChatGPT could actively break the hegemony of English in the realm of documentation. A point to stress is the importance of a student asking, in another language, for a large language model to explain an idea to them in their native language. This is what separates it from Google Translate. Google Translate doesn’t explain concepts to students in natural language. It functions as a multi-language translation solely.
Decoding the Learning Experience: ChatGPT's Impact on Programming Education
Learning to code can be an arduous journey, filled with complex concepts and intricate problem-solving. New developers give feedback they are turning to it in favor of search engines before asking tutors for assistance. They prefer it precisely because they can ask it to explain concepts using natural language in ways that make more sense to them verses searching for abstract concepts. One example comes from Ziqing Xu’s creative prompt engineering to ask it to explain Object Oriented Programming as a metaphor.
In our unit Coding Two, navigating the complexities of audiovisual programming can be challenging, even for the most experienced developers. Complex signal flows (e.g., getting audio, video, or sensor input from the world or from a file, and analysing or applying effects to these) are not intuitive and often take multiple attempts from even an experienced developer to understand. ChatGPT can step into the place of what previously would have been an internet search as a first port of call to understand complex computational ideas. For instance, Mae prompted it with, "I can't figure out how to make two oscillators play on top of each other using Open Framework and Maximilian" (two tools used for creating visuals and audio in C++ programs). It immediately helped her to understand the ideas because the answer came in human readable language. In such cases, ChatGPT can step in as a virtual tutor, offering support and guidance in a way that was previously only possible through internet searches or human interaction with a tutor. Jinglin Zhang said, “It is like an intelligent teacher online. It can write the source code through my keywords. As I refine my keywords, it gives me the answers I want based on my thoughts. I think sometimes this is more convenient than looking up information on the Internet or looking for answers on forums, because the answers on the Internet are sometimes just similar to my problems, but not exactly the same, so the corresponding solutions may not necessarily apply to my difficulties.”
Debugging ChatGPT’s sometimes incomplete or non-functional code requires from their side an integration of understanding and improves debugging skills, which are paramount to success as a developer. Often ChatGPT will output incomplete answers or reference variables which are not declared or functions that do not exist. The process of editing and filling in the blanks in the the code is akin to the common coding exercises in Daniel Shiffman’s Learning Processing2 where he offers incomplete code snippets learners must complete themselves.
This fill in the gap style learning re-enforces their prior knowledge. Mae is using it to understand error messages. "When I get an error message, I don't understand chatGPT is great at helping to figure out what is wrong. It will type out a list of potential problems that may be causing that error message and thine I can work down the list to try and sort out which one it is."
In many cases, even professional developers may create code for a new project by cobbling together pieces of existing code from multiple tutorials and websites. However, when one does not fully understand how these existing code snippets function, this can lead to errors. ChatGPT fills the role of a helpful tutor, offering explanations and assisting in the documentation of code. As students use ChatGPT to write natural-language summaries of what pieces of code are doing, they not only improve their understanding but also make their work more accessible to their peers, fostering a culture of collaborative learning. One student commented, "It can write a readme [file] for the code, which helps me to understand the code." By generating documentation, they are not only improving their own understanding but laying the groundwork for future developers to be able to use and share their code, contributing it back into the collective student commons within CCI. Additionally, they have found this same approach useful for asking ChatGPT to document its own code. By taking the same approach and applying it back towards generated code, there is a potential to improve their understanding.
In summary, ChatGPT is reshaping the programming experience by offering personalized guidance, enhancing understanding, and fostering collaboration among learners. By stepping into various roles – tutor, debugger, explainer, and documentation assistant – ChatGPT can change the way students learn and write code, making it a more engaging, enlightening, and ultimately, rewarding experience.
HustleGPT: Students' Edge in the Competitive Job Market
Students are increasingly turning to ChatGPT for assistance in crafting compelling cover letters, completing job applications, and pursuing employment and business opportunities. Gaining expertise in prompt engineering gives them a significant advantage in the competitive job market. Through the length of this study, students not only learned to work as prompt engineers but also developed a deeper understanding of how to harness the power of natural language models. As a result, integrating ChatGPT into education proves to be a learning experience with real world pay off.
Learning from Limitations: Students' Insights on ChatGPT's Shortcomings
Despite the benefits of ChatGPT, students encountered challenges and expressed concerns that shed light on their unique needs. Multiple students commented on the challenge of learning to write prompts that would yield results they were seeking. Students also reported issues with ChatGPT generating false citations and unreliable information, prompting them to verify its assertions independently. Developing digital literacy around the validity of facts and figures in large language models will be an academic skill for the future. Developing software to fact check outputs of these systems is an emerging field.
Moreover, the lack of visual aids and diagrams in ChatGPT's explanations highlights its limitations in catering to diverse learners. Some students found the code generated by ChatGPT to be overly complex for beginners, suggesting the need for a more adaptable teaching approach. Others wished for more output options, feeling constrained by the model's single-solution responses. For example, asking ChatGPT how to draw a circle with JavaScript could generate multiple possible solutions for learners to select from and then it could elaborate on the selected answer.
Furthermore, students expressed a keen interest in exploring the ethical implications of working with big technology and expressed a desire to investigate alternative AI tools. These same students and educators would be well situated for a possible University Advisory group on ethical AI in the classroom. Academic Support at UAL might be well positioned to offer students guidance on these issues University wide with a set of best practices or a hands-on workshops. At the CCI on the MSc in Creative Computing, this student curiosity aligns well with the content we have planned for next term, which focuses on teaching students to create and work with a variety of AI technologies which propose more ethical alternatives and use their own datasets. My aim when writing technical units for the CCI on the BSc and MSc in Creative Computing and the MA in Internet Equalities was to lay the groundwork for a place where students could not only question the ethics of these systems but then be empowered to go out into the world and do something to improve them.
Navigating the Future of AI in Education
As the academic sphere grapples with the implications of ChatGPT, it is essential to listen to students' experiences with it and explore if it is useful in enhancing learning. Addressing the concerns and challenges students face with ChatGPT, we can refine its applications to better suit their unique needs and create a more inclusive educational experience.
As we move forward in the rapidly evolving world of AI, we must recognise the importance of engaging with these powerful tools and their ethical implications, while also empowering students to explore alternatives. By integrating ChatGPT into creative and technical teaching, as well as into multilingual classrooms, we can help students develop the skills needed to navigate and contribute to a future interwoven with AI. However, just because ChatGPT is an easy tool to access, does not mean its the correct tool for the classroom. Similar and more ethical tools such as Standford’s Alpaca are newly released options that offer much of the functionality while protecting user privacy. Alpaca also allows for the custom creation of datasets by its users, which could offer powerful opportunities for the CCI to generate its own coding datasets lecturers could deploy in the classroom. Technologist and UKRI must continue to invest in making open-source Machine Learning and AI tools and developing ethical AI educational centers, as we continue to do at the CCI with tools like InteractML, and Wekinator.
At the same time, it is vital to acknowledge and address the limitations of ChatGPT, such as its inability to cater to diverse learners and provide consistently accurate information. By embracing the paradox, we foster critical examination of broader social issues within AI and enable students to envision alternative approaches. By doing so, we help them become not only successful in their careers but also responsible contributors to the ongoing evolution of AI technologies.
References
Debby R. E. Cotton, Peter A. Cotton & J. Reuben Shipway. 2023. Chatting and cheating: Ensuring academic integrity in the era of ChatGPT, Innovations in Education and Teaching International, DOI: 10.1080/14703297.2023.2190148
Daniel Shiffman. 2008. Learning Processing: A Beginner's Guide to Programming Images, Animation, and Interaction. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.
Loved reading this. Thorough, interesting and balanced, unlike other scaremongering or frothily excitable articles out there. More writing like this please! From people in your position (deep tech and social understanding in an educational context) writing about things like this (young people using technology in education)