AI Will Revolutionize Education? But How?A Look at Early Experiments in Using AI as a Teacher

AUGUST 31, 2023

HUDSON HARPER, Education Data Strategist/Consultant

AI Feedback

Everywhere we turn, there's growing buzz around how AI might reshape education. Claims about AI personalizing learning experiences for teachers and students are enticing. Advocates suggest AI can lift the burden of mundane tasks like feedback provision from educators, enabling them to prioritize the classroom environment. Yet, a practical showcase of AI's application in a classroom at scale is hard to come by. With a sense of curiosity, I embarked on a journey this academic year to integrate AI into my teaching and assess its merits.

My AI Setup: A Glimpse

  • The spotlight in AI for education largely hovers over ChatGPT, with some attention given to Anthropic's Claude. My hesitance to use ChatGPT stems from ethical concerns. Using student data to further train models stands at odds with my school's privacy policy and my personal ethics. Furthermore, the process of manually inputting each student's response into ChatGPT, especially when giving feedback, felt cumbersome.
  • My programming background afforded me an alternative: direct access to OpenAI's GPT-3.5 and GPT-4 models via their API. This choice, although not available to every educator, offers enhanced data privacy. With OpenAI's API, there's a commitment to not repurpose data for model training, bolstered by its SOC 2 Type 2 compliance and a 30-day data retention policy. Those especially concerned can even request a zero-day retention policy from OpenAI. Using an API also gives me an opportunity to pre-process student submissions, anonymizing them as necessary.

The AI Feedback Test

I kicked off with a straightforward task: giving feedback on a brief survey. This survey assessed students' foundational understanding as they began the course I’m currently teaching. Ordinarily, I'd manually go through each, sometimes falling back on general comments during the busiest times of the year. For the AI test, I emphasized the need for personalized feedback and avoidance of generic comments. The results? Mixed. Here’s an example of an AI generated comment next to the type of feedback I typically give students.

AI Generated Feedback:

Great job completing the assignment and providing your understanding of the topics. To further develop your thinking, considering exploring different perspectives.Keep up the good work!

My Feedback:

Ah but what does it mean to be a citizen? Is it purely legal or are there societal expectations that come with it? Consider this as we go through the course. Also, it's interesting to consider if ethics can be extended beyond the individual. Consider this especially as we discuss codes of ethics. Are ethics intrinsic or extrinsic?

The AI indeed returned uniquely tailored feedback, but often, it didn't quite adhere to the personalized feedback directive. Despite the clear instructions, phrases like "Great job" slipped through. Also, the responses were relatively narrow in scope and didn’t show much insight. This led me to speculate:

  • Maybe OpenAI’s training data for such specific and effective feedback was too limited, affecting the quality of AI responses.
  • The AI's tendency to lean towards positivity, something that companies like OpenAI have purposefully put in place as a form of content moderation, might be a built-in measure to avoid negative outputs, but at times, it felt too generous and uncritical, diluting the feedback's impact.

Reflections

  • Capability vs. Current Reality: Today's AI, while impressive, might not be completely ready to handle feedback as crucial as being part of assessment. In fact, OpenAI even acknowledges in their own Usage Policy that using AI for assessment is inappropriate. And while GPT-4 promises improved output, its price point is a consideration.
  • Cost Implications: Providing feedback to 35 students via GPT-3.5 cost me $0.03. On the other hand, GPT-4, with its $0.03 per 1000 input tokens and $0.06 per 1000 output tokens, would have set me back by $0.60 for just this task. Project this over a year, and the costs quickly ramp up.
  • Privacy & Practicality: Safeguarding student data while efficiently using AI demands technical proficiency, which can be a barrier for many educators. As AI tools in education proliferate, questions about their affordability and data governance will emerge.
  • Value of Feedback: There's a gap between the rapid feedback AI offers and the depth and relevance it needs to achieve. However, I do think it’s worth noting that for some schools and students. According to data from surveys like the High School Survey for Student Engagement, there’s a disturbingly high number of students who don’t receive regular feedback from their teachers. It’s hard to argue for those students that AI feedback isn’t better than not receiving feedback at all.

Wrapping Up

My exploration into integrating AI in teaching has been enlightening, prompting me to reflect on my approaches. While the journey with AI continues, the learnings are invaluable. If this peek into my experience intrigued you, consider following my future blog posts or signing up for my newsletter by visiting my homepage, harpertech.io. Collaboration thrives on shared ideas, so if you've got any AI-in-education thoughts or experiments, drop them my way