Understanding What Students are Doing: An Internal Combustion Engine of eLearning

sloan-c.png
Presenter:John Kaliski (Minnesota State University)

John’s purpose of his research is to capture some student behavior from their Learning Management System (LMS).

  • How often and for how long students log into the system
  • When students access reading material and notes
  • When students start online assignments
  • How long students take to complete assignments
  • How productive discussion forums are.

John proposes to dramatically expand monitoring of student learning behaviors online. He wants to offer a huge suite of tools to make raw data more useful (statistics, data mining, business intelligence). Two systems are already commercially available.

He wants a learning environment that is adaptive, but doesn’t have to adapt to all 200+ students at the same time.

Why “Internal Combustion Engine” as the title of the presentation? It is a somewhat loaded term that has both positive and negative implications for society. Likewise, introducing new architecture and technology for student behavior monitoring has both positives and negatives. Positives include mass customization, adaptive learning environments, large classes, improved retention, and assured learning reporting automation. Negatives include perceived (and actual) invasions of privacy–students find it creepy–as well as the fact that there are unintended consequences. Ethics of using such a tool are somewhat unclear.

LMS systems today give a core dump of raw data with no real analysis built in.

John’s new tool includes the same raw material found in LMS data. However, he also borrows ideas from Google Analytics and other tools. Data is collected at the event level–keystrokes and mouse activity. It also tracks hyperlink activity and how much of the content is viewable on the screen as well as how long it is on the screen. Huge amounts of data are collected–200,000+ records from 400 users in one month.

Raw data by itself is meaningless without context. The instructor communicates with the system what the expectation of the learner is. It measures for the alignment of learner activity and instructor expectations. For instance, when the instructor is working on a course objective, student traffic patterns on the online course components should increase. This system refines itself over time, especially for instructors that teach the same course repeatedly over time.