## Candace Thille and Joel Smith about the OLI in Carnegie Mellon

### October 30, 2007

Joel Smith about the OLI in Carnegie Mellon

Originally uploaded by openlearn2007

Joel Smith is presenting right now and he is from Carnegie Mellon.

Create online courses that are of a quality that would match found at the Carnegie Mellon University. They said they are pretty deep in cognitive science. Also, needed to make this available to any student around the world … I think they perhaps did this after the OU …

Some problems in the slides … not sure what – it doesn’t want to go forward – but he seems to know what the slide is about – as he is continuing to talk about it.

So, there are courses that are scientifically based that are online. They’re using something called use driven design/evaluation (not sure what that is!). And it is trying to transform rather than transpose instruction – think they’re trying not be like traditional teaching??

His expectation that the courses they have developed would not be used at the university, but what they have found that the courses were so good, that they have become a prototype to be used. The have 4 feedback loops based on the student learning data … these feedback includes, instructor activities, science of learning, course design and student performance.

Candace has begun to speak … they’re going to talk about the design in the course based on statistics. They bring students in the lab after they have finished the course to see how much they have learned. They found that students who did the traditional course after 2 months, were kind of stuck trying to figure out what they should do and they’re not certain how to connect these abstract tools and skills in the real world.

In the Openlearn initiative (OLI), they do a lot of tinkering with applets to play around with the notion of medians and means. And then they have to reflect on what they have observed (is it kind of like a self-explanation) and then the students do a compare between what they reflected on and what an expert would say (that’s interesting – it should help students in trying to answer a question in academic terms).

The hints and feedbacks they have on applets are based on the mistakes that students have been made and then contextualize them to the applets … students are often surprised that the computer knows what they’re doing wrong.

They’re talking about cognitive tutors now (my stuff …. Again … this is a lot more interesting that I thought it would have been!) … They have made mini-cognitive tutors to help the students along. Students have several levels of hints or feedbacks or they might just work through the problem straight.

Hypothesis: Student in the OLI course can learn the same materials as students in the traditional course with equal learning?

But isn’t this is pretty much just like comparing an online course vs a traditional course, therefore, the students would have the same motivation to pass the exam etc., but OpenLearn students are not students who have to pass a course but just doing stuff.

There were no significant difference between the students on the online and the traditional course.

They decided to use the CAOS – comprehensive assessment of outcomes in a first statistics course – to determine how well they learn statistics I reckon at some time after the course completion.

Second hypothesis: Students in the OLI course can learn the same material in a shorter amount of time as students in the traditional course with equal gains …

Hmmm … all this is saying to me that cognitive tutors help students learn – in particular where students are able to reflect on their work and have some kind of feedback (this is therefore similar to the work that is being done by Renkl and Atkinson or was it Mayer?) …

I would like them to compare the OLI with students in the university and just normal everyday people. That would show whether the OLI is actually working towards learning without having the ‘bootcamp’ ideology that Patrick and Andrew were speaking about before.

The OLI Accelerated group did better than the traditional groups, even the OLI from the prior year who did it at the same pace of the traditional groups. The students in the OLI accelerated had a 20 pnts gain using the CAOS, whilst the traditional group only had a 3 point gain.

They did this in 8 weeks rather than 16 weeks …

Outside of class, the traditional and the OLI group spent the same amount of time doing statistics … they collected this through log files.

They did a retention study where they brought students back into the lab where they did 3 different assignments, so the OLI people had 1 ½ semesters in which they didn’t do statistics whilst the traditional only had 1 semester. They did the CAOS test again and another statistical question. The CAOS remained the same … however, in the open-ended data-analysis problem there was no significant difference but it was close …as the p<0.13 (I think?) …. The students in the OLI course were able to be articulate in their answers as well as say in statistical terms.

She is saying that students have a problem in understanding the = sign – which I’m also finding in my linear programming study. It’s interesting that this is all about individual learning – wonder what the social learning people would say about that??? 😀 ….

They’re doing this for other courses and for other colleges as well to see if this holds.

Question time!

I asked the question about whether they would compare between the informal and formal learning since they have different motivations – but Candace indicates it is difficult for her to find out since usually it is a cohort at the university tend to register as a known user rather than anonymous. Also, that the engineering courses tend to have anonymous people working all the way through as it is possible that these are people who are going to another university or course. Also, that she thinks that in courses such as French would be more casual rather than a motivation to learn completely.

October 30, 2007 at 4:36 pm

Candace has just finished her talk, showing a real role for comparative studies – Anesa asked the key question about whether the study (comparing accelerated learning and standard learning) would be extended to those who are not registered. The motivation is unlikely to be the same. Candace explained the way the site operates and she (as with OpenLearn) has the problem of a large number of unknown learners for whom all she knows is the session data. Peter Bateman asked how CMU might take a system that cuts study time in half inside the university. This seems to be what Candace would like CMU to think about as well.

October 30, 2007 at 4:59 pm

Aside about understanding of the equals sign (apologies to the OER crew): There is a huge literature on this in relation to primary school children. Problems still persist into secondary school (and cause huge problems with algebra and statistics). But nothing that I’m aware of in relation to university students. These students learning statistics – what’s their major?