May 15, 2007
Summary of a panel discussion at CADE-AMTEC 2007 in Winnipeg, featuring Michele Jacobsen, Terry Anderson, Heather Kanuka, and Margaret Haughey.
Michele Jacobsen:
Context: special issue of CJLT on e-learning with an emphasis on Canadian research. Systematic literature review of over 700 documents. We took the report and sent it to several researchers in the field. So we have the report plus four commentaries. This puts the peer review process right out there in the open.
What the researchers found, public response:
- e-learning is a rapidly growing field
- it provides greater access
- funding may divert resources
- teacher classes remain essential
Policy makers:
- generally supportive, but
- must be used in appropriate contexts
Others said:
- it requires careful attention to design, etc
- we need to review policies
Reviews of e-learning (about 300 documents) were generally positive. E-learning requires sustained support. But there is a lack of empirical evidence to support e-learning.
Heather Kanuka
It was a fine job of reviewing the literature. Much of the literature was qualitative (only 15 percent quantitative). That makes it hard to say what works and what doesn't. The e-learning research community has to face the reality that its efforts have failed to provide guidance. Much of the research in e-learning is based in realist philosophy, the assumption that there are natural laws that can be observed. If this assumption is erroneous - and I think it it - then we are asking the wrong questions. We simply can't pile up the generalizations fast enough.
Many e-learning researchers guide their research on the assumptions that government natural laws. Hence the lament of there being only 15 percent quantitative. So a large body of the research has little social relevance. It doesn't reflect the messiness we face as practitioners. We need to find practical solutions to the everyday problems we face, and to recognize that we are living in a 'swampy lowland of practice'. Need to use more emergent methodology that addresses messiness.
So I want to put a different sin on the finding of the researchers, that we can't find best practice. I want to say that the majority of researchers are using emergent paradigms and are finding socially useful results. Over the years I have found what works and what doesn't, and I'm comfortable with saying that, and I'm fine with not being able to generalize best practices.
Terry Anderson
There is a lack of funding for research in education in Canada. It's bad enough that only 15 percent were quantitative. 7 quantitative studies that were Canadian. There's a limited amount of insight you can get from that. The only way to generate value that you get so many of them that the effect of individual contexts are wiped out. If we had a few hundred, a few thousand, we might be able to extract some meaning from that data.
So then we ask, why aren't we doing more. And we have to say that, as Canadians, we are simply n ot committed to research in e-learning, or in education generally. By contrast, the UK, 74 different institutes. Also awards, eg. National Teaching Fellowships. And they have the JISC and a whole bunch of things.
Other countries can do things, even Russia can do things, it's not just the fact that we are in a federation.
What I really liked what they did was that they looked at all the other documents, eg. government studies, media, and so on. And they came back and made correlations between them saying positive things and why, eg., saving money, and they were happy to say the reason for e-learning is access and distance learning.
It's interesting you see the rebuttals, the old paradigm wars are alive and well, the left and qualitative studies vs the right (and I notice the right were too scared to come here today).
Margaret Haughey
We were asked to respond to the documentation itself, and in the first place I thought of it as a classification scheme, so I look at the effort to create a classification scheme that was based on different types of work. They didn't quite pull it off. The translation of it into numerical data and the trying to give things a plus or a minus with a rating scale didn't really work.
E-learning - what does it mean? We tend to talk of it as though it were dropped from the heavens - what is the impact? Or thinking of it as a tool. But the world has moved into an environment in which technology is everywhere. What's happening in e-learning is really a reflection of what's happening in other fields.
When I did my review, at the same time Ungerleider did his (but I didn't get the same kind of press) it was full of case studies. Look at this conference - personal learning environment, connective knowledge. There's a whole new way of thinking. Even as you're trying to study e-learning, it's transforming itself. Informal learning. So I put back to the authors that their definition was so broad it included everything, and when you include everything you can't draw any useful conclusions. Kids learning to keyboard - is that e-learning?
Also - integration, immersion. Thinking about it while we're doing it.
I would disagree with the generality, you use such general categories everything gets dumped into it. I see work on dialogue, on developing community. These are areas where there is a body of work.
Discussion
Dave Cormier: "Do we really need the 'e' any more?" As long as we're still thinking of technology as something 'other' it will never be really integrated.
Margaret - I agree. It puts the emphasis on pedagogy.
Terry- agree, but it is interesting that the authors classified it into instructional, presentational, etc. As long as we keep in mind that it's multipurpose, we'll be ok.
Rick: I wonder if we're back to the old 'media effectiveness' question, just smudged. I wonder if people are thinking of online learning environments, Moodle, when they're thinking of this. But the 'media effectiveness' thing was debunked 20 years ago.
Terry - that zombie has risen from the coffin.
Heather - we get into conversations about what the impact is of the use of technology. Not technology determinism, but belief that technology is non-neutral. There is an impact. Telephone is the same as not face-to-face. We need to know, eg., how does text-based mediated communication change activities. Asking what the consequences are and how they impact our communication.
Rick - isn't the problem the term? We don't get specific. We talk about text-based, that's one thing. But there's Elluminate, which is audio, that's a different thing.
Diane Janes - wondering about the zombie and the effectiveness debate, and tying it with funding, that wants us to be effective. "I don;t want my child to be damaged by something that's not effective." Was the zombie ever dead? The world comes back and says, "How effective is this?"
Margaret - the conference where we presented these papers had a whole lot of OECD people at it, and then Charlie spoke, saying, you should stop it, you're not accomplishing anything. But the better analogy would be, the automobile. Once, we had maps with gas stations, because we needed them. And roads. These days, we just assume they're there. There are so many things that are part of the infrastructure of the automobile. The question of impact was asked then - why should we do this. The horse, it doesn't need gas stations. If it were only that we introduced tech, and everything else was the same, we could do the research. But things change - the curriculum, the testing, the teaching strategies.
Terry - the easy out is to say 'how do we know that was effective'. The only answer is to say, 'How would I learn'. To say with some authority, that's what I'm using.
-- the labling issue was a bit of a barrier. We're so hung up on computers, we wonder what we are going to do with voice over internet, oh no, we know nothing about VoIP. But if we say 'audio', we have a lot to say about audio. Because we say 'computers' we're losing a lot of what we learned from distance education when there were no computers. It was only in rare models that we had no contact. The labeling can really get in the way so much.
Heather - I agree. Peters wrote an excellent article on building on the foundations. There's a lot of talk about old wine in new bottles...
Terry - it's in the Bible.
--- I don;t know anything about the methodology or the breadth of the study.
Michele - journals, public policy, scholarly academic reviews....
--- maybe one of the reasons we got 15 percent is that there was a huge pile that was missed. Eg. Journal of Medical education - was 95 percent empirical research. Parallel reviews. If those were missed in many of those disciplines, empirical research is a requirement...
Michele - It was a focus on Canadian educational journals.
Terry - listed pretty broad sources.
Michele - It is online, cjlt.ca
--- Re: people looking at technology as a value-neutral tool - it's not, of course -- people tend to think of technology as always a good thing. Wonder if you have a response to that attitude. Have we created an industry that becomes self-propelling.
Terry - well the most-used technology is the blackboard, so I don't think we're overwhelmed. I think we've got lots of conservatives.
Heather - we have to justify the way things are currently done.
--- I was upset with my son's science teacher, he was using a 1981 textbook, there's no computer, it's 1970s style education, for a junior high school in Alberta it's not very good stuff. My daughters history text is from 1983, it says there are a million computer users now in the U.S.
Heather - I was wondering what's the difference between animated objects vs transparencies?
Margaret - the real shadow piece is around th questions that is asked around impact and testing and accountability, we can see from the United States, there's a big emphasis again on objectives.
Michele - I'm also looking at the issues of equity. Also in Alberta, a school, fully funded by Apple, with one-to-one computing. There's no system of accountability, no system for assessing whether it's being used responsibly. Or access is so locked down you can't reasonably do research.
