Friday, December 11, 2009

"Facilitating Online" 2009: Evaluation

For the last few months I have been facilitating and teaching the online course "Facilitating Online". This is the first of three posts that will report on how the course went. In this post I will report back on how students felt about the course, and my recommendations for the future based on their feedback. In my second post, I will talk more about my own learning about being an online facilitator. And in the third post, I'll ask at whether open educational resources work.

Student numbers
When the course started back in July, we had 15 formal enrolments and 22 informal students. Of the 22 people who indicated an interest in being informal students, five got past the first couple of weeks. Of those five, two ended the course - one student completed all the assignments and was presented with a "certificate of participation".

Of the formally enrolled students, two dropped out and 13 completed, which is a 87% completion rate.

Student evaluation
The students had three ways of evaluating the course. The first way was to give oral feedback at our last live meeting. The second means that students could give feedback was on their blogs. The third method was via an anonymous online survey - three students used this method of feedback. The general consensus amongst the students was that the course was a great learning experience.

The most enjoyable aspects of the course
  • Organising and attending the mini events. The mini conference put things into perspective for people, and was a great way to put theory into practice.
  • Understanding the difference between facilitation, teaching and moderation
  • Exploring other technologies - the course opened eyes to potential and possibilities of online communication tools.
  • Interacting with other participants - a very supportive group.
  • Having synchronous meetings via Elluminate - helped people feel more connected.
  • Blogging - useful way to learn and interact with other participants.
  • Support and feedback from the facilitator.
  • Feeling confident to have a go and explore further.
The most challenging aspects
  • Managing the workload - the work load is too great for a course of 10 credits.
  • Managing the online events.
  • Finding more time to interact with their blogs - participants see the value of blogging by the end of the course.
  • Getting the hang of things and building up enough courage to 'talk' online.
  • Elluminate's unreliability made life difficult for people who already were challenged by technology.
  • Technical difficulties were a barrier at times.
  • The course wiki was the most difficult tool to interact with.
Suggestions for improvements
  • Increase credits of the course, or reduce workload - look at the weekly "to-do" activities, which were difficult to keep up to date with.
  • More practice with communication tools before the mini conference - more experience at facilitating live events as the course unfolded, instead of being dropped into things at the mini-event.
  • Preferred a set time each week for live meetings, rather than my approach of changing days and times each week.
  • Participants felt they were thrown in the deep end - would appreciate some sort of pre-course preparation.
A number of participants said they would like to do the course again now they have the hang of the technology, and understand what the course is trying to achieve.

My thoughts
I really enjoyed being involved in the course. I found it fascinating to watch the movement that participants made from being totally confused to having an understanding of how and why networking and connecting is so important. I felt that the group moved from being individuals struggling to organise their thoughts, to a learning community who actively supported each other. I was blown away by how supportive and patient people were, especially at the time of the mini event.

For me, the highlights of the course were:
  • watching the self-organisation that participants did eg the setting up of a Pageflakes page by Chris Woodhouse;
  • discussions that evolved that were driven by students, not by the course eg personal information security and Twitter;
  • involvement of informal students who added different perspectives and helped keep everyone motivated;
  • the mini-conference - diversity of subjects, speakers and communication tools.
Moderation conversation
During a conversation I had with Bronwyn Hegarty as part of the moderation process, we talked about how the students did with their facilitation in the mini event. We agreed that there needs to be more emphasis in the course on the practicalities of how to facilitate, not just focusing on technology - thinking about what makes a session interactive, being prepared with questions and activities that lead the audience in discussion.

My recommendations for the future
  • Increase the credits of the course, rather than reduce activities. My understanding is that is going to happen in 2010.
  • Spend more time looking at the theory of how to facilitate online - have a look at work of people like Nancy White and Gilly Salmon. I think there is a tendency to get hooked up on the technology and we forget that we are there to learn about facilitation. If people are better prepared to facilitate, maybe we won't have the problems that we had this year with the technology failures at the mini event eg empathize the importance of having a back-up strategy that works.
  • Give people opportunity to practice with the technology- I did try this at the beginning of the course but people were too 'shy' at that stage...obviously is a concept worth pursuing. Having said that, there's nothing stopping people from trying out tools at their own instigation.
  • Be consistent with times for live meetings. This is difficult to manage when you have people attending from different time zones. Maybe the best thing to do is alternate an evening meeting with a lunch time meeting.
  • Look at facilitation in more general terms, not just in the educational context - think about facilitation of events, not just as a way of delivering educational content. Important to remember that participants may be from areas other than education.
  • Think about a follow-up to this course as a way of maintaining people's interest and support them to develop their facilitation skills further.
In my next post I will reflect on what I learned from my experience of facilitating this course.


Leigh Blackall said...

Those stats don't look good. Hope you will compare with other instances of the course, including when it was a closed course in blackboard.

Would appreciate it if you changed the sentence around my name in this post. I did not leave the course, the manager of EDC decided not to continue with the development work of seperating learning facilitation from assessment. This was very much against my wishes.

Sarah Stewart said...

Hi Leigh

Have you written up a statistical report from previous courses that I can compare with? If so, where are they? cheers Sarah

Leigh Blackall said...

I did, but is not in a compatible structure. All data is available however, and I could help build review. do you think you could prepare a standard, and i will attempt to enter past data? Would be good to include brons courses too. we should be able to get pre open data from mit.

Btw, a link to my post about leaving op is still not an accurate depiction of why I ceased to be involved in the course. I came to Canberra expecting to finish my commitment. I did not leave fo09.

Leigh Blackall said...

A link to >my initial evaluation of of the 2008 course.

I need to find the numbers for the course before in was made open.. Oriel will know.

I also hope we can account for impacts on the development of the open course such as confused messages going out to formals from the hosting institution (enrolment packs), as well as our inability to transfer all enrolments to being informal and enroling on completion (=100% completion rates and I'd be interested if numbers fell below that of the current model of enrolling up front).

Sarah Stewart said...

I'll chase up the stats for previous courses. It would be good to look at material written about retention rates in formal courses as well as open courses, may be "Connectivism"?

Leigh Blackall said...

We have to allow for the very different motivations of informal vs formal enrolments. The drop off rate will always be higher for informals, and there's not much we can or should do about that imo.

The key thing I think is the effect informal participation has on formal participation. Does informal participation aid the learning of formals? Does informal participation help promote the course? Does informal participation increase formal enrolment? Does informal participation give access to people who would not otherwise have access? Does informal participation undermine formal participation?

Sarah Stewart said...

Those questions need to be asked of both informal and formal students, plus lecturers/facilitators.

Leigh Blackall said...

absolutely.. If only EDC would provide HEDC at Otago Uni the contract they've been waiting on. A video crew and researcher there are wating to ask this and other values based questions of a range of staff experienced with a variety of open ed models, this being one - you being one of the staff. It would be great if you had an idea on the student numbers and impressions before that interview :)

Sarah Stewart said...

I don't know anything about that, Leigh...

Wayne Mackintosh said...

Hi Sarah,

I have a passion for open education and the "Facilitating Online" course is arguably one of the best exemplars of authentic learning in this field.

mmmm --- I don't think that the drop in student numbers and/or active engagement from "informal" students (don't like that label :-() is in anyway a reflection of the potential of this course. It is based on a powerful model of authentic engagement in real online communities.

Let's be bold and target 1000 participants for the next iteration.

The OER Foundation is committed to supporting the course in achieving these ideals. After all -- Otago Polytechic is the world's first institution to adopt an OER friendly IP policy. As founder of the WikiEducator project -- I commit my time and energy to realising these futures for a better world.

Let's make OER future's happen for the planet :-)


Leigh Blackall said...

This would be great to see the OER Foundation and Wikieducator throwing weight behind a Wikiversity course.

Wayne Mackintosh said...

There may be a few features I have in mind which may be difficult to implement on the Wikiversity site -- but fortunately both projects subscribe to free cultural works licensing so we will be able to widen access and reach to learners by hosting on both sites :-)

Sarah Stewart said...

Hi Leigh and Wayne

I am interested in this conversation as a wiki novice. I understand there are communities supporting each wiki platform, and as an educator, I am keen to tap into both communities to get maximum support for my work, and of course, get students.

I do not want to get into a conversation about which is the best wiki, because it would be great to incorporate both wiki in future courses. However, I would be interested to compare the two wiki in terms of access to students ie which wiki was more 'successful' in recruiting students, advertising the course etc? Is there any way I can do that?

Occupational Therapy University said...

Thank you very much Sarah for this wonderful article about Facilitating online education it is very useful.