Reflection on Practice: Evaluation

The online course I was running has come to a close so now I need to bring together my thoughts, students' feedback and peer review in order to evaluate the course and discuss any changes I would recommend - this is a long post written to fulfill terms of a 'design and construction course' I am currently taking, so you may want to give it a miss if you're not interested in education.

My plan for evaluation was:

1. Feedback from students as we went along on blog, email, Elluminate, and one-to-one phone calls.
2. Anonymous weekly poll on course blog.
3. Formal institution student evaluation at end of course
4. My personal reflections and evaluation in my blog that were open to students for comment - could be anonymous if they wished.
5. Peer review during development and following course completion.
6. Blogging network to run issues by throughout development and implementation.

Feedback from students
Content
I used a number of methods and media to provide course content, including video, PowerPoint presentations with audio, and synchronous lectures and discussions. I also suggested students worked their way through several modules pertaining to reflective practice that are embedded in the Preceptor Education Program which has been developed in Canada.

• Delicious - excellent. Very valuable means of making working at computers more portable for me.
• Sarah, first of all I want to thank you for all the incredible posts and links that you have made available. They are very inspirational and have given me a huge collection of resources, to be use now, and in the future.
• Elluminate was excellent: great way to interact with fellow students, and very simple to use.
• I have finally had a chance to sit down and listen to the recording from Monday night. What a wonderful session. It has really got me inspired to write a reflection? I found the PEP modules all very interesting.
• Just wanted to say although I have only done the first P.E.P. module I thought it was excellent. I liked the video's as they really highlighted what reflective practice is, and how it is done, with live people!

Sense of community
I was surprised how quickly a sense of community developed with no formal intervention from me. I am positive this is because of the synchronous sessions we had on Elluminate which quickly grew from an initial plan for 2/3 sessions for the whole course, to weekly sessions. Even now, I really miss our get-togethers.

• I have felt a friendship between the group which started very early – I think from honesty and respect.
• Highlight of the course for me was the generosity of everyone in sharing their reflections and comments: nice to know midwives all over have shared common experiences: and the generosity of Sarah in sharing all her cool web sites and links.
• The highlights of the course for me have been the introduction and interaction with other midwives from other areas.

Use of technology
I was also surprised at how quickly the majority of students connected with the technology, but I did have to make a few adjustments: "Reflection on Practice: up and running" and "Reflection on Practice: starting to rock". Using a 'short and sweet' poll was a great way to get feedback from students as we went.

• I enjoyed learning about new forms of communication such as blogging....
  I enjoyed the technological aspects of the course....The different technologies have help engage and maintain my enthusiasm.
• Although I have not been an avid Blogger (yet) I have enjoyed reading all the posts and have found the links amazing.
• The blog is a great resource to access and I enjoyed the elluminate sessions.
• I have not used blogs before or the other online tools such as eluminate and skype, but find I am really enjoying learning about and playing with these programmes. I have downloaded skype and bought a skype phone off trademe - all ready to go. I have told family members about skype and they have downloaded too, what a neat way to keep in touch. (I’m also hoping to save on toll calls).
• I am so proud! It was great to have a play but feel free to cut me off if I start to sound like I`m on a soapbox! My 7yr old daughter also became fascinated with mummy talking to the computer I didnt realise she was stood behind me!

Blog vs BlackBoard
Students appeared to enjoy the blog compared to BlackBoard

• I have done two papers via the blackboard medium and your paper Sarah using the blog. It has been a learning curve for me to come to grips with how the blog worked. Overall I would say I enjoyed the opportunity to come to grips with a new mode of working with the learning material mainly because it has opened up a new world for me. I can see a lot of potential for using blogs and such mediums; eg wikispaces for an eportfolio, and am even exploring carrying these ideas over into different arenas of my life. In summary: the blog took a while for me to get a handle on but once I knew how to work with it - I liked it. I liked the use of illuminate, inviting guests, live chat, all of which make a learning experience more valuable because ideas are drawn out and articulated in the context of conversation. Blackboard I got used to more easily and was getting more familiar with it after 2 papers. I'm not aware wether live chat can be used on blackboard. The chatroom part is good though and is similar to the email part of our blog.
• I used Blackboard for 4 papers this year and much prefer a Blog. Much more fun and easier to use (once you get the hang of it!). I found Blackboard time consuming opening up all the comments.
• I know I haven`t used blackboard but I just thought that it may be worthwhile to let you know that because I didn`t know any other way of doing things that the blog seemed straight forward. 1. The information was easily accessed 2. It was very easy to make a comment and worthwhile contribution to each blog. So I know I can`t give you any comparisons I felt I had an advantage not having any preconceived ideas,if that makes any sense.
• Overall I enjoyed, and therefore felt as though I learnt more using blogs. Blackboard was not as user friendly. I always seemed to have difficulties logging in, and making posts to the discussion board. That may have been software issues, as things improved a little when I started using firefox as my internet browser. No logic to this statement, but I found Blackboard a little intimidating, as I felt as though everything I posted had to be 'proper'.In a strange way once I started posting on the blog I felt a dramatic improvement in my confidence to post comments, I felt more creative. Words that sum up a differences between blog&blackboard. Blogging is more colourful, whereby blackboard is black and white, and flat!!!!not open-topic.

Housekeeping
A number of 'housekeeping' issues cropped in the feedback that students gave as the course ensued which required that I provide additional support and information:

• blog log in
  
• how to write a comment on the blog
  
• access as remote student to online library databases
  
• referencing
  
• option for future study - students found it extremely useful to be able to have issues clarified by the postgraduate program coordinator on Elluminate.

Unexpected outcomes
Nearly all of the students have started up their own blog/wiki or ePortfolio. This was not a requirement of the course, so it will be interesting to see how these develop now the course has ended.

• Tania - Tania's E-portfolio
• Valerie - Rosebud's blog
• Julia - Tales from Foxton Beach
• Pam - oh my god, I have a blog
• Bryn - brynportfolio

In future course, the students wanted to be able to contribute to course blog as authors, so I will need to look at how I can facilitate that next time.

• My suggestions for next course would be perhaps the ability to contribute to the blog itself, i.e. being able to make posts.

The other unexpected thing was that a troll found two of the personal blogs and behaved in a very offensive manner. As a consequence, the students had to look at how they would manage this, and activated their ability to moderate comments. It was a salutatory lesson as to the downside of blogging and online communication.

• I'm not pleased with the troll though and think that if this type of situation is not able to be resolved it does make a blog less workable.

Ongoing learning and development of a community of practice
The students have also formed their own group blog: New Zealand Midwives Practicing Reflection.

• I have really enjoyed this paper, and are sad to say goodby. Perhaps as a suggestion we could form a 'Lounge' type of blog, as a means of keeping in touch and sharing our progress?
• After my week's rest from the computer, I am itching to get started! I am planning BIG. I hope do do a e-port. as part of going for QLP program
• I am so pleased that I chose to do this paper. I will spend my spare time now writing Reflections to complete my Portfolio (on paper) then work on my e-portfolio.
• I haven’t done any writing abut these situations as yet however I find myself challenged by this paper to begin some sort of journal to record my feelings, ideas and reflections within my practice. I would like to experiment with a blog as I use the Internet quite often and it may be a helpful way for me to get over the hurdle of putting pen to paper. I have registered one but haven’t started using it yet.
• I have a paper portfolio but in the next few years I envisage having an internet based portfolio as well...The beauty of Web 2.0 is that I can combine reflections with computer mediated applications. I know it is not a new concept for you Sarah, but for me it is revelation that has motivated me. I feel engaged in the process of integrating it into my professional development and ongoing learning, rather than filing my portfolio away for another year or two, as I have done in the past.

The general consensus is that they want to become more faimliar with the concept of ePortfolios so they can advise Midwifery Council if/when the time comes that Council opens a debate on the place of ePortfolios in the profession. I plan to investigate how this developing community of practice can be integrated into a research project which would look at ePortfolios and midwifery.

My personal reflections and evaluation
I was happy with my use of Elluminate to deliver content. And I think one of the reasons the meetings were successful was because I ran them in the evenings which appeared to best suit people's commitments. It was also good to have a guest speaker at one of the sessions which took soem of the spotlight off me. And using cell phone text to remind people of the real-time sessions was very effective-I'll definitely do that again. But after attending the session led by Sue Waters and Jo Hart, I know there are ways I can make these sessions more interactive.

With less motivated or interested group of students, I may need to facilitate more activities to encourage community-building and help familiarize students with the technology. And clearly, it is important to foster a sense of honesty and respect, especially when looking at reflective practice which requires people to be very open about their feelings and actions.

• Firstly I have to say that my biggest achievement so far is just finding out what a blog is and hopefully replying to it. We Will have to wait and see if I can get that far...my son who still lives at home, he is totally into computers and spends a lot of time tutting and rolling his eyes at me when I ask for any help.
• This is the first time I am posting a comment two weeks Late!! I am still trying to get a handle on blogging as some things seem mysterious to me.

Thus, it will be important to set up strategies accordingly to meet their needs. Keeping in touch in an individual basis, I felt, was extremely beneficial although I have had no specific feedback to corroborate that.

Growing a learning network
Whilst the students have gone a long way to develop a community and they have very enthusiastically seen the potential of the tools I have introduced to them, I do not know if I can say that they have been effectively networking. But having said that, the course only lasted seven weeks, which is far too short a time to develop an effective network. However, even in that short time, the students made connections with each other and started to self-organize and support each other.

• I wonder if I could offer some student help to... I know how she feels. I am not sure how I could help, but let me know if I could.
• I have taken advantage of the free Elluminate room on offer. If anyone wants to have a play let me know by email and I will send an invite to you to join the room for a "orientation party".

Moving away from a learning management system
From my point of view, moving from BlackBoard to a blog and email group has hugely reduced constraints on the way I deliver content, facilitate communication and link to external resources (I had previously delivered a similar course to this on BlackBoard). I felt it was much easier to link out to external resources, and there was generally a more relaxed approach to the course. I enjoyed myself far more, which in turn motivated me to be more creative eg I organised the real-time sessions in the evening when I normally would do it during 'working hours' - I didn't mind doing this in 'my own time' because I knew it was appreciated by the students and I gained a lot from our sessions.

I was concerned that this more relaxed approach would result in a drop in academic standards, but students all passed the course with very good grades. However, I feel this is something that needs to be monitored - it is easy to fall into the trap of just chatting in a blog environment which is great for establishing social relationships, but I also want to ensure that students think and write critically and 'academically'.

When it comes to looking at numbers, initially it looks as if the discussion board on BlackBoard was utilised more readily:

• BlackBoard - 18 comments per person
  
• Blog - 16 comments per person
  

However, alongside the blog were 151 emails which were mostly initiated by me. And the weekly Elluminate sessions which had a minimum of 67% weekly attendance rate - this is very successful compared my use of teleconferencing in a previous course when average participation was 25%. Students also shared information with each other through individual blogs and wikis that they started to develop. So I would conclude that at the least, the blog course did not reduce communication compared to the BlackBoard course.

Openness?
One of the reasons I am so keen to move away from the LMS is that I believe an open course encourages networked learning. In this instance I was required to keep the blog closed which initially did not sit well with me. I believe there is a lot of interest in reflective practice, not just in the midwifery profession so it would be great to network with people who have the same interests. Having said that, students were incredibly honest in their comments on the blog which contributed to this great sense of trust and respect in the group, and I wonder how much that would have been affected by having the course open. Students had mixed feelings about having the course completely open: Reflection on Practice: end of week three. However, I can see no reasons why you could not completely open a course that dealt with less personal topics.

I did open some of the Elluminate sessions and we had a couple of visitors who were not part of the course, which I enjoyed because they added different perspectives to our discussions: Midwifery and ePortfolios. However, I did not make all the recordings openly available because I took the decision that the discussions we had were too personal to publish. I know I could have asked the students, but instead I chose to make unilateral decision about this.

Overload
I was mindful of not overloading people by requiring them to engage with too many technologies. However, I thought that using different media to present content would role model how people could use technology for learning. And on the occasions I did introduce them to tools, I tried to make it fun, and did not require them to do anything with it. However, students went way beyond my expectations and quickly explored the tools with little or no instruction from me: have a look at how this student used Wordle to explore her personal philosophy.

• I have been playing around on a wikispace since the weekend trying to figure out how to do an eportfolio. I tried using glogster but wasn't so successful because I needed someone to tell me what to to. Anyway What I have done is opened your eportfolio on wikispace Sarah and kind've copied it. Is that OK. That way I am getting to figure out how it works and I have a primitive type eportfolio happenning!! I am really pleased. I am due for review early next year so have a goal in mind and hope it work

Peer review
Two colleagues gave me feedback. Deborah had facilitated the course when it had previously been hosted on BlackBoard.

• I am just so impressed with the content, presentation and student response to this course. The course looks fabulous and interesting and is a far cry from its previous life (well it was more like a corpse actually) in Blackboard... I can see that you have facilitated student learning in a way that encourages their independence and provides them with skills that will carry through to all other aspects of their professional lives. Sarah, I cannot say enough about the student response! I have never seen students so enthused about reflecting on practice! I am also really impressed that the students are so keen to maintain links with other students in the course as they continue to focus on their professional development.

Bronwyn also reviewed the course, particularly paying attention to the assessments. She suggested that I scaffold the assessments as opposed to expecting them both at the end of the course. She also felt that I should make the development of the portfolio a more integral part of the course and encourage the students do small group collaborative work in Elluminate. I am keen to consider this for the future, but I am also concerned that students engage in constructive discussion on the blog and am not sure that is guaranteed unless I make it an activity that is assessed.

Input from blogging network

• My little teaching dilemma
• Ditching BlackBoard
• CCK08: Assessment and Networked Learning
• CCK08: Motivating and engaging students in networked learning

Image: 'Hair Selection Practice' chefranden
www.flickr.com/photos/48394718@N00/1882123525