I am feeling a tad uncomfortable with the assessment I chose for the online 'Reflection on Practice' course I have been running.I asked students to write at least two comments on the blog course, dealing with the issues that were being discussed - I asked that the comments be 'meaningful'; demonstrate a critical review of information presented; demonstrate reference to appropriate literature, and demonstrate reference to midwifery practice experience. The assignment was to be completed by the end of the course, which was seven weeks long.
Rationale
The reason I attached assessment to the comments was to ensure that the students communicated with each other and got some sort of critical dialogue going. It has been my experience in the past that motivating students to engage with each other with computer-mediated communication is difficult, especially in a learning management system such as BlackBoard.
Feeling uncomfortable
The reason I am feeling uncomfortable is that I am not at all sure that assessing blog comments is actually achieving anything - does 'forcing' students to comment facilitate learning? Will they write meaningful posts or just the 'bare minimum' in order to pass the assessment? How do you assess someone else's reflections? What makes it a good reflection, or not, as the case may be? If reflection is all about personal learning, what right have I to come along and put a mark to it?
On the other hand, if the students do not communicate with each other and start the networking process, the course will be extremely dry and flat. Indeed, one criticism of connectivism is that it favors self-directed learners (Stack, 2008), so people who need more structure and guidance are less likely to engage without some sort of carrot and stick.
Assessing e-learning
Clearly there will always students who recognise the value of networking and participation for their learning, but a number will only take part in activities if they are aligned to assessment - I know this - I've done it myself. No doubt there are a number of reasons for this, not least time constraints, prioritizing with other assessments, and lack of interest in the content.
Assessing online activity is difficult because of uneven rates of engagement of students (Goodfellow, 2001 cited by MacDonald, 2002) . And then there are the lurkers, the people who not participate - how do you 'measure' their learning? Just because they do not participate doesn't mean they are not learning (MacDonald et al, 2003).
So what is the best way of going about assessing online activity and learning? Helcat (2008) writes "we must spend some time rethinking assessment in order to create assessment centered classrooms that foster learning rather than simply measure it".
Designing assessment
There appears to be a lot of agreement that assessment needs to be integrated into an online course, especially formative assessment because that produces valuable feedback for students as they progress through the course (Caplan & Graham, 2008). Feedback not only gives students an idea of how they're doing, but it also acts as motivation to keep going (MacDonald, 2002). At the same time, teachers need to get a balance between being driven to give instantaneous feedback to students and setting realistic time frames so they have a life of their own (Anderson, 2008).
Assessment should also be congruent with the activity it is based around. It's no good me setting an assessment that involves demonstrating how to deliver a breech baby, when I have been teaching shoulder dystocia (MacDonald, 2002). However, networked learning can be chaotic and distributed, so the learner may not have learned what I think she should learn. Nevertheless, it is valuable learning for her because it meets her own particular needs. Jenny Mackness makes a similar point in her blog post 'Intervention in students' learning'. In this instance it would pay to be flexible in one's approach to assessment in order to capture that 'distracted' learning. And it may be more appropriate to use a reflective assessment framework as opposed to a more rigid assessment criteria (Anderson, 2008).
Assessment of networked learning should be done carried out in a way that reflects the way that knowledge is produced. By that I mean that assignments should encourage student participation (McCloughlin & Luca, 2001). MacDonald (2002) advises
If students are to be given greater autonomy in their networked study, then assignments which encourage greater student participation may help them to develop a self directed approach. Networks can be employed to deliver enhanced versions of innovative assignments used in face to face situations. For example, electronic scrapbooks, online peer review and iterative assignment development.
Helcat (2008) suggests using a range of tools and strategies from researching and writing collaborative reports in Google documents and wikis, to reflecting in blogs, discussing in Ning and evaluating resources in social bookmarking sites such as Delicious. And then all the artifacts generated during this learning and assessment can be deposited in an ePortfolio.
Assessment rubrics
Requiring students to participate by commenting for course marks appears to be common practice. But if you are going to use that strategy, you have to be clear to students what exactly is required and how it is to be presented. A number of rubrics have been developed over the years (Anderson, 2008). I think it is really important to know what you are aiming to achieve when you develop a marking rubric. Is it just participation you are wanting which may mean anything from a supportive statement from one student to another or exchange of resources, or do you want specific reflective statements or critical examination of evidence/literature.
Examples
If you are wanting to see assessment of online learning and get some ideas, have a look a few open courses that are currently running:
- Connectivism and Connective Knowledge
- Facilitating online communities
- Designing for flexible learning practice
If you are a teacher using online courses, what assessment strategies work for you? What assessment rubrics or criteria do you use? If you are a student, how do you feel about assessment? What works for you?
References
Anderson, T. (2008). Teaching in an online learning context. In T. Anderson (Ed.), The theory and practice of online learning. (pp 245-263). Athabasca: Athabasca University Press. Retrieved 18 November, 2008, from http://www.aupress.ca/books/Terry_Anderson/anderson3.pdf
Caplan, D. & Graham, R. (2008). The development in online courses. In T. Anderson (Ed.), The theory and practice of online learning. (pp 245-263). Athabasca: Athabasca University Press. Retrieved 18 November, 2008, from http://www.aupress.ca/books/Terry_Anderson/caplan.pdf
MacDoanld, J. (2002). Developing competent e-learners: the role of assessment.
Learning Communities and Assessment Cultures Conference, University of Northumbria, 28-30 August 2002. Retrieved 18 November, 2008, from http://www.leeds.ac.uk/educol/documents/00002251.htm
MacDonald, J., Atkin W., Daugherity F., Fox, H., MacGillivray, A., Reeves- Lipscomb, D., Uthailertaroon, P. (2003) Let's get more positive about the term 'lurker', CPsquare Foundations of Communities of Practice. Retrieved 18 November, 2008, from http://www.groups-that-work.com/GTWedit/GTW/lurkerprojectcopworkshopspring03rev.pdf
McCloughlin, C. & Luca, J. (2001). Quality in online delivery: what does it mean for assessment in e-learning environments? Retrieved 18 November, 2008, from http://www.ascilite.org.au/conferences/melbourne01/pdf/papers/mcloughlinc2.pdf
Helcat. (2008). Rethinking assessment. Retrieved 17 November, 2008, from http://helcat.org/wordpress/?p=76
Stack, R. (2008). Curriculum as Connectivism. Retrieved 17 November, 2008, from http://hent.blogspot.com/2006/06/curriculum-as-connectivism.html
Image: 'Nathan Setting The Tone For The Exam' rileyroxx
www.flickr.com/photos/41216460@N00/162970439
7 comments:
Hi Sarah,
I'm wondering if you've read Jenny Moon's work on Reflective Practice and Critical Thinking. She has written a lot about assessment and assessing reflective writing and practice. In her Handbook of Reflective and Experiential Learning she writes (p.156)
'We do not expect to assess learners' notes. We ask them to reprocess their notes into an essay or report or we test their learning from the notes in an examination. In this sense, if we assess the reflective writing directly, we are assessing only the relatively unprocessed (raw material).
In her book Learning Journals, she suggests that a preferable choice is to assess secondary material. The secondary piece of work is likely to be a 'report' or 'account' (it is preferable not to use the term 'essay'). Depending on the purpose of the journal - the account may require the student to describe what she has learned as a result of the working with the journal, often by citing examples or quoting from the journal'.
Her books also contain a lot of practical exercises that you can use to with students to help them with their reflective writing and thinking.
You may already know these books. If not they are both published by RoutledgeFalmer
Good luck with your assessment!
Jenny
Hi Sarah
I read your post with great interest - and followed your informative links.
I've just completed running a year-long 'blended learning' course and empathise with some of your discomfort because I also asked students to comment on my course blog and was disappointed with both the quality and quantity of those comments.
However, I think the rest of your post also fits with my experience in that the quality (reflective and critical thinking) of the students' own blogs and their presentations and reports were better than I expected by the end of the year.
In the end I just numerically counted their comments on my class blog to encourage students to write something.
My students (aged 16-18) were not reflective thinkers at the start of the course (or at least didn't write as such) but by the end of the year they were much improved. It took a whole year!
In retrospect I think my class blog was an important model for them and they learnt much - both from my posts and from those who did comment.
I also asked students to use Twitter, delicious and Google Notebook and so was able to track their learning journey in other ways as well.
In the end however the external assessment panel found that students' face-to-face final presentations (there was no written exam) were in most cases all that was required to assess levels of reflection and critical thinking - along with other course criteria.
In fact the top third of the class were thought by the panel to be operating 2-3 years ahead of their year level.
Something worked and I have yet to unpack exactly which processes and structures contributed...
It may have had something to do with my continued expectations that they comment; something to do with modeling; something to do with using a mix of assessment practices FOR, OF and AS learning; something to do with using a range of tools...
Anyway thanks for your post (and attribution) - it will help me to reflect on my own class - which by the way involved 40 students doing a course called Student-Directed Inquiry :-)
Hi Sarah!
Thanks for your comment in my blog, I'm glad you ejoy visiting.
You raise a critical point here, I resonate with you. Offline and online I've always considered that I have no right to assess or evaluate my students, there is no effective or realistic way to see if they have learned and how.
You can observe a change in the way of thinking in their blogs and comments but how do you mark it?
For better or worse the only online course I've opened had a practical approach, evaluation was not included, they were expected to learn how to use a program. Evidence of their learning was an end product using the program they were learning, evidence could be also found in the tutorial site itself which reports progress and areas covered.
This is not the case with a teoretical content, I think there is no single answer. As a teacher you have to ensure a way to observe learning, considering the way assessment is dealed with in other online courses I think you are right on track.
Thanks for the readings you point out here, I learn a lot visiting you. I enjoyed the "I got the power" video.
Have a nice week. Maru
Hi Sarah, I read your post with interest. Good food for thoughts.
Assessment is one of the biggest challenges in teaching and learning, especially if it is done online. May be the questions are:
1. What do they want to achieve in the response post?
2. What do you want them to achieve in the response post?
3. How would they measure and evaluate their responses?
4. How would you measure and evaluate their responses?
As a teacher, you may have developed the best rubric in the world. Are these tools agreed by your learners? Have these tools been developed together and/or negotiated? What do they think about the assessment tools? Are they useful in the reflection process?
I think it's really difficult to evaluate people's learning through the response post mainly because it is a "reaction" to other's post and views, and that reaction could be pretty personal and subjective (and it could be based on reasons with emotions). The quoting of sources and information as guides are part of the evidence of reflection only. As mentioned by Jenny, a reflective journal summarising the learning may be a better alternative tool as it coulds provide a more "holistic" way of assessment. Also, I think Maru has got an important point of looking at this as it is really based on the situation and no single solution would address this.
Most importantly, I think the emphasis would be back to the question of "What have the learner learnt through such connection and interaction?" from the learner's perspective and that "What sort of changes (in attitudes, perspectives or actions etc.) have these brought about through such interaction or connection?"
Do you think the learner could also assess his/her own post or response?
Assessment is a collection of evidence and so it is important to ask: Are the evidences valid, reliable, sufficient and authentic against the tools used, and outcomes or performance measures achieved. So how about using the 2 responses as part of a more holistic assessment?
Thanks for your questions, pretty thought provoking.
http://suifaijohnmak.wordpress.com
Cheers.
John Mak
John, Maru, Roger and Jenny, Thank you all so very much for taking the time to make such wonderful responses to my post. I feel I should respond to every comment as I normally do, but to be honest, my head is buzzing so much, its going to take a while to process what you have all said. I cannot thank you all enough for contributing to my ongoing thinking - you have really proved the value of blogging. I will get back to you to continue the conversation. Assessment appears to be an issue that requires consideration despite the plethora of research and commentary that is available to support teaching practice.
I just found your blog and was very interested in your experience. I am both a blogger and a reader and enjoy the conversation as much as the reflection.
In May, I returned to grad school after twenty years out of school. I have received several assignments to comment or write to a class blog, wiki, or discussion board (my school uses Blackboard too). I am astonished at the number of students who use it as a public homework device i.e. they do the assignment but never read or respond to any of the other student input even when directed specifically to them.
My cohort has now been together over six months and I am just starting to see conversation happening among our blogs. The collaborative piece seems to be a cultural shift that hasn't happened yet.
That's interesting, Kathryn, and good to hear from someone the other side of the fence, so to speak. As I said in my post, students have very busy lives and many calls on their time, so have to see the value in contributing in this way. It's hard to see beyond marks and grades. I wonder if there's a difference between undergraduate or postgraduate education?
Post a Comment