I had a great day on Tuesday when I went to a meeting of the Southern Hub of Ako Aotearoa staff developers group in Christchurch. This is a meeting of people who work in New Zealand polytechnics who work with institute staff to develop their teaching and learning skills.The main reason for going along is because I have a new mini-project which is funded by Ako Aotearoa: to develop a proposal that can be submitted to Ako Aotearoa for further funding to carry out a collaborative project. The project will consist of researching and collating digital resources that can be used to support teachers as they improve their teaching and learning skills.
The question I am pondering is: how can we as staff developers measure the difference it makes to staff when we share resources with them ie if I share a video with a teacher showing her an example of best-practice, how can I measure what effect that video has had on the teacher's practice?
Any ideas?
Image: 'Yellow Tape Measure' Darrren Hester
http://www.flickr.com/photos/42179515@N06/3989949630
5 comments:
sarah
sometimes we cannot measure our impact at all.
SO how when faced with something like "evidence based practice" do we measure Impact?
ok
we ask people to report to us:
1)impressions
2)changes in practice
3)insights into why they might already do something or change something?
this type of selfr report research is very subjeactive and cannot be generalised at all. And it is very valuable.
Willie
Totally agree with you, Willie. The other things is...the impact you have on people may not be visible until years down the road.
The problems is these days is that the people who provide the funding for research projects want you to be able to show your intervention etc makes a difference...so they can see they are getting their money's worth.
The framework you have put here is really useful for me - thanks a lot :) The other thing I was thinking of was a 'before' and 'after' attitudes/skill level measurement...self-reported.
Here is some incoherent stream of thought that was provoked by your post.
Before you start thinking of methods of measuring you have to know, what it is you are to measure. Sounds trivial, but many times we forget to link the learning objectives to the different forms of evaluation. E.g. if the objective is to have an effect on the attitudes, it is no point in checking, how the knowledge or skills have changed. Indeed attitude is a tricky one: somebody once told me that it takes at least five years to change an attitude. So, there really couldn't be a detectable change after a couple of months, could there?
Do you only meet your students online? I ask this is because there are some tools we used in an evaluation workshop at the end of a project that I could share, but they demand physical presence. Unless of course somebody wanted to make online modifications :)
One of the methods is a "card game". Here is a very short and rough recipe.
Preparation
1. Pick out key concepts of the course (ours were words like "competence based curriculum", "co-construction", "acknowledgment of prior learning"...).
2. Make playing cards with one concept on each card (I made 5 sets of cards since I had five groups that would play. Each set had different color on the back. I laminated the cards). Add two Jokers for concepts that the players might want to add (it might be that you forget something of importance, so it's good to have this option available)
3. Make a scorecard. In a table, list out the concepts in alphabetical order, and add next to them three columns where the final choices of each group are marked.
4. Prepare the game board. In our last board we had four sections: actualized, emerging, not applicable in near future, and buffer zone. (At other times I have used a game board with section titled: at once, in a couple of years, not in the near future, buffer zone. And other zones depending on what I want to accomplish. Often the purpose of the game can be prioritizing, and in that case there are only like 5 slots in the "at once" section and people have to make choices).
Game
5. Groups of approximately 4 - 6 people deal the cards. One by one the cards are put on the board, so that the person first tells what he/she thinks the concept means and weather or not he/she thinks it has actualized in his/her organization. The rest of the group join the discussion, tell their points etc. The card is placed in the unanimously agreed section on the game board. Sometimes the card is put on the buffer zone and only after all cards are on the board it's moved to its final place. (buffer zone is actively used when there is an objective to prioritize.)
6. End position of the cards is marked on the scorecard.
7. The groups represent their final board and discuss why each group ended up in the constellation they did.
Evaluation
Discussions during the card game can be recorded for further evaluation. Most times I settle with the scorecards, though. It is usually very easy to draw conclusions from these scorecards: what are the things all groups have adopted? Where are the biggest differences? How could this be interpreted?
Äh, very difficult to explain in short, but this "thinking out loud together -method" put in the form of a game really works well!
The other collaborative method we also use a lot is "remembering future". In it people are first taken to a future date and they are then supposed to describe how things are at that time + tell what actions have taken place so that the situation is what it is.
best wishes
Irmeli
In this, I assume your are thinking about transformation change in practice.
Seems to me the really profound changes, the transformational changes, are reflected in changes ways of seeing, in changed ways in being and in changed ways of understanding.
These changes are reflected in the metaphors and the strings of words that people use when they talk about what they are trying to do, or what they want to do.
Assuming they can get the resources needed to do what they want to do, and have the self-efficacy to undertake the work, then they will start to put their ideas into practice as best they can.
It may take them a while to put it all together, but the significant change happens when they see the the problem in a new way. After that, there is no going back.
After a change of mind, what happens is
emergent.
And being emergent, it very hard to predict in chaotic systems in which we live what you will see and what will happen.
@Irmeli
Thank you so much for your detailed 'game'. It will not be one I can use for this particular project, but it will go into my 'bag of tricks' for the future.
@Malcolm Loving the idea of 'emergent'. Again, not too sure where it would fit into the project I am currently working on, but its got me thinking about what's happening in the FO2010 course. What is the connection...if any...between this and chaos theory?
Post a Comment