How do you know that clinical simulation makes a difference to learning?

I have been mulling how I can evaluate the Second Life Virtual Birthing Unit in a way that shows it makes a difference to student midwives' learning. A couple of things have happened in the last week that has moved my thinking a little further.

Simulation
Last week I introduced the Second Life Virtual Birth Unit to Associate Professor Margaret Hansen and some of her nursing students. The two main comments that came up were that the virtual birth unit felt a "safe" learning environment. The second comment the students made was that this simulation felt a lot more realistic and they could get more involved than they could with the simulations in their clinical laboratories.

In the "hot seat"
Yesterday, I had a fascinating conversation with Swee Kin Loke who is working with medical students in the Otago Virtual Hospital which he has developed in OpenSim. When I asked him how he measured learning outcomes, he said that this was very difficult and something he had not worked out yet. What is clear, though, is that for all the various simulations in Second Life that have been developed to teach health professionals, there are very few rigorous evaluations that show how the sims impact on teaching and learning, other than the "soft" data such as the students' experience of the sim.

Difficulty in quantifying learning
When it comes to measuring the learning experience of health professionals, in the clinical setting, there are many variables that need to be taken into consideration - the Second Life Birth Unit and OpenSim Hospital are not about teaching students new information, but rather giving them the opportunity to practice being "real" doctors and midwives...in other words...being in the "hot seat". Swee Kin said that in paper scenarios, medical students are given the whole context and all the information. With the simulations in the Virtual Hospital, they are thrown in the deep end and have to get on with things...just like they will have to do when they are registered doctors. As far as he was concerned, the learning in virtual simulations is more about the "a ha" moments, which are very hard to quantify and generalise.

Making comparisons
Going back to my question about how to measure learning outcomes, I have been leaning toward looking at hard outcomes like exam grades before and after exposure to the Birth Unit, but I appreciate there are a number of problems with this approach. Since my conversations with Kin and the American nursing students, I am wondering if it would be better to do a comparison between simulations held in the face-to-face classroom and laboratory, and the Second Life normal birth scenario.

Developing expertise
The other idea that Kin has is comparing how an expert practitioner behaves in the simulation...solves problems...communicates...makes assessments, compared to the students. This would help me draw conclusions about what the student has to do to progress to the level of "expert" but I am not sure how it would answer my question about the difference the Birth Unit makes to students' learning.

What Kin reminded me about was that it is possible to video and record students' interactions in SL which makes it easier to do content analysis, and also use for debrief with students.

How do you know clinical simulation works?
What has been highlighted to me is that I need to do a lot more reading about how students learn from clinical simulation in the traditional forms, and maybe I'll be able to take ideas from current research and adapt them for the virtual environment.

I'd love to hear from health educators who use simulation to teach nurses, doctors, midwives etc. How do you assess the effectiveness of this form of teaching and learning?