Assessment Mistakes by E-Learning Developers

By Stephen Downes - Stephen's Web. Will Thalheimer[Edit][Delete]: Assessment Mistakes by E-Learning Developers, Will at Work Learning [Edit][Delete] December 13, 2006

The diagram in this post is a lot more insightful than may be grasped at first glance. Here is the diagram:

The point Will Thalheimer is trying to make is that "any assessment at the end of the first learning curve is likely to be a poor predictor of future remembering---and show a definite positive bias." Why is that? Well, when learning is tested in the same context it was taught, it is more easily recalled. But introduce the learner to new contexts, and a chaos of variables intervene, making the previous predictions unreliable.

A 'chaos of variables'? That's being a little liberal with the language, don't you think? Well, maybe not. Take a look at this diagram:

It's called a bifurcation diagram, and it is part of a theory, chaos theory, that describes what happens in dynamic non-linear systems. As we can see, there occurs, at the beginning, what appears to be a nice linear path, but this breaks down into a wide range of possibilities after a certain threshold. So why does this matter? Well, the short version is, if you want to know how your learning is performing, then you need to evaluate after the threshold point. The longer version is that there are implications across the board in our profession. To get a sense, look at this article on categorization in dynamic systems. How naive static systems of metadata and taxonomies look after that. But not just those: consider static theories describing learning designs, learning content, learning management and assessment. More...