I'd be interested to see the results of applying the test repeatedly at n-week intervals throughout the course.
One interpretation of the original finding that everyone who passes the test passes the course, but not everyone who fails the test fails the course, is that some people who did not have a consistent mental model before the course develop one during it (and nobody who has developed such a model loses it). That should show up in a longitudinal test: you would expect that as people develop a model, they switch from the failing to passing groups.
This would be consistent with the sequential learning model. If you did a basic class, and didn't have a model by the end of it, then when you do a more advanced class, you have even less of a chance of learning anything from it, and less of a chance of developing a model during it.
I wonder if a way of structuring a course would be to start with a sort of mental model bootcamp, where the teaching was aimed specifically at developing a model, and where nobody would progress to the rest of the course without doing so. That way, students who take longer to get ready are not confronted with later stages of the course that they will not be able to make use of, and have a progressively greater share of the teaching resources in the first stage to help them do so.
One interpretation of the original finding that everyone who passes the test passes the course, but not everyone who fails the test fails the course, is that some people who did not have a consistent mental model before the course develop one during it (and nobody who has developed such a model loses it). That should show up in a longitudinal test: you would expect that as people develop a model, they switch from the failing to passing groups.
This would be consistent with the sequential learning model. If you did a basic class, and didn't have a model by the end of it, then when you do a more advanced class, you have even less of a chance of learning anything from it, and less of a chance of developing a model during it.
I wonder if a way of structuring a course would be to start with a sort of mental model bootcamp, where the teaching was aimed specifically at developing a model, and where nobody would progress to the rest of the course without doing so. That way, students who take longer to get ready are not confronted with later stages of the course that they will not be able to make use of, and have a progressively greater share of the teaching resources in the first stage to help them do so.