tag:blogger.com,1999:blog-7928515094798031520.post7777679196122701538..comments2023-07-02T05:42:08.720-04:00Comments on Brain, Mind, and Education: Cargo cult psychometrics? Setting standards on standards-based standardized testsAnonymoushttp://www.blogger.com/profile/14015173340703535140noreply@blogger.comBlogger2125tag:blogger.com,1999:blog-7928515094798031520.post-44952793932905749602009-01-10T12:56:00.000-05:002009-01-10T12:56:00.000-05:00Joe - my reply is many months tardy, but I'm hopef...Joe - my reply is many months tardy, but I'm hopeful you'll see it. I think your point is excellent that inferring knowledge from the results of large-scale multiple-choice standardized tests rests on a lot of assumptions, many of which are at the very least oversimplified, and at worst, just plain wrong. I suspect that there are a variety of "real world" reasons forcing these assumptions, not the least of which include the costs of administering and grading a different type of examination. I don't think that multiple choice exams don't give us information about what students have learned, but I do think that the inherent limitations should be much more of a factor in using the results to make choices about, say, funding schools.Anonymoushttps://www.blogger.com/profile/14015173340703535140noreply@blogger.comtag:blogger.com,1999:blog-7928515094798031520.post-25098244831368694802008-06-01T15:36:00.000-04:002008-06-01T15:36:00.000-04:00Is IRT a Cargo Cult? Maybe so. It seems to make ...Is IRT a Cargo Cult? Maybe so. It seems to make a very simple -- and unjustified -- assumption about the nature of knowledge and learning. Explicitly, they seem to assume that there is a 1D variable describing "how well you know this." That fails to take into account the structuring of knowledge -- context dependence, framings (such as "playing the testing game"), and so on. I am very leery of large-scale multiple-choice testing. It's like doing a "gold-standard" medical study on the efficacy of willow bark in preventing headaches. If we haven't identified the relevant variables, we'll have GIGO, no matter how extensive the stats.An Unabashed Academichttps://www.blogger.com/profile/06066249043372653288noreply@blogger.com