If greater, the examinee gets the item incorrect. If that number is less than the probability of correct response, the examinee “gets” the item correct. The next step is to generate a random number from the set of all real numbers between 0.0 and 1.0. Well, we can simply plug those numbers into the equation for the three-parameter model and obtain the probability that this person would correctly answer this item. The CAT algorithm looks in the bank and says that we need to administer item #17 as the first item, which has the following item parameters: a=1.0, b=0.0, and c=0.20. Start by supposing we have a fake examinee with a true theta of 0.0. Here, let’s assume that we are using the three-parameter logistic model. Well, it differs based on the model that is used as the basis for the CAT algorithm. How does the response generation process work? Likewise, the examinee thetas can be from a database of past data, or generated. The item parameters can either be from a bank of actual items or generated. The Monte Carlo simulation approach only uses the response generation process. The monte Carlo simulation approach is defined by how it addresses the third requirement: it generates a response using some sort of mathematical model, while the other two simulation approaches look up actual responses for past examinees (real-data approach) or a mix of the two (hybrid). A way to determine how an examinee responds to an item if the CAT algorithm says it should be delivered to the examinee.Examinee scores (theta) from item response theory.Item parameters from item response theory, though new CAT methods such as diagnostic models are now being developed.We can use that same algorithm on imaginary examinees, or real examinees from the past, and simulate how well a CAT performs on them.īest of all, we can change the specifications of the algorithm to see how it impacts the examinees and the CAT performance.Įach simulation approach requires three things: How a Monte Carlo Simulation works: An Overviewįirst of all, what do we mean by CAT simulation? Well, a CAT is a test that is administered to students via an algorithm. Let’s begin with an overview of how Monte Carlo simulation works before we return to that point. This is because it requires no actual data, either on test items or examinees – although real data is welcome if available – which makes it extremely useful in evaluating whether CAT is even feasible for your organization before any money is invested in moving forward. Monte Carlo simulation is the most general-purpose approach, and the one most often used early in the process of developing a CAT. There are three types of simulations: m onte carlo, real data (post hoc), and hybrid. Simulation studies are an essential step in the development of a computerized adaptive test (CAT) that is defensible and meets the needs of your organization or other stakeholders.
0 kommentar(er)
