The high point, or perhaps end point of my theoretical thinking was 25 August 2009. It all went to shit after that. I was disappointed by the results of my further iterations, and by the imbecility of the contemporary local school teachers.
I don't know why I was obsessed with the idea of multiple iterations. A single adjustment of item difficulty to compensate for the ability of the students tackling them, and of student ability to compensate for the difficulty of the items they tackle seems, on reflection, more than adequate. So I shall now revisit the calculations I did 2 years ago and focus on the results of the first pass.
A couple of things occur to me as I run my eyes over the data again. The first is the size of the dataset, at around 15,000 records, it is much larger than I remember. The second is the depth of the item list, at around 380 items for addition alone. So there is room for many more than the five difficulty levels that I currently use.
I am not sure what got me into such a negative frame of mind. I dug myself into a catch 22 mindset. that I needed more data to make it better, and I needed to make it better to get more data. But in fact I was already sitting on enough data at least to make some improvement.
So now I am sorting the item list by numeric value of the left and right hand numbers, and observing the completeness of the set. The number 1 has been combined with the numbers 1 to 7. The number 2 combined with the numbers 1 to 6. The number 3 combined with the numbers 1 to 5. The number 4 was combined with the numbers 1 to 8. So there are some gaps, but they are quite narrow. Certainly I think the first step now is to use all the number combinations in the existing data, and then later the gaps can be filled. I shall also order the items exactly by the results in the data set, with no personal juggling.
One of the problems with never having documented what I did in the past, is that I have spent hours writing something to produce item arrays from sorted items, but I have now idea what it was or where is it.
My first thought was that the arrays would have been produced with a few lines of Java, but the folder, where I found text files populated with arrays, had no sign of any java code, and a hunt for source files containing the term array yielded nothing. I then found a spreadsheet, which had probably been used to produce the files, containing cuttings from an Access query. Eventually I found an Access database dated March 2009, six months prior to my attempts at systematic estimation of item difficulty. So the logical thing to do now is to cut and paste those queries into the September database.
The first problem is that the September database uses compound items, whereas the March one uses the individual terms. Note to self - why did I ever put the compound item into a database and thank goodness I've changed that. A second is that after spitting the dummy two years ago, I did no analysis of the operations other than addition.
But with a bit of juggling and a tiny bit of cheating, I produced four new item arrays. The cheating is not an issue, because this is just an opening array set. They will all be adjusted regularly as new data comes in.
No comments:
Post a Comment