Client Driven Modifications

There are now three thousand entries in the database. A thousand came from a single evening's work by my niece. Another 500 came from a nephew. The remainder probably came from my daughters. A number of design changes are prompted by the data so far.

The first is a mechanism to prevent multiple errors. If a student can't solve say 20 divided by 10, nothing is gained by having them submit 10 consecutive wrong answers. It wouldn't happen in a pencil and paper test and it shouldn't happen in my Java Math Test. For the purpose of pure Rasch theory, all that matters is that one student did not know the answer to one question. So including 10 wrong answers in the database gives rise to quite a serious distortion. But from the perspective of the user, allowing only a single attempt might be confusing; or would it?

My first instinct had been to add a "cheat" or "panic" button, to get the user out of a bad loop. And the current code just leaves the item there for a second (third, fourth ...) attempt. But, after thinking about the theory, and the traditional pencil and paper test, perhaps the correct procedure is to log the wrong answer, and go straight on to the generation of the next item. In this case, something needs to be added to the messages, to remind the student what the item was, and to inform them of the correct answer.

The second is a mechanism for me to track more accurately when the Applet is being used by outsiders as opposed to family members. Getting the IP address of the visitor to a web page is pretty easy with PHP, but the question is how to store it. In the current format, the SQL query is more or less written by the time a PHP script is called, and there is no field in the database for an IP address to be stored.

The third change, requested by a school in the Philippines, is the ability to track individual students. I had deliberately left this out, because of the Australian/Western obsession with privacy.

Australian data protection legislation enshrines ten privacy principles, which begin with:

An organisation must not collect personal information unless the information is necessary for one or more of its functions or activities.

The key word here is personal. I don't need personal information, but if users elect to personalize their own transactions for their own purposes, they must deem it necessary. Furthermore, if schools are encouraged to use avatar ids, which only they can associate with real students, my data will remain impersonal to anyone except those who created it.

Another principle worth mentioning (Number 8 on the list) is:

Wherever it is lawful and practicable, individuals must have the option of not identifying themselves when entering transactions with an organisation.

So notwithstanding what tracking mechanisms certain schools elect to use for their own purposes as long as I maintain the open (and anonymous) portal, the project as a whole will comply with this principle.

Of course if schools or individuals ignore my recommendation to preserve anonymity, there are eight other principles to consider, including data quality, security, openness, access and sensitivity. I think I need to cut and paste a suitable "conditions of use" from somewhere for those who elect to track their own data.

Leaving aside these potential legal issues, if I am changing the database to enable schools to track their students, I can pop in a field for the IP address at the same. And while I'm at it, I'll modify the Applet structure to collect some data before use as well as storing data after use. In my first post on Applet/Javascript Communication, information was passed from a web page to an Applet, so I can use that model to pass the student id and IP address to the Applet as parameters. They can then be woven into SQL statements to be passed back to the host page.

In the current version, users are taken straight to the page hosting the applet. In future versions, users will go first to a PHP page, which offers the option to log in, or to remain anonymous. They will then go to the page hosting the applet, and carry with them an id parameter, which will either be a real id for those who have logged in, or an anonymous id generated from the date stamp and IP address. Technically of course, the IP address could be used to identify a user, but most ISP would only give personal information to government agencies is special circumstances. As the users of my applet are not doing anything illegal, this is not going to happen.

The fourth change, which might as well be woven into all these others, is to allow operations with decimals. The current Items table only allows operations with integers. In fact the field type is actually small int, which further restricts the fields to plus or minus 32767. This is fine for multiples of ten in the order of magnitude, but it could lead to problems with larger multiples. I think for the foreseeable future, a single precision float will be more than adequate to store decimals in the item columns, as well as larger number if required.

The question is, should the IP address be stored alongside id with every transactions. It is a bit of a Catch 22. With the current infrequent use, the start time in milliseconds should be more than adequate for unique separation of sessions, because the odds of two sessions beginning simultaneously are very low. But with the current infrequent use, space is not an issue so IP could be included without running into storage problems. And as the frequency of use increases, and the odds of two simultaneous starts increase, so space becomes more of an issue as well. I think for now, I'll shove the IP address in. After all, computer space is pretty cheap these days, so even if I hit my web host limit, I can always siphon data off and store it at home.

For the handful of schools who want unique tracking, I can add an extra table to store their user id's and any other data they want to store alongside it.

Local Command Line Diagnostics

Being stuck in the middle of a typhoon with no Internet connection, I found myself browsing my computer for the pre-Applet version of my Java Math Test. And when I stumbled across a collection of code, I couldn't believe how advanced it was - with all the core functionality, but a slightly simpler interface - identical in fact with the April 2009 Applet. It ran perfectly, and better still, ticking away on the command line, were all the diagnostic messages, which are just not practical to display on a production or near production Applet.

So while the storm raged outside, I busied myself bringing the front end of this local version up to the level of the current applet and used it as a test bed for new code. This worked much better than testing code in a public web space, because no matter how much you label it as a test version, somebody will stumble across it, try it, and get put off if it doesn't work. And it pretty well eliminated the need to have diagnostic messages painting themselves across the face of the Applet.

Coding for Improved Flexibility

My Java Math Test Applet is designed to respond to student ability, but I have found difficulty fine tuning the responsiveness for able and less able children. The most able children want to race through the difficulty levels, so that they can get to the top level by the end of the activity. The less able children get put off if the items get too hard too quickly, so they want a more gentle gradient.

The current promotion code, called by a correct answer, is as as follows:

void raiseLevelnum(int oldLevel) {
if (promoscore > 2 && timeonTask <= 3) {
if (oldLevel < 5) {
levelnum = oldLevel + 1;
} else {
levelnum = oldLevel;
}
promoscore = 0;
}
}

The relegation code, called by an incorrect answer, is:

void reduceLevelnum(int oldLevel) {
if (oldLevel > 1) {
levelnum = oldLevel - 1;
} else {
levelnum = 1;
}
}

So to get promoted, the student needs 3 consecutive correct answers and a rapid response on the last item. That is about right for an Australian Year 2 or 3 student, but it is too slow for a year 5 or 6 student. My plan is therefore to offer students a choice of gradient, but I also need to tighten up the student responsiveness code.

And while I'm at it, there is hard coded maximum difficulty level of 5 in:

if (oldLevel < 5) {

This needs changing to a soft parameter so as to make future changes to the number of levels easier to implement and to allow for different numbers of levels for different operations:

if (oldLevel < levelMax) {

Now giving students a choice of gradient requires a new dropdown box on the front end, which in turn requires changing the gridy value of everything below. This all seems a bit archaic, but it can be researched another day.

The gradient selection needs to recorded for use in the promotion code, and then the code needs adjusting:

void raiseLevelnum(int oldLevel) {
int promoParam = 3 - gradIndex;
float paramRate = (float)(gradIndex);
float relegRate = 4*paramRate;
paramRate = 10*paramRate;
if (promoscore > promoParam && promoRate > paramRate) {
if (oldLevel < LevelMax) {
levelnum = oldLevel + 1;
} else {
levelnum = oldLevel;
}
resetPromo();
}
if (promoRate < relegRate) {
if (oldLevel > 1) {
levelnum = oldLevel - 1;
} else {
levelnum = 1;
}
resetPromo();
}
}

I have included in the adjustment a promotion rate as well as a promotion raw score, and I have added in a relegation rate. I have observed with children using the software that when you set an addition or subtraction item which is too hard for say a Year 2 or Year 3 student they will use their fingers to produce the correct answer, although it takes them a very long time. So using an incorrect answer to bring them back to the appropriate difficulty level is insufficient as an effective trigger.

Applying Theory to the Item Arrays

The high point, or perhaps end point of my theoretical thinking was 25 August 2009. It all went to shit after that. I was disappointed by the results of my further iterations, and by the imbecility of the contemporary local school teachers.

I don't know why I was obsessed with the idea of multiple iterations. A single adjustment of item difficulty to compensate for the ability of the students tackling them, and of student ability to compensate for the difficulty of the items they tackle seems, on reflection, more than adequate. So I shall now revisit the calculations I did 2 years ago and focus on the results of the first pass.

A couple of things occur to me as I run my eyes over the data again. The first is the size of the dataset, at around 15,000 records, it is much larger than I remember. The second is the depth of the item list, at around 380 items for addition alone. So there is room for many more than the five difficulty levels that I currently use.

I am not sure what got me into such a negative frame of mind. I dug myself into a catch 22 mindset. that I needed more data to make it better, and I needed to make it better to get more data. But in fact I was already sitting on enough data at least to make some improvement.

So now I am sorting the item list by numeric value of the left and right hand numbers, and observing the completeness of the set. The number 1 has been combined with the numbers 1 to 7. The number 2 combined with the numbers 1 to 6. The number 3 combined with the numbers 1 to 5. The number 4 was combined with the numbers 1 to 8. So there are some gaps, but they are quite narrow. Certainly I think the first step now is to use all the number combinations in the existing data, and then later the gaps can be filled. I shall also order the items exactly by the results in the data set, with no personal juggling.

One of the problems with never having documented what I did in the past, is that I have spent hours writing something to produce item arrays from sorted items, but I have now idea what it was or where is it.

My first thought was that the arrays would have been produced with a few lines of Java, but the folder, where I found text files populated with arrays, had no sign of any java code, and a hunt for source files containing the term array yielded nothing. I then found a spreadsheet, which had probably been used to produce the files, containing cuttings from an Access query. Eventually I found an Access database dated March 2009, six months prior to my attempts at systematic estimation of item difficulty. So the logical thing to do now is to cut and paste those queries into the September database.

The first problem is that the September database uses compound items, whereas the March one uses the individual terms. Note to self - why did I ever put the compound item into a database and thank goodness I've changed that. A second is that after spitting the dummy two years ago, I did no analysis of the operations other than addition.

But with a bit of juggling and a tiny bit of cheating, I produced four new item arrays. The cheating is not an issue, because this is just an opening array set. They will all be adjusted regularly as new data comes in.

Reducing Transaction Frequency

My niece was using the Applet last night, and in a couple of hours she generated a thousand lines of data, each one requiring it's own transaction with the database. A quick forum post confirmed that I need to modify the code so as to send more data with each transaction.

My first instinct was to upload data at the end of every activity, and I coded for that.

So the addItem3 method, which previously posted data after every item, was reduced down to accumulate a multiline SQL insert command in a new string , and the guts of it were put into a new addItem4 method, which posted the now multiline command:

private void addItem3(String newWord) {
if(firstPass) {
firstPass = false;
sqlbuffer = sqlbuffer + newWord;
}
else {
sqlbuffer = sqlbuffer + ", ";
sqlbuffer = sqlbuffer + newWord;
}
}

private void addItem4(String newWord) {
if(LIVE) {
sqlbuffer = sqlbuffer + ", ";
sqlbuffer = sqlbuffer + newWord;
firstPass = true;
if(jso != null )
try {
jso.call("updateWebPage", new String[] {sqlbuffer});
sqlbuffer = "";
}
catch (Exception ex) {
addItem2("jso call failed... ");
ex.printStackTrace();
}
}
}

But problems arose with posting 15 and 20 line inserts. So I modified the code again such that on longer activities, addItem4 gets called after 10 items, as well as on completion of the activity:

if ((oldItem == 10) && (NoOfItems > 10)) {
addItem4(qTrack.datInsert());
} else {
addItem3(qTrack.datInsert());
}

Then there was a typhoon, which prevented anything happening for a few days.

Fixing the PHP connection

I was so keen to minimize disruption to existing code that I forgot a couple of crucial steps. The first was to add the following line to public void init():

jso = JSObject.getWindow(this);

I also added the following diagnostic code to public void start():

if(jso != null )
{
addItem2("jso filled... ");
}

This enabled me to confirm that the jso object variable was filled as soon as the Applet opened.

The next necessary step was to tidy up the datInsert() function, which creates the SQL insert string, so that the string became:

ItemEntry = "INSERT INTO Items " +
"(Partid, OpCode, ItemLeft, ItemRight, Raw, Rate) " +
"VALUES (" +
sessTime + ", " +
OpCode + ", " +
ItemLeft + ", " +
ItemRight + ", " +
Raw + ", " +
SRate + ")";

The new variables ItemLeft and ItemRight were declared, but the old Itemdet was not removed, because it is still needed to populate a field. After that, with a few more comments thrown in here and there, the Applet ran.

Coding for a new database connection

When I worked for Mellon Bank, a favourite expression of my boss there was "If it ain't broke, don't fix it". I love that expression, and it really applies to what I am doing now.

Now I know you can work in the Collabnet working folder, but I prefer to copy files from there into another working folder, so I can just drag and drop from the original folder into another clean working folder when I stuff up. And because I am a little paranoid, the first thing I do is compile the code, just to make sure nothing strange has happened to prevent it doing so.

In the current version, the main class, AMJApp, makes a call to a function/method in my workhorse class QTrack.

The function call, which actually produces an argument for another local function is as follows:

addItem2(qTrack.datInsert());

And the function is:

public String datInsert() {
try {
ItemEntry = "INSERT INTO Items " +
"(Partid, OpCode, Itemdet, Raw, Rate) " +
"VALUES (" +
sessTime + ", " +
OpCode + ", '" +
Itemdet + "', " +
Raw + ", " +
SRate + ")";
smt.executeUpdate(ItemEntry);
buffer = "insert succeeds... ";
} catch(Exception ex) {
buffer = "SQLException: " + ex.getMessage();
}
return buffer;
}

Now, I want to minimize disruption, so the first thing I'll do is add another local function, which I shall call addItem3(String newWord). It is initially identical to addItem3(String newWord):

private void addItem3(String newWord) {
if(dbDEBUG) {
System.out.println(newWord);
jTADiagnosics.append(newWord);
}
}

I quickly recompile to see if this has destroyed the delicate balance. It hasn't yet, so I make another small change, making the function call:

addItem3(qTrack.datInsert());

I have to do this three times in different places in the code. And I recompile again, just to be safe. All OK so far.

My next change is a very subtle one, changing the value returned by the function in QTrack from buffer to ItemEntry. I recompile both classes again, and so far all is OK.

Now comes the crunchy part. I've been putting it off for a long time because I know it will stuff everything up. I have to make the addItem3(String newWord) function do some work. This is the function which will pass the data line to PHP.

I shall borrow the code from a previous post in my Learning Java blog. This code includes a JSObject, and that is why it gets messy. The JSObject requires the import of netscape.javascript.JSObject, which in turn calls on a collection of classes which can be found in a java archive called plugin.jar, while my main class also calls on other classes. So the compile command is no longer a simple javac but rather:

javac -classpath "C:\Users\MJHIPP\Documents\Current\Java\CodeLocal\AMJ110920\plugin.jar";"." AMJApp.java

I recompile again just to be safe, and then to be ultra safe I simply add the import line before recompiling with the new command. There was a bit of fiddling around, because that class path is a bit of a fiddle, but that is why I like to recompile after every small step. I can cope with one error, but not a whole screen full.

The next small step is to add the JSObject, and I put it right at the beginning, with my other supporting class declarations. Another recompile and all is still OK. Finally I have to add code to the addItem3(String newWord) function:

private void addItem3(String newWord) {
if(LIVE) {
if(jso != null )
try {
jso.call("updateWebPage", new String[] {newWord});
}
catch (Exception ex) {
ex.printStackTrace();
}
}
}

The modified class compiles just fine, but I now realise a fatal flaw in my methodology. I complied at every step but I did not check to see that the modified Applet actually ran. I was therefore very disappointed when it did not.

After some discussion on the Java Applet development forum it seemed to have something to do with JDK versions. And after fiddling for a while with cross-compilation options, I removed everything to do with Java from my computer, installed JDK 6.27, and recompiled, and now the new Applet runs.

It's not posting data yet, but hopefully that is a small diagnostic problem, to be fixed another day.

Database modifications

I need to modify the structure of my database, because I noticed in my practice AJAX call to PHP, the "+" sign was getting lost. I'm sure I could fiddle around to get it back, but I'd lose generality and I don't need it. I already store an operation code in the database, so I simply need to record the numbers being operated on in their own fields.

Rather than changing the fields in an old table, I'll just create a new one. As this is an open source project, for future reference and replication, it seems cleaner to record a single create command, than to recall a dud one and a series of changes. The SQL command was:

CREATE TABLE Items(
Itemid INT NOT NULL AUTO_INCREMENT ,
Partid BIGINT,
OpCode SMALLINT,
ItemLeft SMALLINT,
ItemRight SMALLINT,
Raw SMALLINT,
Rate DOUBLE,
PRIMARY KEY ( Itemid )
)

I ran this with phpMyAdmin, because my ISP offers it, and it was quick and easy to do so.

I am now ready to make the coding changes in the Applet.

Review of current data insertion methods

Before changing the database and attempting to write new data insertion code, I need to refresh my memory of what the Applet currently does.

So in the second of four statutory Applet methods, public void start(), the Applet includes the following code:

if(LIVE) {
addItem(qTrack.dataDriv());
addItem(qTrack.dataConn());
}

I like this construct, and I shall be sorry to lose it. The theory behind the code is described in a post entitled Functions in my Learning Java blog.

Essentially the explicit purpose of the method is to return a string for display in the Applet, but the database connection code is written into the code of the method, and the string returned is either connection succeeds or connection fails, depending on the outcome.

Anyway sadly, it has to go, because under the new rules, the connection code has to be contained in a PHP file.

In the third statutory Applet method, public void stop(), is the following code:

if(LIVE) {
qTrack.dataClose();
}

This is just a void method, I guess because if the Applet is closing, there is nowhere to display any returned message.

Then in the core of the Applet code is the following:

addItem2(qTrack.datInsert());

This is another diagnostic display function, with data insertion code written into it, and it is run every time an item is completed. This is the code, which needs to be modified carefully to send a query to PHP, without disrupting too badly the functionality of the Applet.

CollabNet Subversion

CollabNet Subversion can be downloaded from the CollabNet Subversion Downloads page. The manual can be read or downloaded from here, although the manual should come with the product download, and in Windows is even accessible from the Program menu.

To access code from an externally hosted project, such as my Rasch Itembank Project, only the client software needs to be installed.

And although they are kind enough to put the manual in the Windows GUI menu, the program itself has to be run from the command line. There is no need to set a binaries path, and if you navigate to the folder where you want to store your working copies of the code files, there is no need to set a target path either.

I had logged into my project from a browser before running the checkout command, and I was not asked for authentication when I ran it. The command for me was:

C:\Users\MJHIPP\Documents\Current\Java\Code\110918>svn checkout https://svn.java.net/svn/rasch-itembank~svn

This put a complete copy of my working files, including subfolders, into my working folder, as shown below. A quick glance at the screenshot will also reveal a batch file, which I used to run the svn command, to save typing.

On this occasion I simply added a stable working copy to the tags folder and reuploaded. To add the copy I had to use the svn copy command:

C:\Users\MJHIPP\Documents\Current\Java\Code\110918\rasch-itembank~svn\trunk>svn copy client C:\Users\MJHIPP\Documents\Current\Java\Code\110918\rasch-itembank~svn\tags\Client110918

The basic syntax of the command is similar to DOS copy, except that you simply have to name a folder and all contents will be copied.

I then wanted to add a text file explaining the significance of the copy. Here I had to create the file first and then use svn add. So this command does not create files, but adds them to some sort of a list.

C:\Users\MJHIPP\Documents\Current\Java\Code\110918\rasch-itembank~svn\tags>svn add Client110918.txt

Before committing the changes I wanted to run svn status. Initially I tried to run the from the parent directory into which I had downloaded my working copy, but it actually had to be done from the root directory of the working copy:

C:\Users\MJHIPP\Documents\Current\Java\Code\110918\rasch-itembank~svn>svn status

and the report came back:

A tags\Client110918.txt
A + tags\Client110918

The svn update command confirmed that nothing had changed on the server while I was working. I then added a comment to the svn commit command:

C:\Users\MJHIPP\Documents\Current\Java\Code\110918\rasch-itembank~svn>svn commit -m "Added working copy to tags folder."

My web log in had timed out by the time I did this, so I was prompted to log in again on the command screen.

Authentication realm: Subversion Repository
Username: jhipp117
Password for 'jhipp117': ******
Adding tags\Client110918
Adding tags\Client110918.txt
Transmitting file data .
Committed revision 12.

Having done this, I am now free to start hacking away at the main code.

Introduction

This is the opening blog for the Rasch Itembank Project, owned by yours truly, and hosted very kindly by java.net. Much of the thinking behind the project has already been described in my Learning Java blog. Entries of interest might be my brief description of the Danish mathematician Georg Rasch, and some of the other entries in April 2009. A later entry, of pivotal theoretical importance to me and to the project, is entitled Scores versus scoring rates. This entry explains why I believe scoring rates are a more interesting and meaningful metric than dichotomous raw scores.

Later still, an entry in January 2010 explains my decision to go open source, and some of the subsequent entries go through the mechanics of posting and updating code to the host server.

This blog picks up just after Sun Corporation was taken over by Oracle, and all the java.net projects have been moved to a new server. I have not tried to modify my code since the move, so that will be one of my first tasks to document.