Wednesday, October 18, 2006

Duplicate Form Submissions

Introduction


In web applications, sometimes when a user submits a form and it takes very long for the form to be submitted, a user will tend to click on the submit button again. This can cause for dupes to appear in the application, e.g. submitting the same payment twice (sorry for my financial background...), uploading the same file twice, etc.


There are two ways of avoiding this. The first involves writing server side code that spawns threads that check whether the exact same request is submitted again. The problem with this approach is that it involves spawning threads inside the server, an undesirable and sometimes even disallowed practice (e.g. J2EE application server security policy disallows spawning threads).


The other solution, found on the web, is to use JavaScript to disable the buttons on submit and reenable them after e.g. 60 seconds using the setTimeout function. It is then impossible for the user to click on the button twice. I saw a problem with this however. What if a user wants to stop submitting by clicking the browser stop button? Obviously, after 60 seconds, the buttons will be reenabled again. But why wait for 60 seconds? The user clicked stop and now the submit button is disabled for a long period of time.


A solution would be to keep the button disabled as long as the page is loading. If the user clicked the browser stop button, reenable the submit button again. Variations on this could include showing message boxes warning the user.

The Client


Testing revealed that using the onstop and onreadystatechanged events don't work, as they only fire from a page that has loaded, they do not fire when another page is loading. For this reason, the setInterval method is used to start a "checker process" that checks if the browser stopped loading the other page. Below is a simple HTML page that includes the basics of reenabling as soon as the user clicks stop:


<head>
<script language="JavaScript">
traps = 0;
timerId = 0;

function catchDC() {
if (traps > 0) {
return false;
} else {
timerId = setInterval("handleStop();", 100);
traps++;
document.forms[0].submit();
}
}

function handleStop() {
if (document.readyState == 'complete') {
traps = 0;
clearInterval(timerId);
}
}

</script>
</head>
<body onload="javascript: trap = 0;">
<form method="POST" action="http://127.0.0.1:9999/">
<div onclick="javascript: return catchDC();">Click</div>
</form>
</body>
</html>

In the above code, a fictional server (sample code below) is called when the "Click" div is clicked. Function catchDC checks if the user had already clicked the "Click" button by checking if steps variable is greater than zero. If this is the case, the event is cancelled and the submit is not executed twice. One could also display a message box to the user when this happens, warning him not to do this.


If the form was not submitting, a timer is started with an interval of 100 msecs. This timer calls handleStop every 100 msecs, and this function checks the document.readyState to see if it is still loading or if the user had clicked the browser stop button (which would cause the readyState to be set back to complete).


If the user clicks the browser stop button, in the handleStop function called every 100 msecs the variable steps is set back to zero, essentially reenabling the "Click" div again. One could also set steps to a different value, which could then be used to determine if the user had clicked the div before and if so, issue a warning explaining the risk of duplicates and asking the user if he/ she really wants to submit again (are you sure? yes/ no...).

The Server


Below is a simple sample server that accepts a socket and then waits for ten seconds before closing the socket again. This will cause the web browser to stall, giving you enough time to test the double click (or triple or etc.) and to click the stop button and find out that this causes the "Click" button to work again. The server writes "Ding dong!" to stdout for every call made from the browser to the server:

package tst;

import java.io.IOException;
import java.net.ServerSocket;
import java.net.Socket;

public class Servertje extends Thread {

private Socket s = null;

public Servertje(Socket s) {
this.s = s;
}

/**
* Main
* @param args argh!
* @throws IOException on error
*/
public static void main(String[] args) throws IOException {
ServerSocket ss = new ServerSocket(9999);
while (true) {
Socket s = ss.accept();
Servertje servert = new Servertje(s);
servert.start();
}
}

public void run() {
try {
System.out.println("Ding dong!");
Thread.sleep(10000);
s.close();
} catch (InterruptedException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}

}

Friday, December 02, 2005

Getting a City from an Unstructured Address Field

I was asked a couple of questions about converting an Excel sheet into another format. Seems the target format had a city field, but the source format did not. The source only had an address field. This is not where the problems started tho, the problems started with the fact that the address field contained addresses from various sources, and that those addresses were completely unstructured. How do you parse completely unstructured data...

To give an example, the following could have been an address from the source Excel sheet:

10620 Jasper Avenue NW, Edmonton, AB T5J 2A3, Canada

Now this could have been easily parsed using a string tokenizer and getting the second token. However, in other cases an address could just as well look like (and no, I am not kidding!):

10620JASPERAVENUENWEDMONTONABT5J2A3CANADA

Now try and parse the city out of this one...

I ended up telling this person to use another field that was called branch and that this field contained a city name in a lot of cases, but not always (e.g. sometimes the branch was not a city name, but the name of a department). This would give us the best result possible, as trying to parse the unstructured address data would have been an impossible task.

But was it really impossible?

I kept thinking about the problem and I soon realized that maybe there was another approach. Instead of parsing the city out of the address field, this approach would scan the address field to see if it contained a city name from a table of known cities. E.g. if we had a list of cities like "Edmonton, Leduc, Nisku, Spruce Grove", we could take each city and see if we could find a match in the address field. There was only one problem: the sheet contained addresses from around the world. How do you find a complete list of all the known cities in the entire world, let alone all the known cities for one country... A colleague told me that there is such a list tho, available at some US geographical department. Googling around I found:

The National Geospatial-Intelligence Agency's GEOnet Names Server (GNS)

You can download a zip file for a country, which contains a text file with information of the cities of that country. For every record in that file, one could take the name of the city and then scan the address field for this name. If the city name was found in the address, then we known what city the address is in, we can map this city to the city field in the new format and we would all live happily ever after. Even though the amount of processing would be tremendous, it would probably yield a much more reliable result than just taking the value of the branch field. The processing can be reduced if the country is known (e.g. this Excel sheet I received contained addresses from India, so using the India city names file only would suffice).

Now I was getting all excited about how this would be a very interesting exercise and how it would be even more interesting to see the results. Unfortunately, using e.g. the Canada file, the following address in Edmonton:

2874 Calgary Trail Edmonton, Alberta, T6J 6V7 Canada

Would cause Outback Steakhouse in Edmonton to end up in Calgary...

Bummer...