I’ve mentioned before that I’m a big fan of WhatDoTheyKnow.com, the mySociety website which makes it easy for anyone to submit a Freedom of Information (FoI) request to any organisation which is obliged to respond to them.

One thing it doesn’t do, though, is give any kind of top-level overview of the requests being made, so you can’t, for example, easily see which authorities get the most requests or which are the best at answering them. So I decided to generate my own statistics.

Using a combination of the API and some judicious screenscraping, I’ve built up a database of FoI requests made through WDTK and then written a front end to display a selection of stats. You can see the results here.

Most of the stats should be fairly self-explanatory, but a little explanation of the methodology is probably in order. FoI requests at WDTK can have a number of different statuses, including “Successful”, “Partially Successful”, “Refused”, “Information not held”, Internal review” and “Waiting for response” as well as a few other, more esoteric, ones. To obtain the lists of most and least responsive authorities, I’ve added together all the statuses which indicates that a request is complete – that is, the information has either been supplied or a final reason for non-supply has been given – and called them “Completed” requests. I’ve then added together the figures for “Successful” and “Partially successful” (since, in most cases, the reason for most partially successful outcomes is that the requester has either asked for too much information or worded the request badly, which isn’t the fault of the authority) and calculated that as a percentage of completed requests. I’ve further limited this to authorities which have had at least 100 completed requests, in order to minimise the distortion caused by authorities which only have a few requests (or even just one) and a 100% success rate.

It’s pretty tight at the top, with less than 1% difference between the top three, but at the time of writing the most open authority in the UK is York City Council. In fact, it’s notable that all of the top ten, and most of the top one hundred, are local authorities. That’s almost certainly partly because local authorities get fewer of the unanswerable (and, frankly, stupid) requests that go to some central government departments, but, even so, I think this is something which reflects well on local government in the UK.

To get the list of least open authorities, I’ve taken the number of refused requests as a percentage of those complete. This makes more sense than simply taking the lowest number of successful or partially successful requests, since a request can be unsuccessful because the information is not available rather than because the authority refuses to give it. And, while some authorities undoubtedly do evade openness by falsely claiming not to hold information, on the whole most “not held” outcomes are the result of misdirected or badly worded requests.

I’ve also excluded the University of Salford from the list, despite the fact that, on the raw figures alone, they would be at the top. Unfortunately, the UoS has been a victim of a series of maliciously motivated FoI requests which all appear to stem from a single former employee with a grudge. The Information Commissioner’s Office (ICO) ruled that the UoS was entirely justified in refusing these requests on the grounds that they were vexatious, so it would be unfair to include them here. Excluding them takes the UoS below the 100 completed requests threshold for inclusion in the stats.

Unlike the most open authorities, here we have a clear winner (or should that be “loser”?). Standing more than 5% clear of the next worst, the least open authority in the UK is the BBC.

This is probably unsurprising, and some at the BBC would probably argue that it’s unfair, given that the BBC is permitted by law to refuse requests which relate to material held for journalistic or artistic reasons. But, reading through some of the requests which have been refused, I’m not at all convinced that this is being used as much more than a convenient loophole – just because you can refuse to supply some information, it doesn’t mean that you should. The second worst offender, Her Majesty’s Courts Service, has a much stronger justification for refusal in many cases as it gets a lot of requests which relate to specific, named individuals. These are generally barred from FoI disclosure under data protection rules.

Also, despite the generally good showing of local authorities, some do appear in the least open list as well. I wonder why Elmbridge Borough Council finds it necessary to reject 30% of requests?

The next two sets of stats on the page, for authorities with the most overall requests and authorities with the most currently unanswered requests, are pretty much what they say on the tin. They give an idea of the workload faced by authorities dealing with FoI requests.

The stats for authorities with the greatest percentage of “information not held” outcomes is titled “Most Ignorant Authorities”, but that really is unfair for many of them – particularly those at the top of the list. The Prime Minister’s Office, for example, gets huge numbers of requests that either should have been directed to a different department or are not FoI requests at all – they are just political rants and moans. So “Authorities who get the most requests from numpties” is a perfectly good alternative title!

Finally, the table for the “Tardiest Authorities” shows which currently have the largest percentage of overdue requests. Again, there are some very different reasons why authorities appear in this list. The presence of the UK Border Agency in second place, for example, is almost certainly a reflection of excessive workload and a shortage of staff rather than deliberate obstructionism. But it’s hard to see why there should be any excuse for the local authorities in the list.

Alongside the number-crunching, I also decided to use the data I’ve obtained to create my own interface directly to the requests themselves. While the principle behind WhatDoTheyKnow is absolutely brilliant, it has to be said that the user interface could do with some improvements. I’ve taken the liberty of creating an index to FoI requests which, in my opinion, addresses some of those weaknesses. I’d be interested in any comments on this.