The launch in July 2010 of legislation.gov.uk to little fanfare (there was no marketing budget!) was a significant landmark achievement, both in terms of official legislation and of the whole Berners Lee inspired concept in Government of “linked data” and the semantic web. And, like many aspects of public sector information, it raises enormous issues about data quality, our expectations and priorities.

Legislation.gov.uk for the first time brings together in a single view the “as enacted” statutes (previously published by HMSO), and the consolidated primary legislation contained in the Statute Law Database – all under the auspices of The National Archives (TNA).

The website has been vastly enhanced by the design – TNA have taken Government web experience and interface design to ever more sophisticated levels (as grateful users of the Censuses will testify).

“Cool” visuals of which legislation affects which administrations in the UK and a time travelling feature which enables the user to see statutes at different points in time, are only some of the features of this release.

There is also a new and sophisticated API (application programming interface) which gives access to the data structures and relationships embedded in this sophisticated database – the real promise of Linked data: URIs assigned down to paragraph level, and sufficient metadata to enable programmers to utilise the data in new applications.

Legislation.gov.uk is in many ways the very acme of Tim Berners Lee’s vision of the world and a significant credit to the legislation.gov.uk team which has laboured for many months in difficult budgetary circumstances to bring this off.

But does it fulfil those twin objectives espoused by the Prime Minister and the Government’s new Transparency Board viz in this case making the law more transparent and opening up the legal markets to innovation and competition through the release of public sector information in linked data format?

Transparency of the law

The nation’s statute book has long been criticised for being too opaque – criticisms which have not eroded in time. The current Statute Law Database enterprise has been going since the early 1990s. It has proved a mighty labour of Sisyphus – a boulder being continually rolled up the hill only for it to roll back down again. In 2006 the Government’s response to a House of Lords Select Committee report which recommended it be expanded to include secondary legislation (Statutory Instruments as well as Acts of Parliament) stated the obvious viz. that it was an “immediate priority to ensure that a fully revised and up to date version of the official statute book is delivered for use by the public” i.e. a plea to please finish the primary legislation first!

The consequences of the Statute book’s not being consolidated and up to date aren’t just for the nonexpert citizen: Government lawyers and the judiciary also fall foul of the lack of currency. Lord Justice Toulson put the case memorably in R v William Chambers [2008] EWCA Crim 2467, where a Government department pursued a prosecution using out of date statute information. He commented: “…the courts are in many cases unable to discover what the law is, or was at the date with which the court is concerned, and are entirely dependent on the parties for being able to inform them what were the relevant statutory provisions which the court has to apply”.

Legislation.gov.uk’s technical sophistication distracts from the bare facts that its consolidation of primary legislation is only complete (with some exceptions still) till the year 2002! Thereafter the proportion of consolidated primary legislation falls away (2006’s legislation is only 64 per cent complete, 2009’s only 47 per cent). Secondary legislation is excluded altogether though the Tables of Legislative Effects facility alerts users to the impact of subsequent changes and enables them to hunt these down.

But the lack of up-to-date consolidation – no fault of the Legislation.gov.uk team who have laboured valiantly on their Sisyphean task – must be a concern to those who harboured greater ambitions (not least Government and judiciary). It leaves access to an up to date and consolidated statute book in the hands of those who have invested in and deliver highly exclusive legal information services.

Opening up the market to innovation

Legal publishing has long been dominated by two huge duopolists (Reed Elsevier’s LexisNexis and Thomson Reuters) whose scale alone enables them to provide a consolidation of the mix of primary, secondary, case law which characterises our common law system. This has created “a two-tier justice system with only the very rich able to access the full consolidated law while those lawyers doing pro bono work are discriminated against” (The Times, May 2006).

Those duopolies remain in place – indeed subsidised by Government itself. If any efficiency expert were to look at what Government itself purchases (MoJ’s bill to the duopolists amounts to a seven figure sum), then what it spends in departments trying to consolidate their own secondary legislation (for example netregs.gov.uk within the Environment Agency), the resource might be found to enable the legislation.gov.uk team to complete their work!

Options for legislation.gov.uk

The legislation.gov.uk team have pointed to two pragmatic routes to overcoming their resource constraint. The first a methodology of prioritising updates by the volume of web requests. A Top 50 list (but excluding critical tax and finance bills) have been identified based on user demand (users in academia, the rest of the world, here in the UK?). But again the updating even for these oddly selected Acts is very incomplete (for example, the Companies Act of 2006 is only half complete).

The second route being mooted is to invoke the participation of others in the updating process – a sort of expert crowd-sourcing model. It’s an approach which has been lauded in the US by the Obama administration’s championing of the Peer to Patent project. There the critically important work of approving patent applications was speeded up by the use of experts in the field – a Wiki type model.

But the law of the land requires a degree of “authentication” unlike any other web resource and it is surely the responsibility of Government to ensure that the statute book is at least up to date. TNA could do with some help from the Law Commissioners, the legislators and the bench in focussing attention on other ways of rolling this particular boulder up the hill.

And Government might rationalise its spending around both producing and purchasing law information services, and then give the private sector some really up-to-date critical data on which to build reliable applications which actually do open up the market.

One year on

Recent developments of legislation.gov.uk merit attention and indeed congratulation.

First, there is the extension of the work on secondary legislation drafting. All SIs are now originated via a Word template in which is embedded the schema of the overall grand legislative linked data design. More significantly, at least one major department is engaging the services of the legislation.gov.uk team to help them consolidate and streamline their secondary legislation inheritance.

A second significant feature is a reaching out to legal publishers – previously kept at bay for good reasons of independence and procurement purity – for help in filling in gaps in the legislative corpus. Westlaw’s contribution of access to the originals of pre-1987 SIs is the first such co-operation between Government and the legal publishers and the team at legislation.gov.uk are keen that it be followed by other gap filling co-operations with publishers and other parties.

So hats off to the team at the National Archives (formerly OPSI, formerly and still by law HMSO – is branding by any chance an issue?). Though they might sometimes get more publicity for their undoubted semantic web/linked data technology prowess, their lasting and sturdy contributions are to building access to the entire corpus of law and integrating over time the process of law drafting and consolidation.

With metadata disciplines driven upwards via common tools and the pooling of resources, Donald Macrae’s original vision of the Statute Law Database becoming an aid to drafting may at last be coming closer, twenty years after he first championed the concept in the then Lord Chancellor’s Department.

Shane O’Neill is founder of Shane O’Neill Associates, a leading public sector information strategic consultancy. His private sector roles have included publisher for the British Library’s Catalogues, Managing Director of the former Library Association Publishing and Managing Director at TSO (The Stationery Office).

Email shane@shaneoneill.co.uk.