Legislation.gov.uk, managed by The National Archives, provides an essential – and free – public service. Millions of people use it to find and access the legislation they need, lawyers and non-lawyers alike.

Bringing legislation.gov.uk up to date

Quick, easy to navigate and use, with an advanced timeline feature so you can see how legislation has changed, there has been one major snag with legislation.gov.uk: the revised versions of the legislation, which shows how it has been amended, is out of date. The National Archives has been working towards meeting the target that all of the primary legislation on legislation.gov.uk is up-to-date by the end of 2015.

How to catch-up quickly, applying thousands of amendments? Our solution has involved thinking differently about how we update, or revise, legislation, as well as exploiting the potential of open data.

The revision process

The first change was to rethink the complex process of revising legislation. This involves reading every piece of legislation to identify amendments, researching all the information about the amendments, such as when and where it has been brought into force, then applying any textual changes, complete with an annotation that explains the authority for the amendment.

What’s different? We used to think we were maintaining a database of documents. Now we are maintaining a database of amendments, from which we can derive the documents, showing how the legislation has changed. It is a completely different mind-set. It means, instead of researching each amendment just before applying it to the text, we research all of the amendments up front – creating a rich database of amendments that includes when and where they have come into force. It is a unique approach that has enabled us to vastly accelerate the update process, at the same time ensuring accuracy and consistency. It also means we can better utilise the people in our team, with the most complex tasks, for example around understanding commencement, being done by specialists, whilst more people can be involved in the simpler activity of applying the changes to the text.

Editorial tools

The second change was to build on the advanced technology that underpins legislation.gov.uk to develop online editorial tools that anyone can use, rather than using tools than can only be used by people at The National Archives.

Developing new editorial tools post-devolution means we have also been able to think through how we fully take into account the extent of the legislation – displaying where the law is different in different parts of the UK.

Expert participation

The third change involved benefitting from the potential of open data. By allowing other people to re-use the data held on legislation.gov.uk for their own purposes, those re-users then have a stake and an incentive to help us improve the data. The National Archives has an Expert Participation programme, where partners can contribute to revising the legislation on legislation.gov.uk. The expert participation process, involving review by a qualified review editor, ensures accuracy, with collaboration expanding the number of contributors who are updating legislation.

Whilst much of this work has been behind the scenes, the benefits will be increasingly visible on legislation.gov.uk, with a rapidly growing number of “green status” messages, indicating to the reader that they are looking at a current, up-to-date view.

We are always keen to collaborate. If you would like to join The National Archives’ Expert Participation programme as a partner or individual volunteer please email us at legislation@nationalarchives.gsi.gov.uk.

Understanding patterns in the legal system

Advances in technology are opening up new opportunities, not only to improve access to up-to-date law, but also to deepen our understanding of the legal system. It would be perfectly possible to research legislation, for example, using so called big data technologies, if only the data, tools and methods for doing this were more readily available.

To kick start data-driven research into legislation The National Archives has led an Arts and Humanities Research Council funded project, called “Big Data for Law”. This has set out to create a legislation data research infrastructure; a new service to stimulate new types of legal research. As well as developing new technologies this has also involved discovering how best to equip researchers with the data, tools and methods needed to research the statute book as a whole system, using data analytics.

One application has been research into the architecture of legislation; the shape of the statute book. Borrowing ideas from software engineering and architecture, we have explored the concept of a pattern language for legislation, by identifying commonly occurring “design patterns” in law. These are commonly occurring legislative solutions that are repeated in different parts of the statute book. We have found patterns for “licensing”, “registers”, “disease control”, “regulators” and so on. Might these patterns provide a way to abstract legislation, mapping the statute book? We think so. Finding patterns, and developing a pattern language to describe them, creates a shared common vocabulary that can be used by experts and non-experts alike. For example, can we support policy makers not trained in law to think more clearly about available legislative solutions, or help readers to more easily understand legislative intent? Or from The National Archives’ perspective, to better contextualise legislation to lay-users when presenting it online at legislation.gov.uk? Again, we think so.

Easy ways of interrogating data

Is the legal research community ready and equipped to make good use of big data technologies? We discovered, through surveys and interviews, a clear ambition to carry out research across the statute book, but a general lack of confidence and capability to access, download, process and analyse raw data. How could we support researchers to carry out original research without requiring sophisticated programming skills in the niche technologies used for legislation data?

One solution is to develop a set of pre-packaged analyses that researchers can just find and use – a census of the statute book, as it were. Researchers can confidently quote and use this to support their research without having to analyse the data themselves. This provides a starting point, but most researchers want to conduct their own original research. To do this, we have set out to provide sophisticated, but easy- to-use, tools for interrogating the legislation as data.

The first tool we developed is a “words explorer”, which allows researchers to enter a word or sequence of words they are interested in and see how frequently this occurs over time. For example, a phrase like ”˜to or in respect of’ is often used in pensions legislation as shorthand for the requirement that money should be paid directly to a person on their behalf. Entering this phrase into the words explorer generates a graph that counts and plots its use over time. The underlying data is readily available in Excel, which is, overwhelmingly, the data analysis package most legal researchers felt confident about using. Researchers can also find the individual pieces of legislation that the phrase “to or in respect of” has occurred in and how frequently. Identifying when phrases like this have been used, how often and what usage trends are provides useful insights into the changing styles of legislative drafting.

Often, though, legal researchers have much more complex questions. They want to find legislation made under a particular power say, where there is also a requirement to consult. To support this kind of research we have developed a query language specifically for the statute book. Researchers can use the “query builder” tool to search for words or phrases that are in the same subsection, provision or Part. Before the tool was developed, building complex queries like this would have required in depth knowledge of X-Query and the Crown Legislation Markup Language.

These two simple but powerful tools equip researchers to carry out very different types of research, for the first time being able to explore and analyse the entire statute book. We are using data analytics to find new ways of describing and mapping the data to find and share good practice. We are using technology to drive new and quicker ways of updating the legislation on legislation.gov.uk. It is a new era with many new possibilities – and not just for the geeks but for lawyers too. Keep an eye out for the launch of research.legislation.gov.uk later in the year.

John Sheridan is Head of Legislation at The National Archives, and Principal Investigator on the Big Data for Law project.

Email jsheridan@nationalarchives.gsi.gov.uk. Twitter @johnlsheridan.