Internet Newsletter for Lawyers
Networking LIIs: how free access to law fits together
by Graham Greenleaf, Andrew Mowbray and Philip Chung, Co-Directors, AustLII
When did legal information systems first contain significant content from multiple countries? By the early1980s Lexis (now LexisNexis) had added UK and French content to its dial-up service. EURONET Diane also contained legal databases from a number of countries (Bing 1984).
By the mid-80s other content from Europe, New Zealand and Australia was available on LexisNexis. Today, this has expanded to cover content from 19 countries, under the LexisNexis ‘Global Legal’ section (previously less polite as ‘Non-US’). Thirteen are Commonwealth countries, and ‘Commonwealth’ is the only multi-country search provided. Provision of caselaw and legislation varies. Caselaw and legislation cannot be searched together so it is impossible to simultaneously search the whole of LexisNexis.
Westlaw International has gradually expanded since 2000 to cover caselaw and legislation from six jurisdictions: the United States, United Kingdom, the European Union, Australia, Hong Kong, and Canada (Westlaw 2007). The maximum scope of a simultaneous search is 10 databases, and you need to be able to select 13 to search everything they have. Cases and legislation can be searched simultaneously, with other content. The third potential commercial network has not yet happened. The Wolters Kluwer/CCH companies have content scattered on sites around the world (US, UK, CCH Asia, Australia, New Zealand, Canada etc) but no network or mega-site.
The LexisNexis and Westlaw International models were based on the model of one centralized location of data from multiple countries, and remain that way. In the mid-1980s there were attempts to establish another model: a dial-up network of collaborating commercial providers of legal information from multiple countries. For example, Australia’s then-CLIRS system (later bought by LexisNexis) tried to set up a network with the UK’s Eurolex and Canada’s QL, but this was scuttled when LexisNexis bought Eurolex, despite litigation challenging this (see Greenleaf, Mowbray and Lewis, 1988, p37). Such networks were in any event gateways at best, not allowing for simultaneous searching. During the CD-ROM era of legal research, from about 1988-95, it seems the emphasis was on circulating plastic, not building networks.
The Legal Information Institute started at Cornell in 1992 with a number of databases primarily of US federal law, and was the first significant source of free access law on the Internet. The Australasian Legal Information Institute (AustLII) was started by UNSW and UTS Law Schools in1995. It borrowed the ‘LII’ suffix from Cornell, as others have done since, but was the first LII to attempt to build a comprehensive free access national legal information system rivalling that of commercial publishers.
From 1999 AustLII started to use its search engine (Sino) and other software to assist organisations in other countries to establish LIIs with similar functionality and usually with academic roots: in 2000, the British & Irish Legal Information Institute (BAILII), based at the Institute of Advanced Legal Studies, London and University of Cork; in 2001, the Pacific Islands Legal Information Institute (PacLII) at the University of the South Pacific Faculty of Law for fourteen island countries of the Pacific (now 20); in 2002 the Hong Kong Legal Information Institute (HKLII) at Hong Kong University; the Southern African Legal Information Institute (SAFLII) with Wits University Law school in 2003, now operated by the South African Constitutional Court Trust for sixteen countries in southern and eastern Africa; and the New Zealand Legal Information Institute (NZLII) established by Otago University Law School in 2004. For all these LIIs, AustLII established the initial system and ran it for a while, with the aim of assisting full local take-over as soon as possible. Cylaw in Cyprus developed independently from 2004, but using AustLII’s search engine.
Meanwhile, the long-established LexUM team at the University of Montreal established the Canadian Legal Information Institute (CanLII) in 2000, initially by using the AustLII software but then developing their own. They also developed Juri Burkina (2003), Droit Francophone (2003), and are assisting developments in other countries.
BAILII and PacLII were multi-country regional systems from inception, but did not involve material from other LIIs. A linguistic rather than regional focus was taken by Droit Francophone (2003), with initial content concentrating on West and Central Africa, but some databases from across the francophonie. It is a single LII but now includes content derived from Juri Burkina. WorldLII at www.worldlii.org (2002) was the first multi-LII site, initially incorporating data from AustLII, BAILII, PacLII, HKLII and CanLII. Initially, the concordances for all the databases were located on the WorldLII server in Sydney, except that searches were sent to CanLII, and the returned results merged with the locally-produced results from the other LIIs. The latency of the CanLII results often meant that users had to presented results stating ‘CanLII results not available’.
Two multi-LII sub-portals followed. In 2005 AustLII developed the Commonwealth Legal Information Institute (CommonLII, 2005) for Commonwealth and Common Law countries (‘droit Anglophone’ is its nickname), which added databases from 20 new countries, but also relied upon the content of existing LIIs: AustLII, BAILII, CanLII, PacLII, HKLII, SAFLII and CyLaw. The Asian Legal Information Institute (AsianLII, 2006), drew on CommonLII, PacLII (for PNG) and HKLII, but also databases from 12 additional Asian countries. All of this fed back into WorldLII.
There are a lot more countries with databases in WorldLII than there are members of the Free Access to Law Movement. PacLII, Droit Francophone, CommonLII and AsianLII have incorporated into the LIIs the legal content of many organisations beyond their ‘home’ jurisdictions. How this is done varies a great deal. Sometimes the local involvement required is quite passive: a court or legislation website indicates that other are allowed to republish its content, or the copyright law of a jurisdiction states there is no copyright in legal documents (eg in Macau or India). In other cases obtaining republication rights requires requests and negotiation, but they are rarely refused if the data is already being published for free access. Otherwise, negotiating the supply of a steady stream of otherwise unobtainable data is sometimes very difficult particularly if a commercial provider is already obtaining it (eg Malaysian legislation). In other cases the institution concerned is pleased to obtain an opportunity to publish on the Internet (eg some Ugandan courts). PacLII and SAFLII have staff members who travel from country to country to obtain data not otherwise available. A discussion of the many complexities of data acquisition would be very lengthy. It is sufficient to say that the LII network has only scraped the surface of the valuable legal information which is potentially available.
The main constraining factor is of course funds. Every LII looks after the funding of its own system. The models on which LIIs are funded vary a great deal. AustLII has a ‘multi-stakeholder’ model, with more than 50 stakeholders in that case (see AustLII funding 2007). BAILII is similar. Most LIIs have had a considerable deal of academic funding and institutional support. CanLII is funded primarily by the Canadian legal profession. Aid agencies have made significant contributions to PacLII, SAFLII and AsianLII. Strategic alliances with some legal publishers have helped AustLII along. A small LII like CyLaw is a personal project. Yet others like NZLII live on ‘the smell of an oily rag’ and a little help from their friends, while they search for longer-term funds. There is no Daddy Warbucks (or even a George Soros) who would fund global free access to law long-term, but that doesn’t mean it can’t be done. It has been done with ever-widening scope for over a decade. There is not one formula, but as with many other aspects of open content, there are many non-business models by which numerous stakeholders can be engaged.
All of the LII’s operated by AustLII (WorldLII, CommonLII, AsianLII) share the same file structure on AustLII’s servers. For example, all Indian databases are located at …/in/, so www.asianlii.org/in/, www.commonlii.org/in/ and www.worldlii.org/in/ all address the same files. The structure is therefore simple enough to accommodate databases from any country in the world, and any number of different interfaces, without changes.
Other LIIs contribute less extensive replication facilities. The PacLII system is maintained in Vanuatu, but is mirrored in Sydney. Traffic originating from the Pacific (or more accurately, the USP network) is directed to the Vanuatu server, with the rest of the world defaulting to Sydney. Similarly, the SAFLII system is maintained in South Africa but is again mirrored in Sydney. At the moment, all traffic is directed to Sydney, but as larger servers are installed in Johannesburg, load balancing/ failover similar to PacLII will be deployed. BAILII and HKLII are also exploring replication options.
A geographically distributed network of independent LIIs poses significant technical challenges if it is to be robust and to ‘scale up’ successfully. Desirable attributes include replication of data sets across a global network of machines, with synchronisation approaching real-time; flexible load balancing and allocation of requests across the network; and failover access to local machines with replicated copies where the host LII is unavailable. As part of a project to enhance the robustness of the LII network, AustLII has developed a replacement DNS server known as the Load Balancing Daemon (‘LBD’). LBD is currently being deployed on an experimental basis, and is being gradually introduced across the free access to law network (Mowbray et al 2007).
The technical sophistication of the network, and corresponding collaboration between LIIs, has increased a lot recently, and augurs well for long-term robustness. A related challenge is to increase the interconnections between data on different LIIs, beyond common search results to direct links between data. CanLII’s Reflex (CanLII 2007) and work underway at other LIIs recognises the importance of citations to adding value both locally and across the LII network.
An on-the-horizon problem is the relationship between the LIIs and those who wish to build law-oriented services which attract large advertising revenues partly by spidering LII content, but do not bear any of the ongoing costs of building the quality legal information that the LIIs provide. The attitude of LIIs to spidering is an ongoing question, not only on privacy grounds.
The future for the LII network, its interfaces, and the Free Access to Law Movement is to increasingly provide a global alternative to the expanding global reach of the current legal publishing duopoly. In doing so, it can help to provide better access to law in many countries, developed and developing, and can encourage them to join in a global project that supports economic progress, the rule of law and democracy.
AustLII funding 2007 – ‘Funding Sources’ page www.austlii.edu.au/austlii/sponsors
Declaration 2007 Declaration on Free Access to Law’ 2002-07 www.worldlii.org/worldlii/declaration
LexisNexis (2007) ‘Company History‘ www.lexisnexis.com/presscenter/mediakit/history.asp
LVI Conferences (1997-2005) – ‘Law via the Internet Conferences’ www.austlii.edu.au/austlii/conference
Greenleaf G. ‘Global Legal Research: WorldLII and the Future’ Internet Newsletter for Lawyers, Jan/Feb 2005 www.venables.co.uk/n0501global.htm
Mowbray, A., Greenleaf, G., Chung, P. and Austin, D. (2007) ‘Improving stability and performance of an international network of free access legal information systems’ ARC eRsearch Seminar, Canberra, February 13-14, 2007, available at law.bepress.com/unswwps
Robots Exclusion (2007) – The Web Robots Pages www.robotstxt.org/wc/exclusion.html
Westlaw (2007) Westlaw International ‘Welcome’ page www.westlawinternational.com
Back to Contents.