Storing, publishing and sharing information – Complying with requirements efficiently

Extensions to the Freedom of Information Act that came into force from 1 September 2013 add to a raft of existing rules and incentives on councils to store, register and publish records. More than 50 councils met at the LGA on 1 October to review means of storing, publishing and sharing their information efficiently whilst adding value to information consumers.

Kent Country Council unveiled the records retention guidance they have developed by for every service in the Local Government Service List. The meeting also launched work lead by Peterborough City Council to standardise inventories of datasets and share schemas. Over the coming months the standard approach will be implemented in the London Borough of Redbridge's DataShare utility and in extensions to

Speaker details, a summary of each session, Power Point slides and videos of each presentation are given below. You can use this Knowledge Hub group to discuss standards for publishing local open datasets.

Information management – a story of sticks and carrots

Catriona Taylor of Peterborough City Council @CityPtbo described compliance 'sticks' for information management including recent additions from the Local Government Planning Act and the Protection of Freedoms Act. Cat quoted the Public sector information: market assessment from BIS as saying

"a minority of stakeholders have complained that datasets released by different PSIHs are not always easy to combine and work across, because of variations either in the content or the format. This is a particular issue with data produced by local authorities, with each local authority often adopting their own standards and procedures."


See presentation

Expectations of transparency and open data

Gesche Schmid @GescheSchmid, Programme Manager Transparency at the LGA described drivers for open government – both legislative and policy. She said that the pace of transition from closed to open should be lead by local demand. Gesche emphasised that use of standards allows linkages between datasets to understand the bigger picture.

Legal drivers are extensions to the Freedom of Information Act and the Inspire European directive on geospatial data. Government Policy applies to departments, not local authorities. The Department of Communities and Local Government Code of Practice for Transparency is currently recommended practice. Following consultation, the code of practice will be issued as what is likely to be a mandatory code but with minimum requirements. It is promised for this Autumn.

See presentation

Code and Guidance in publishing datasets under the Freedom of Information Act

Carl Wiper from the Information Commissioner's Office @ICOnews talked about the dataset amendments to the Freedom of Information Act. Amendments came into effect at the start of September 2013. The changes represent a move from an 'access' regime to a 're-use' regime. Re-use of datasets positively encouraged.

Datasets are defined as collections of raw data obtained or recorded by an authority carrying out its functions. Carl said that local government is in a good position compared with other sectors because esd-standards defines the services delivered to meet its functions linked to legal powers and duties.

See presentation

Outputs of Records Retention Project work and how they can be applied to your council. What services you must and can deliver and what records should be kept.

Elizabeth Barber, Records Manager at Kent County Council @Kent_cc accompanied by a member of the esd-toolkit team @esdtoolkit described the methodology used to construct generic records retention guidance for every service in the Local Government Service List for English councils. She explained that guidance also extends to internal services.

There is no 'Classification Scheme' because guidance is associated with each service and councils are left to group services according to the standard Function List, Navigation List or in their own chosen structure.

The second half of the video illustrates tools for subscribers to the LG Inform Plus (formerly called esd-toolkit) programme to browse, search and download guidance to help councils construct their own retention / destruction policies.

See presentation

During the lunch break the LG inform Plus Programme Manager, Tim Adams @DrTimAdams from the Local Government Association demonstrated the LG Inform and LG Inform Plus tools

Recording information assets, publishing datasets and sharing inventories with

Mike Thacker @MikeThacker from the esd-toolkit / LG Inform Plus programme introduced the afternoon session designed to explore practical ways in which inventories of datasets can be published and shared.

Inventories and aggregators of local dataHe described a model whereby council inventories in a standard format are harvested by and similar schemas and datasets from different councils can be identified and shared. Councils can implement the standards independently or use the DataShare product from Redbridge, which is being enhanced to apply the standard approach and rolled out to many councils.

See presentation

Overview of

Ross Jones @RossJones from the Cabinet Office team showed the catalogue and how to search and sort it. There is a map-based location search. The site has recently added lists of datasets from that are not published. Search can be limited to a single published, eg a local authority. Users include a mixture of end users and developers. Everything that can be done by a user can be done via an API.

Data quality is an issue. It is important to differentiate between documentation resources and (machine readable) data resources. Ross is keen that data complies with schemas so it is easier to process.

There are more than 1,000 publishing organisations and 14,000 datasets, of which approximately 4,000 are unpublished. There is an attempt to add 'themes' to standardise tagging of datasets.

Datasets can be manually input or harvested via automated means. We expect to implement harvesting for local authority datasets.

See presentation

Publishing open data via DataShare

Clive Davis @RedbridgeLive of the London Borough of Redbridge and John Wallace of CIPFA @CIPFA presented the DataShare product which is currently being used by Redbridge for all its open data. DataShare is being modified to support the standards discussed and automated harvesting by It is also being changed so that mapping uses OpenStreetMaps.

Clive explained that Redbridge's Chief Executive strongly promotes transparency and DataShare is their way of being transparent. DataShare lets the public view and extract data. The public can also submit their own ideas for datasets. Redbridge currently published about 150 datasets. Data within datasets can be filtered on values in one or more columns. The results can be downloaded. Some linked data is supported by building URIs into datasets, eg the URIs of companies in the Open Corporates data. Data with longitudes and latitudes can be mapped.

DataShare addresses the more generic FOI requests. It can make requests more specific. Reporting by 'clusters' within the council shows which clusters have published which datasets and encourages publishing.

CIPFA is providing assistance and support to up to 100 councils who wish to run their own instance of DataShare. John handed out copies of a CIPFA introduction to DataShare document which has his contact details for councils wishing to use DataShare.

See presentation

Workshop on mechanisms for sharing inventories/registries and what metadata is important

Paul Davidson @Paul_Davidson of the Local eGovernment Standards Body (LeGSB) and Sedgemoor District Council introduced the workshops with these three breakout sessions:

  • What Experience do we want for our customers? lead by Hendrik Grothuis @HendrikG of Cambridgeshire Online
  • What Metadata would enable that experience? Lead by Mike Thacker @MikeThacker of esd-toolkit
  • What are the priority datasets? Lead by Cat Taylor of Peterborough City Council

Paul gave a quick introduction to LeGSB, describing its mission as "To promote Standards for Efficiency, Transformation, and Transparency of Local Services". He summarised the extra challenges faced when promoting open data in local government with hundreds of councils publishing subsets of the same type of data.

Paul summarised the types of customer we are servicing with data, as described in the Registries and inventories for local data document. The first group breakout group looked at the needs of each customer type. Paul distinguished between a full inventory, individual datasets and different renditions of each published dataset. The second breakout group looked at the metadata describing an inventory, a dataset and a rendition.

The workshop conclusions, summarised in the video, are being fed into the work of the project lead by Peterborough. Peterborough will publish popular datasets using DataShare with well documented standards. Candidates for populate datasets to be published early are:

  • Spend data
  • Allowances and expenses
  • Grants
  • Contracts
  • Property assets
  • Election results
  • Job salary bandings

Common request are also for commercial business rates, parking information (charges and spaces) and details of empty properties.

Cat Taylor drew the day to a close by saying we are just starting the work and we need to collaborate on an ongoing basis to efficiently publish and share datasets. Cat warned that people will just issue FOI requests across all councils if they are not getting the data they want, irrespective of what individual councils think is the business case to publish.

See presentation

posted by Mike Thacker on 7th October 2013

Leave a comment

Leave a comment here (sign in or registration required)