Community Digital Archiving

Self-Sufficient Culture, Heritage and Free Software
You are currently browsing the Principles category

Upgrading community archive software

  • July 7, 2014 4:32 pm

One of the constant themes you will find on this web site is the concept of taking a long view regarding running a digital archive.  This truism is sometimes in conflict with the world in which it operates, the technological and digital world, which is driven by constant expectations of “upgrades”, “features”, “faster” and other implications of improvement.  In the consumer digital world, we are used to the short life spans of technology, but in the world of systems providing particular services, such as a community archive, change for the sake of change is not always welcome.

But we must live in the real world, and the reality is that, after a while, the developers no longer wish to support older systems.  This is fully understandable; if you have spent time improving your software and solving bugs, you don’t want to have to deal with those bugs in older versions when you have already fixed them in newer versions.  The Free Software world does give you the option of providing your own support, so you are not bound by the services your supplier wishes to offer.  But sooner or later, part of the longevity equation means keeping reasonably up to date.

In the case of the Assynt Community Digital Archive, we received an email at the beginning of 2014 pointing out that the version of DSpace that we were running was considered to be at the end of its life, and it was recommended that we upgrade.    We chose not to do that at the time for a variety of reasons, but summer is a good time to work on the systems, as in communities like ours, a lot of voluntary effort happens in the slower-paced winter months.

One of the beauties of the way in which the Assynt Archive is implemented is that it uses the concept of virtualisation.  This type of technology, and the reasons for t suiting a community project so well, are explained elsewhere on this website.  Virtualisation allows one to run an entire system independently of the physical hardware that underlies it.  In the case of doing an upgrade, this means that it was possible to take a copy of the entire virtual machine, and work on that, such that the live Archive was not in any way at risk as part of the process.  It also means that, as you go through a complex update procedure, you can take “snapshots” along the way, so that any oopsies do not mean hours of wasted effort.

An update to something like DSpace is not always easy for lay people to understand,  Updating systems software such as DSpace is not like updating productivity software on a laptop or desktop, where it;s a case of inserting a CD or downloading a zip file, and clicking “Setup.exe” or an “Installer” icon.  DSpace needs a runtime framework and a build framework which consists of the Java runtime system and various other components.  In addition, it needs am industrial-strength database.  So it is the type of process that only suitably skilled people should undertake.

Another early design decision was to do the least amount of customisation possible, ideally restricting the customisation to a logo and naming.  This pays dividends when it comes to upgrading.  It means simply following the processes outlined for the upgrade: download and unpack the new version, build it using the supplied tools, make the required changes to the database, deploy the new system and start it all up.  The bits of customisation., if they are restricted to the minimum, need not affect that ideal process too much.

In our case, though, we needed to go through upgrades from version 1.7.2 to version 4.1.  As this is not advisable in one step, it meant carrying out the upgrade process from version 1.7.2 to version 1.8, then 1.8 to 3.0. then 3.0 to 4.1.  At each step, it is necessary to carry out a battery of tests to ensure that each step is working.  The skills involved are a mixture of Linux/Unix skills, some Java development skills, PostgreSQL database skills and some experience as to how these types of things work.  Verison 4.1 also required updates to the deployment system, tomcat, as well as the build mechanism, maven, and ideally to the database, PostgreSQL.  This meant that one step was also to upgrade the operating system running the virtual machine from Debian 6 “squeeze”, to Debian 7, “wheezy”.  Fortunately, this is a well documented and bullet-proof procedure.   But with all that work, we are now…. I nearly said “future-proof,” but maybe immediate-furture-proof would be more accurate – until the end of 2016 or maybe 2017 anyway, when we expect the next steps to be very similar to these.

The virtual machine can then be transferred back to the live Archive, and it will magically be running the new version.

Archiving Project Open Day

  • May 10, 2014 6:17 am

The Assynt Field Club, as reported here, set up a project to begin the process of archiving digitally a huge set of physical collections of sighting records and other information they have accumulated over many years.  Key to these collections is the work of two local residents, Pat and Ian Evans.  Through the North Highland Initiative, funds were made available to employ an archivist to carry out a pilot project.  The pilot seeks to establish whether the Assynt Community Digital Archive is a suitable an practical repository for digitised information of this kind, and whether the collections are amenable to such processing.

Avril Haines was appointed as the Archivist.  Her first responsibilities were working with Pat and Ian Evans, as well as the Filed Club chairman, Andy Summers, to work out what part of the collection to use for the pilot.  It had to be manageable but also representative section of the whole.  The reptile collection was chosen as well as the material and a full copy of Pat and Ian’s unique book, Flora of Assynt.

This project is now coming to an end.  To date, Avril has added about 400 records to the Assynt Community Digital Archive.  The quality of the metadata and the consistency of recording are especially noteworthy regarding this project.  Avril has also used the project to demonstrate to other groups in the area the value of the Archive as a ready-made legacy for their work, in the present as well as older material.

On Friday 9th May 2014, the Field Club had an Open Day to demonstrate to members and others in the community the work that this pilot has achieved.  Refreshments were provided by the newly re-opened Mission Café.

DSC_4111 Field Club Open Dat

The funders for this project were represented by Eilidh Todd.  Eilidh has been especially helpful in easing the administrative burden of such a project, which relies otherwise on voluntary commitments.   While Avril was the Archivist, the project relied on others, including Eilidh, Andy Summers, and, of course, Pat ad Ian Evans.

Avril will now complete the project, including writing a report to examine the project, document any specific learnings and use it to inform the development of  future such archival projects.  From the point of view of the Assynt Community Digital Archive, this has been a wonderful project, professionally completed, and further demonstrates the way in which community archives can drive direct as well as more oblique value in a community.

Contact Stevan for more information.

Assynt Community Archive – Home safe again

  • April 2, 2014 9:52 am

If you have read something of the history of the Assynt Community Digital Archive on this site or elsewhere, you will know that the Archive was set up as part of the project that brought the old Fishermen’s Mission building into community ownership as a bunkhouse and café in 2009, opening in 2011.  The Assynt Community Association, a long-standing umbrella body for various Assynt initiatives, took the lead in this, and the building is owned by that charity.  The Archive was given its own Archive Room upstairs in the Mission building, appropriately wired and equipped for its purpose.  A trading company was formed to run the café and bunkhouse, but late in 2013, the trading company ceased to trade.  It has now been wound up.

Now one of the issues that concerned us when the Archive was set up was to ensure that there was a complete legal separation between the trading entity and the long-term initiative which is the Archive, which was seen to be a possible issue as they both occupied the same building.  Further, the temptation must exist to see the Archive as a short term exploitable asset rather than the long term proposition with societal rather than purely economic value.  So effort went into ensuring a separation both legally and practically between the two.  This separation was sometime over-effective, with a perception developing that the trading company staff were not allowed to direct visitors to the Archive, though improved communication helped that issue.

When the trading company ceased to trade, the bills that were paid to keep the heat and power and connections on also stopped, causing the Archive some short term problems.  However, these were overcome over the winter, and now the Community Association is taking a different view on how to gain community value from the asset which is the building.  One of the effects of this was the possibility that the Archive may be required to move.  This is technically possible, and indeed, the initial design foresaw this eventuality, but of course, the question was where it could conceivably go.  Other building in Assynt were not suitable.  The upshot was that the Archive was asked to move from one location in the Mission building to another, which would easily have been possible.

But eventually the plans have worked out that there would no longer be a requirement to move, and a new trading company has been formed which will follow a slightly different business model.  We wish Assynt Community (Trading) Ltd all the very best for a long and fulfilling existence and we look forward to working with the trading company in future, though, of course, remaining separate.

The point of this is that it is very important to get the legal framework right when setting up a community archive.  It could easily have happened that the Archive may have had a legal connection to a trading company, exposing the archive to the vagaries of the market and the risks of commerce.

Omeka – “WordPress for Museums”

  • October 3, 2013 9:44 am

There are some principles for deploying a digital heritage collection which 25 years of IT experience and supporting technical concepts make self-evident.  Most of these are documented on this site, such as the idea of using virtualisation to separate out the various services that make up a digital collection, and the separation of the long-term storage of digital artefacts from their interpretation and display.  In our case, we use DSpace software along with its robust database (Postgresql) to ensure that our storage warehouse of data meets the requirement of a long term repository.  DSpace can, of course, simply be browsed for content, but from time to time, it may be desirable to raise a project to curate and interpret information contained in DSpace.  I have always thought the display and interpretation requirement is often a short-term one, in comparison. So to my mind, a content management system like WordPress or Drupal would be perfect for such interpretation, simply dipping into DSpace to select the conformation to display.

The Roy Rosenzweig Centre for History and New Media, though, has developed Omeka, specifically to bridge the gap between appropriately rigorous digital storage and displaying collections of material.  Omeka is developed and made available under the General Public Licence, so, as Free and Open Source Software, it provides future certainty to users, and the fact that it is free of cost makes it easy to deploy.   While Omeka can be used as the underlying repository as well, this may be best considered as appropriate for small collections.  It is clear, though, that the combination of DSpace for long term storage and Omeka for interpretation and deployment is a powerful one, which I hope to be exploring more in the near future.

Omeka is as easy to deploy as the blogging platform, WordPress and similarly uses a standard Linux (operating system), Apache (web server), Mysql (database) and PHP (development language) server. Similar to WordPress.com services, it is possible to use Omeka’s own servers at omeka.net to deploy your own site, but most DSpace users will be probably have their own capability to deploy Omeka,

For projects like the Assynt Community Digital Archive, where a decision has been taken to keep the main DSpace repository off the public internet, Omeka offers the possibility of a displaying an informational sub-set of data, particularly useful for public relations or marketing purposes, or to fulfil a possible requirement of funders.  Omeka also simplifies links with Zotero via a COinS plugin that generates bibliographic data, making adding bibliographic data to Zotero a click away. We have already discussed the usefulness of Zotero on this site, so deploying an Omeka site which makes it easy for browsers to populate a local Zotero database makes sense.

Omeka has an “Exhibits” plugin that allows one to use information from the collections Omeka contains to add interpretation.  It seems to me that this is a particularly useful part of Omeka, and provides helpful separation of data from presentation.  The “Exhibits” plugin seems particular useful to adding narrative to DSpace artefacts, but there will inevitably be an intervening step of getting DSpace information into Omeka which may be a clumsy way of managing the data.

There are some limitations or omissions to Omeka from my perspective. One would be that it would be good to have an Open Document Format viewer, allowing the display of uploaded ODF-format files.  A workaround is that Omeka can display Google Doc information, but handing over data to third-party commercial entities does not sit comfortably with me.  Another is that Omeka may be seen in some circumstances to fulfil the functionality of a fully-fledged storage repository like DSpace, so positioning it in a digital heritage project may be quite tricky (but on further thought, maybe that’s a strength of Omeka). From a technical perspective, it would also be good to have a degree of database independence, or at least to offer support for Postgresql as well as MySQL.

Omeka is clearly a strong addition to particular digital humanities project deployments.

ADDENDUM:- Patrick Murray-John, one of the developers of Omeka, has been in contact to point out that the latest version of Omeka has programming interfaces designed to make it easier to exchange data with systems like DSpace. This seems to show an understanding of their target audience, and as the development team clearly found this blog post, an active interest in their project, both of which are positive points for anyone contemplating an Omeka deployment.

More contribution to Assynt’s Archive promised.

  • July 28, 2013 11:07 am

There is a fascinating archaeological underwater project under way at the moment, run by Wessex Archaeology, the SAMPHIRE project.  Details can be obtained here – http://blogs.wessexarch.co.uk/samphire/  Having met the team at a communication event in Drumbeg a while ago, it was delightful to get an email from John McCarthy to say that the project would donate a copy of the report on a local dive, following information received from local divers, for inclusion in the Assynt Community Digital Archive.

Several aspects of this are significant:  One is that someone outwith Assynt recognises the importance of the Archive and is willing to make this contribution.  Another is that this is contemporary information (though it may deal with an historical event – we’ll have to wait to see) and such contemporary information is just as important as historical collections when one imagines community archives.  And finally, the presence of the Archive provides a natural repository for information that risks simply ending up in a box file on a shelf, perhaps not quite so bad in this case, but the point remains.

It’s a generous offer from John and will be well received.