The behaviour of the crowd: a new era for appraisal?

Very interesting to see that the folks in charge of the UK Web Archive at the British Library are planning to adopt a crowd-sourcing approach to informing their selection of websites to archive


When I wrote Managing the Crowd back in 2008 I put forward the view that agreeing appraisal decisions largely on the basis of user opinion of the worth of a record was likely to be the only way to go. Here we not only see an example of this being attempted in practice, but an interesting explanation for the decision:

“we recognise that this manual selection process can sometimes be time consuming for frequent selectors. It’s also inevitably subjective, reflecting the interests of a relatively small number of selectors.”

This reflects two of the major tensions informing my own thinking on this back then (and now). That is that the kind of manual approaches to selection and appraisal traditionally adopted, ie manual processes undertaken by a relatively small number of trained professionals simply isn’t sustainable in the face of the ever growing onslaught of information being created. This will come as no surprise as an issue to any information professional even if solutions to it – this initiative aside - still seem rather thinner on the ground…

The admission of subjectivity within existing approaches to appraisal and the implication of ‘selection bias’ that underlies it is less widely discussed, perhaps because until now we’ve had no alternative. It stands to reason that despite our best efforts any selective appraisal process must inevitably be biased in some way, whether the appraiser is conscious of it or not. But having appraisal decisions based at least in part on user behaviour promises to go along way to resolving some of these issues. What would of course be fascinating would be a comparative study which compares the websites which would have been selected for capture by existing manual methods with those that are captured by the Web Archive according to ‘the crowd’ to go some way to seeing how closely (or not) the two are aligned.

A counter argument to all this may run along the lines of ‘but what if the sites the public are viewing most are not the most important ones.’ In short: ‘do the public really know best?’ Perhaps wisely the British Library are also incorporating “curatorial input to this approach, so we’ll be asking curators from the Library to assess the quality and relevance of resulting selections”. But it does pose and interesting question: should we be seeking to capture as accurately as possible the sites which the public believed to be of interest/use to a particular topic or those that we as information professionals believe they should have been interested in? The former of these may lead to the capture of some surprising, perhaps even ‘unsuitable’ sites, whilst the latter would perhaps provide a more informed, maybe a more ‘official’ version of events. But which would be the most accurate?

Its also interesting to note that this approach to crowd-sourcing isn’t just relying on user opinion but on the results of actual user behaviour. They aren’t just asking people to collectively vote for sites they wish to see included in the archive, but are analysing data from twitter regarding which sites were linked to at the time. Using user behaviour to inform appraisal wasn’t something I considered back in 2008 but have done quite of thinking about since, notably in a paper in the Records Management Journal in 2009 (Vol 19 No.2) titled ‘Forget electronic records management, its automated records management that we desperately need’. That is that we use the data about user behaviour generated by business systems (which records they have opened, whether they edited or just read it, what they looked at next etc) as the means to help inform our records management policies based not on what we believe to be organisational need, but on actual patterns of behaviour. This is something we are all familiar with through sites such as Amazon and their ‘users who looked at this item also looked at these items…’ functionality. Use of such ‘behavioural analytics’ is also gathering momentum within academia with institutions for example using library usage patterns to identify at an early stage which students may be disengaging from their studies. To my mind the ability to closely monitor and analyse user behaviour in this way has the potential to not only increase the scalability of much of records management but also to increase the level of sophistication in which it can operate.

Maybe there is hope yet.

Wisdom of the citizen?

Very interesting to see the new government initiative launched today by Nick Clegg which calls on the public to help decide which laws they want scrapped.

Now I should point out from the beginning that I'm not passing any political judgement here on its relative merit. What interested me was how the proposal reflects the whole 'wisdom of the crowd' ethos which as many of you will know I have long been advocating as a model for ensuring records management.

The idea is explained more fully in the Daily Telegraph and elsewhere, but in summary (and to quote from Nick Clegg)

"Today we are taking an unprecedented step. Based on the belief that it is people, not policy makers, who know best, we are asking the people of Britain to tell us how you want to see your freedom restored"

Certainly echoes there of my belief that the creators and users of records are often far better placed than Records Managers to understand their records and how we should be looking for innovative ways of extracting this knowledge and using it for management purposes.

And how is this to be done? To quote from Nick Clegg again:

"We are calling on you for your ideas on how to protect our hard-won liberties and repeal unnecessary laws... we're hoping for virtual mailbags full of suggestions. Every single one will be read, with the best put to Parliament"

Again, interesting to see an example of how technology can now be employed to gather and quantify information from a large cross-section of interested people and then used to inform the deliberations of those whose formal role it is to make such decisions.

Now whether this is just political gimmick or a genuine attempt at change is not for me to comment on. But as a high profile example of how technology now has the potential to empower individuals (be they 'citizens' or 'users') and how decision makers can and should now make use of such decision does seem worthy of comment.

A step closer to making Records Management2.0 a reality?

Readers of ‘Managing the Crowd’ will be aware that I ended the book with some suggestions regarding how records management functionality could be successfully integrated into and applied to Web2.0 content. One of these was to create “a folksonomy service that can penetrate ddep content, at an individual content level, across multiple service providers". I argued that the logic underpinning social bookmarking tools such as Delicious already took us many steps forward in this direction. Obviously at present they are only used for resource discovery but (I argued) there was no reason why this functionality could not be extended to also allow the individual user and the ‘crowd’ to which they belong to also assign retention and management criteria alongside search metadata.

At the time of writing the book the main factor preventing this from making the leap from theory to practice was that “at present, Delicious works at a level above that required for our purposes. It may allow users to tag individual web pages, but does not extend this functionality to enable tagging of ‘deep web’ content, for example documents within a Google Docs account and presentations within Spresent.” (pg 131).
Earlier this week I was sent the following email announcement being sent to all owners of lists hosted by the JISCmail service.

“From Tuesday 16th June, every list homepage and every posting stored on the JISCMail online archives will include a bookmark/share button which will have links to a selection of social bookmarking/sharing sites.

Social Bookmarking allows you to share, store, organise, search, tag and manage webpages you would like to be able to revisit in the future, or share with others. For example if a posting is made to a JISCMail list that you know will be of interest to someone else you can email a link to that person using our button. Alternatively you can choose one of the social networking sites you are registered with, e.g. Twitter or Facebook, to share the link with a group of people. You might use the sharing button to bookmark a link to your list homepage or a particular posting on a list that you can revisit at a later date on a site such as Delicious.”

So there we have it: it is now possible to extend the reach of social booking down to an individual file level (in this case the millions of emails archived on the JISCmail website). Now, of course there may be many other technical, professional or practical obstacles preventing the realisation of my idea but it seems to me that one potentially major one may just have disappeared…

Every cloud (computing) has a silver lining...

Interesting to read in the Financial Times yesterday that "the current economic crisis could force Web 2.0 into the business mainstream" as "A downturn raises questions about how organisations are structured and how they tap into the knowledge and expertise of employees, suppliers and customers". Indeed the mere fact that the FT sees Web2.0/Enterprise2.0 as a topic worthy of analysis is, in itself, evidence of its growing acceptance throughout the worlds of commerce and industry.

The piece also offers a nice summary of the perceived advantages of Enterprise 2.0 which provides organisations with most of what makes the Web 2.0 movement so exciting whilst also providing the reassurance of offering "audit trails, access control, version control, authentication, provisioning and back-up" (all words to warm the cockles of a records manager's heart).

As those who have read Managing the Crowd or heard me speak on this know I have always been a little sceptical about whether Enterprise 2.0 really is a sustainable longterm option or whether it will always be a compromise too far which ends up pleasing no one. I still have some concerns on this score, but, by the same token if it instead represents a temporary stepping stone towards a bolder future then that may be a different matter altogether

A spot of vanity publishing

There's been a small flurry of reviews of Managing the Crowd being published over the past couple of weeks. It's always fascinating (and a little daunting) reading what fellow professionals have to say about your work and I've been delighted - and relieved! - at the reception it has received. Thankfully the tone of most seems to be 'I don't necessarily agree with all you've had to say, but I'm really glad you said it' which is exactly the response I was hoping for.

Anyway, for those interested in hearing what others have made of it I can point you in the direction of reviews by:
- James Lappin in the Records Management Society Bulletin Issue 46, November 2008
- Barbara Reed and Stephen Clarke in the latest RMAA IQ magazine
- Marieke Guy in Ariadne (available online - hurrah!)

My thanks to all those who have taken the time and trouble to publish their thoughts and views about the book and in doing so are helping to keep debate about the issues well and truly alive

Apple's Genius points the way for RM2.0

In Chapter 9 of ‘Managing the Crowd’ I make the point that “Largely as a result of … technical advances, we now live in a world defined by information storage; ours is now a culture in which size most definitely matters. Just take each new generation of iPod that hits our shops. Have you ever seen an improved ability to decide what tracks you want to delete quoted as a selling point? No. The fact that it now has a 16Gb memory, compared with the 8Gb available last year, or the 4Gb the previous year, however, most definitely is”.

In this context it was interesting to read over the weekend that iPod will no longer be selling their 160GB iPod Classic. As the Daily Telegraph puts it, “ (perhaps) people have realised that, although the iPod has the potential to put their entire CD library in their pocket, they only ever listen to a few hundred favourite songs”. So maybe there are limits to our love affair with storage after all.

More interesting still, particularly in relation to Records Mgt 2.0, is the new ‘music-recommendation engine’ which the new Nano has built into it. According to the article by Claudine Beaumont “Genius will scan through your music collection looking at genres, the number of albums and songs you have by a particular artist, as well as the ratings you have given them. It will also look at the characteristics of the song itself, such as beats per minute… (it then) beams it back to the iTunes mothership. From there, it is able to build dynamic playlists of other recommended tracks, based not only on your library, but that of other iTunes users with similar tastes”.

So here we have the appraisal of large volumes of content based on information value and decided by a combination of both user opinion and user behaviour. Plus a system which combines the views and actions of the individual with those of the broader user community to provide a more informed analysis based on the ‘wisdom of the crowd’. Well whad’ya know. If we were talking about online business records, rather than music records, I’d say that Apple have just taken a pretty big stride towards realising Records Management 2.0.

An admirable summary of Managing the Crowd

I am indebted to James Lappin at TFPL for posting as succinct a summary of the main points contained within Managing the Crowd as you are ever likely to find.

In fact, it makes me wonder what I wasted the other 60,000-odd words on...

Launch of the Records Management 2.0 social networking site

Some of you may recall from previous postings that I have been keen to try to conitnue discussion and debate about the future of records management and in particular with regards to its role in the 'Web 2.0 world'; and hopefully to find ways of enabling those with an interest in this area to work together to create practical solutions.

After investigating a few different approaches I settled on creating a social network within Ning, not least because this platform combines the ability to keep in touch with like-minded colleagues with useful tools including online forums and event management. To be honest, I was still thinking through the details of how best to take this foward when a particularly interesting and robust debate about the future of records management on the Records-managment-uk jiscmail list convinced me that now was the right time to run with this (ready or not!). As with any such site, it will succeed or fail depending on the commitment of its members. Initial take up has been brisk, lets hope this burst of enthusiasm is sustained and turned into ongoing collaboration, discussion and progress.

If anyone else would like to join this site, please drop me an email and I will send you an email in return containing the relevant joining instructions.

Separating management from storage

I’ve recently been mulling over the nature of the relationship between where we store the information we create (the repository) and the rules governing its management – not least because it seems to represent one of the fundamental divides between the approach to records management which I am advocating (RM2.0 for shorthand) and the majority of ECM products on the market.

In the pre-Web 2.0 world there was a division between the applications we used to create information (e.g. MS Word) and the repository we used to store our outputs. We didn’t store our documents in Word, we stored them on our C://, on a separate file server, or even a removable storage device (though the MOD are beginning to wish they hadn’t!). All of which created a separate shared repository available for the storage of unstructured information created by a range of applications. This, in turn, influenced the nature of first EDRMS and latterly ECM technologies, the majority of which included their own separate repository for storage and intrinsically linked it to their management/rules layer.

As noted in Managing the Crowd: "The crucial difference with Web 2.0 services such as You Tube, Flickr, Facebook and the like is that they are content storage repositories as well. They no longer just represent the tools, but also the filing cabinet" which changes things considerably – especially when you consider that the majority of these services may be hosted outside the organisation. In this model there is no shared underpinning repository, nor is it possible to create and rely upon an intrinsic link between the management/rules layer and the repository of content we wish it to control. To my mind, this places those systems which are built upon the assumption of a combined repository and rules layer at a severe disadvantage by closing off the ability to manage information which it does not itself ‘physically’ hold.

One of the reasons for musing over this now was in the wake of an interesting chat I had the other day with some folks from Computer Associates marking the release of their new CA Records Manager product. In contrast to most other ECM products it apparently does not include its own integral repository – it’s a management layer only, managing content in its original native location. Though there are still currently limitations in terms of how widely this management layer can be applied (not yet extending to encompass the externally hosted Web2.0 services mentioned earlier) it seems to me to at least represent a more open-ended solution which at least offers the promise of achieving some of these wider, more demanding goals, further down the line.

The 10 (published) principles of Records Management 2.0

Finally, after many months of effort and angst, I have held a copy of my finished, published book in my hand. Authors are prone to equating the process of writing a book with that of pregnancy and giving birth to a child. Rather foolishly I tried this analogy earlier today to my wife who is currently 7 months pregnant….

Without sounding too clichéd I would genuinely love this book to represent the beginning, rather than the end, of this (self imposed) mission to rethink and reinvent records management to ensure it is fit for purpose in the modern world. I have been gratified to receive several supportive emails from around the world from other professionals who share my concerns and desire to initiate change. What I would love to do is to build on this by establishing a community of like-minded records managers, plus those from related professions such as the library world and of course the web technologists. I am sure that collectively there exists the expertise and range of skills required to make a genuine difference to our profession, its just a question of identifying the right online tool(s) to facilitate this creative discussion and, perhaps, finding the odd bit of funding to help make this happen. I make no apologies for the fact that my book raises far more questions than it answers, but now those questions have been raised and are out there for discussion lets move on to actually doing stuff: practical stuff that results in applications and approaches which can make a real difference. If anyone is interested in being part of such a community feel free to let me know, likewise if you are familiar with any ‘business models’ and/or technical platforms to help realise them (or sources of funding of course).

Finally, I thought I would end with some of the conclusions from the book (don’t worry, it doesn’t spoil the ending!). Part of my conclusion is that what we need at present is a set of guiding principles and shared characteristics which help define ‘Records Management 2.0’ and which can be used to set the parameters for any further development work in this area. More detail on each of them is given in the book and naturally they are all open for discussion (see Principle 9) but I thought they might at least get the debate started and were a fitting way to celebrate the arrival of the book.

Records Management 2.0 must be:
1. scalable to an (almost) infinite degree
2. comprehensive: with the potential to address all aspects of the management of information throughout its lifecycle
3. independent of specific hardware, software or physical location
4. extensible and able to absorb new priorities and responsibilities as they emerge
5. potentially applicable to all information
6. proportionate, flexible and capable of being applied to varying levels of quality and detail as required by the information in question
7. a benefits-led experience for users, that offers them a positive incentive to participate
8. marketable to end users, decision makers and stakeholders
9. self-critical and positively willing to embrace challenge and change
10. acceptable to, and driven by, the records management community and its practitioners

Managing the Web2.0 'crowd'

It seems that the Government are beginning to sit up and take note of the potential offered by Web2.0 – and not just from the technical perspective, but in recognition of the social, political and economic change that it promises as well. This is certainly the message which comes through from a recent speech given by Tom Watson MP, Minister for Transformational Government, on Monday 10th March 2008, where he said:

‘So let me tell you where I stand.
I believe in the power of mass collaboration.
I believe that as James Surowiecki says the many are smarter than the few.
I believe that the old hierarchies in which government policy is made and crucially for you in this room the way in which it is delivered – are going to change for ever.
People tell me that we are entering a post-bureaucratic age. I don't accept that. It's just old thinking – laissez faire ideas with a new badge.
The future of government is to provide tools for empowerment, not to sit back and hope that laissez-faire adhocracy will suffice. ‘

The April 2008 edition of Government Computing also devotes its front cover and lead story to Web2.0 and the need for government to "seize the day". In particular this piece points to the need for government to take a step back when it comes to implementing Web2.0 services and rather than going away and "spending a lot of civil service time and money trying to come up with web2.0 applications for themselves, it would be much better to allow more information to become public and allow groups like mySociety… to develop the applications". The piece also points to examples such as Patient Opinion of where sites set up independently from government have proved extremely useful in improving public services.

Interestingly, this piece does not mention any of the issues associated with the management of Web2.0 information. Of course management issues are never the most interesting and attention grabbing elements of a project and so often get consigned to the background, and this might be the case here. It may also be because the whole notion of imposing any form of management control appears out of kilter with the ethos of Web2.0 where, it is often assumed, anything goes – the equivalent of the parents spoiling the kid’s party.

But as our institutions and services increasingly begin to look towards a Web2.0-based future they are going to have to bite the bullet and tackle information management issues to ensure that such services are robust and reliable, as well as novel and democratic.

As I’ve mentioned in passing in previous posts, I have tried to develop this side of the debate by writing a book and I’m pleased (and relieved!) to say that the first draft is now with the publisher and due for release in June. The book is called 'Managing the crowd: Rethinking Records Management for the Web2.0 world' and is published by Facet Publishing. Probably the best way to provide a summary of the central argument of the book is to reproduce the content of the advertising flyer:

Imagine a records management (RM) future where the user community collectively describes the value and properties of a record using the wisdom of the crowd; where records retention, description and purpose are determined by their users, within general boundaries defined by the records manager. It may sound far-fetched, but could represent a way forward for managing records.

It has never been more apparent that RM as traditionally practised will soon no longer be fit for purpose. With the increasing plurality of information sources and systems within an organization, as the deluge of content increases, so the percentage of the organization’s holdings that can be formally classed as records declines.

In the Web 2.0 world new technology is continually changing the way users create and use information. RM must change its approach fundamentally if it is to have a role to play in this new world. This provocative new book challenges records managers to find time amidst the daily operational pressures to debate the larger issues thrown up by the new technological paradigm we are now entering, and the threat it poses to established theory and practice.

A range of stimulating ideas are put up for discussion: why not, for instance, embrace folksonomies rather than classification schemes and metadata schemas as the main means of resource discovery for unstructured data? Adopt a ranking system that encourages users to rate how useful they found content as part of the appraisal process? Let the content creator decide whether there should be any access restrictions on the content they have created?
This is a thought-provoking book which questions received wisdom and suggests radical new solutions to the very real issues RM faces. Every records manager needs to read this challenging book, and those that do may never think about their profession in quite the same way again.

I’ll post a few more details of the books central arguments over the next few weeks, but am happy to answer any specific individual questions directly.