Very interesting to see that the folks in charge of the UK Web Archive at the British Library are planning to adopt a crowd-sourcing approach to informing their selection of websites to archive
When I wrote Managing the Crowd back in 2008 I put forward the view that agreeing appraisal decisions largely on the basis of user opinion of the worth of a record was likely to be the only way to go. Here we not only see an example of this being attempted in practice, but an interesting explanation for the decision:
“we recognise that this manual selection process can sometimes be time consuming for frequent selectors. It’s also inevitably subjective, reflecting the interests of a relatively small number of selectors.”
This reflects two of the major tensions informing my own thinking on this back then (and now). That is that the kind of manual approaches to selection and appraisal traditionally adopted, ie manual processes undertaken by a relatively small number of trained professionals simply isn’t sustainable in the face of the ever growing onslaught of information being created. This will come as no surprise as an issue to any information professional even if solutions to it – this initiative aside - still seem rather thinner on the ground…
The admission of subjectivity within existing approaches to appraisal and the implication of ‘selection bias’ that underlies it is less widely discussed, perhaps because until now we’ve had no alternative. It stands to reason that despite our best efforts any selective appraisal process must inevitably be biased in some way, whether the appraiser is conscious of it or not. But having appraisal decisions based at least in part on user behaviour promises to go along way to resolving some of these issues. What would of course be fascinating would be a comparative study which compares the websites which would have been selected for capture by existing manual methods with those that are captured by the Web Archive according to ‘the crowd’ to go some way to seeing how closely (or not) the two are aligned.
A counter argument to all this may run along the lines of ‘but what if the sites the public are viewing most are not the most important ones.’ In short: ‘do the public really know best?’ Perhaps wisely the British Library are also incorporating “curatorial input to this approach, so we’ll be asking curators from the Library to assess the quality and relevance of resulting selections”. But it does pose and interesting question: should we be seeking to capture as accurately as possible the sites which the public believed to be of interest/use to a particular topic or those that we as information professionals believe they should have been interested in? The former of these may lead to the capture of some surprising, perhaps even ‘unsuitable’ sites, whilst the latter would perhaps provide a more informed, maybe a more ‘official’ version of events. But which would be the most accurate?
Its also interesting to note that this approach to crowd-sourcing isn’t just relying on user opinion but on the results of actual user behaviour. They aren’t just asking people to collectively vote for sites they wish to see included in the archive, but are analysing data from twitter regarding which sites were linked to at the time. Using user behaviour to inform appraisal wasn’t something I considered back in 2008 but have done quite of thinking about since, notably in a paper in the Records Management Journal in 2009 (Vol 19 No.2) titled ‘Forget electronic records management, its automated records management that we desperately need’. That is that we use the data about user behaviour generated by business systems (which records they have opened, whether they edited or just read it, what they looked at next etc) as the means to help inform our records management policies based not on what we believe to be organisational need, but on actual patterns of behaviour. This is something we are all familiar with through sites such as Amazon and their ‘users who looked at this item also looked at these items…’ functionality. Use of such ‘behavioural analytics’ is also gathering momentum within academia with institutions for example using library usage patterns to identify at an early stage which students may be disengaging from their studies. To my mind the ability to closely monitor and analyse user behaviour in this way has the potential to not only increase the scalability of much of records management but also to increase the level of sophistication in which it can operate.
Maybe there is hope yet.
Showing posts with label Records management journal. Show all posts
Showing posts with label Records management journal. Show all posts
Thursday, 8 December 2011
Thursday, 9 June 2011
Paying lip service to the user
Since my last post on the need to rebalance our approach to records management around the needs of the user I have been doing a little research of the RM literature on the subject. Admittedly this has been confined to the Records Management Journal (as a member of the Editorial Advisory Board I have easy access to it!) but looking through the back issues of 20 years for articles relating to RM system implementation has still proved an interesting exercise. I guess I should say at the outset (if not already obvious) that this was no formal literature review, but it at least helped contextualise some of my own thoughts, observations and anecdotatal evidence.
I found 7 papers relating to RM system implementation; most, if not all, it has to be said, relating specifically to EDRMS and I guess this focus has to be borne in mind when making any generalised assumptions. But that aside they nearly all make the point that ‘user buy-in’ is critical to the success of the project and that projects are largely doomed to failure without it.
There is also a remarkable consistency regarding how this can best be achieved: ‘Involve users early’, ‘’invite user representatives onto the project board’, ‘invite user representatives to try it out in a model office’, ‘give users plenty of training and support’ etc etc. All perfectly sound advice you might think and indeed it’s difficult to argue against any of them but at the same time I can’t shake the feeling that despite all these good intentions they are all missing the real point.
It’s a little like we are saying ‘when forcing a pedestrian to learn to drive a car its important to include them in the decision making process from the beginning; invite them to help choose what make and model; let them decide the colour; be sure to offer them plenty of training first and then be prepared to sit with them during their first few trips’. Great. But what if they didn’t actually want to drive a car in the first place? What if walking or taking the bus really suits their lifestyle? In short rather than simply asking them what colour car they would like, why not ask them how we could best improve their journey to work, however they decide to make it?
Biagio and Ibricu (RMJ Vol 18 No.3) inadvertently made an interesting point when they stated:
“users often fail to understand the corporate perspective of an EDRMS implementation and tend to remain focused on how the system will help them perform their jobs more efficiently and which tangible benefits and improvements it will bring”
Is it me, or do I detect a note of criticism of the user’s values here? Almost as if being censured for daring to put how they perform their own roles above what is of benefit to the organisation and its records managers. Well, welcome to the real world. Of course users are going to put their own working practice and working lives first. You’d have to be a pretty strange beast to volunteer to make your own work less efficient and less enjoyable just so someone else can benefit.
This question of just who the EDRMS is for also rears its head in another article by Ganesh Vednere (RMJ Vol 19, No.2) when he states:
“some people have been known to say ‘oh we are not that familiar with the technologies, so we let the technology team do the selection for us’ – well, that’s fine, but remember that ultimately it is the records management team that has to live with the platform”.
Now Ganesh was making a different point here about the need for records managers to get involved in the technical selection process rather than just delegating it to IT but what is striking here is the overt assumption that is the ‘records management team’ who have to live with the consequences of a poorly chosen system. A fair enough point, but what about the poor users? They don’t even get a mention. Doesn’t the same logic apply to them as well? Aren’t we asking them to ‘live with a platform’ that in all likelihood they had little or no input into choosing and configuring?
So my point? That the literature is absolutely right to highlight the importance of user engagement but that they seem to be fundamentally misjudging when in the process this needs to occur and the weight that needs to be placed upon their feelings. And that this process can probably best be started by replacing the question ‘how can we configure this system to try to meet your requirements? With ‘how can we help you do your job better?’
I found 7 papers relating to RM system implementation; most, if not all, it has to be said, relating specifically to EDRMS and I guess this focus has to be borne in mind when making any generalised assumptions. But that aside they nearly all make the point that ‘user buy-in’ is critical to the success of the project and that projects are largely doomed to failure without it.
There is also a remarkable consistency regarding how this can best be achieved: ‘Involve users early’, ‘’invite user representatives onto the project board’, ‘invite user representatives to try it out in a model office’, ‘give users plenty of training and support’ etc etc. All perfectly sound advice you might think and indeed it’s difficult to argue against any of them but at the same time I can’t shake the feeling that despite all these good intentions they are all missing the real point.
It’s a little like we are saying ‘when forcing a pedestrian to learn to drive a car its important to include them in the decision making process from the beginning; invite them to help choose what make and model; let them decide the colour; be sure to offer them plenty of training first and then be prepared to sit with them during their first few trips’. Great. But what if they didn’t actually want to drive a car in the first place? What if walking or taking the bus really suits their lifestyle? In short rather than simply asking them what colour car they would like, why not ask them how we could best improve their journey to work, however they decide to make it?
Biagio and Ibricu (RMJ Vol 18 No.3) inadvertently made an interesting point when they stated:
“users often fail to understand the corporate perspective of an EDRMS implementation and tend to remain focused on how the system will help them perform their jobs more efficiently and which tangible benefits and improvements it will bring”
Is it me, or do I detect a note of criticism of the user’s values here? Almost as if being censured for daring to put how they perform their own roles above what is of benefit to the organisation and its records managers. Well, welcome to the real world. Of course users are going to put their own working practice and working lives first. You’d have to be a pretty strange beast to volunteer to make your own work less efficient and less enjoyable just so someone else can benefit.
This question of just who the EDRMS is for also rears its head in another article by Ganesh Vednere (RMJ Vol 19, No.2) when he states:
“some people have been known to say ‘oh we are not that familiar with the technologies, so we let the technology team do the selection for us’ – well, that’s fine, but remember that ultimately it is the records management team that has to live with the platform”.
Now Ganesh was making a different point here about the need for records managers to get involved in the technical selection process rather than just delegating it to IT but what is striking here is the overt assumption that is the ‘records management team’ who have to live with the consequences of a poorly chosen system. A fair enough point, but what about the poor users? They don’t even get a mention. Doesn’t the same logic apply to them as well? Aren’t we asking them to ‘live with a platform’ that in all likelihood they had little or no input into choosing and configuring?
So my point? That the literature is absolutely right to highlight the importance of user engagement but that they seem to be fundamentally misjudging when in the process this needs to occur and the weight that needs to be placed upon their feelings. And that this process can probably best be started by replacing the question ‘how can we configure this system to try to meet your requirements? With ‘how can we help you do your job better?’
Monday, 6 December 2010
Records management: the plasterer's hammer?
“For a field largely reliant on the active participation of the individual users responsible for creating, using and managing records to achieve its aims, much of records management appears sorely lacking in the depth and sophistication of its knowledge about those same user s, their needs and objectives”.
So begins the conclusion of the paper written by myself and Jay Vidyarthi and published in the latest volume of the Records Management Journal. (Vol 20, No 3)
The paper discusses the way in which records management has focused almost exclusively and to the exclusion of virtually all other considerations on the needs of ‘the organisation’ often to the detriment of the users we are so reliant upon. Records management is a discipline which strives for standardisation, consistency and uniformity; for example in the form of functional classification schemes attempting to map activities across the entire organisation with a view to constructing a ‘corporate file plan’ or shared metadata schemas. This drive to standardisation isn’t just evident within organisations, but across them – be it in the guise of ISO 15489 or any of the specification standards for an EDRMS – all of which have at their heart the desired goal of uniformity of approach.
Read any section of 15489 and it’s abundantly clear who the main beneficiary of records management is intended to be – and its not the user. Virtually every section defines its objectives in terms of the benefits it will provide to ‘the organization’ with the user(s) getting barely a mention. Now none of this may strike the user as particularly surprising, nor in any way negative. After all, records management has long strived to be acknowledged as a ‘corporate function’ alongside HR, finance etc and clearly many of the drivers for it (accountability, governance, regulation etc) tend to apply at the organisational, rather than the individual level.
None of this is intended to criticise, but to shed some light on why it is that records management often struggles to satisfy the requirements of the individual users it relies on for success and why it could be argued that it has given up even trying. At its most extreme this disparity between the design of many records management systems and the needs of the individual user is most succinctly summed up in a quote made by one EDRMS user to me once that ‘making me use an EDRMS is like asking a plasterer to use a hammer’!
This clearly puts records management and the technology we rely on to implement it (whatever that technology may be) in something of a quandary. Is it really possible for it to successfully serve two equally demanding masters? Can we really hope to find ways of meeting the myriad, highly specific, highly personal demands of our user community in a way which not only pleases each individual user but also in a way which continues to meet the obligations and interests of the organisation as a whole?
Carry on as we are and I fear the answer will continue to be ‘no’; but open our eyes and ears to some radical new perspectives and it could yet be a ‘yes’. Human-Computer Interaction, or HCI is a combination of computer science, cognitive psychology, sociology, information science and design which might just represent the ‘missing piece of the puzzle’. A blog post doesn’t provide the space to explore the detail – that’ what Jay and I start to do in the RMJ paper. Here it suffices to describe it as a structured approach which puts the users first to ensure that they can interact with the system in ways which meet their needs whilst also continuing to meet the needs of the organisation. By shining a light on the behaviour, needs, opinions, tendencies and motivations of end-users it’s the first step towards achieving truly effective records management systems. After all, give somebody a tool that patently saves them time, energy and frustration and they would be foolish not to embrace it; but so too we must acknowledge that the reverse is true and that to try to make somebody use a tool that promises to only help someone (or something) else but at their own personal expense and surely we must concede that they would be a fool to use it.
The implications of such a shift in emphasis are profound, for records management as traditionally conceived is a house built from the top down determined by the needs of the organisation, and not one built from the bottom up based on the needs of its users. But it also offers some tantalising prospects: not just RM systems that users actively want to engage with, but also the possibility that we could start to use this new found knowledge of user behaviour to design and create records management systems that can actually manage records ‘automatically’ (at least in part) based on this behaviour – in a way similar to that used by Amazon et al to organise their content to aid the user experience. Desirable? Definitely. Possible? Who knows, but what this space…
So begins the conclusion of the paper written by myself and Jay Vidyarthi and published in the latest volume of the Records Management Journal. (Vol 20, No 3)
The paper discusses the way in which records management has focused almost exclusively and to the exclusion of virtually all other considerations on the needs of ‘the organisation’ often to the detriment of the users we are so reliant upon. Records management is a discipline which strives for standardisation, consistency and uniformity; for example in the form of functional classification schemes attempting to map activities across the entire organisation with a view to constructing a ‘corporate file plan’ or shared metadata schemas. This drive to standardisation isn’t just evident within organisations, but across them – be it in the guise of ISO 15489 or any of the specification standards for an EDRMS – all of which have at their heart the desired goal of uniformity of approach.
Read any section of 15489 and it’s abundantly clear who the main beneficiary of records management is intended to be – and its not the user. Virtually every section defines its objectives in terms of the benefits it will provide to ‘the organization’ with the user(s) getting barely a mention. Now none of this may strike the user as particularly surprising, nor in any way negative. After all, records management has long strived to be acknowledged as a ‘corporate function’ alongside HR, finance etc and clearly many of the drivers for it (accountability, governance, regulation etc) tend to apply at the organisational, rather than the individual level.
None of this is intended to criticise, but to shed some light on why it is that records management often struggles to satisfy the requirements of the individual users it relies on for success and why it could be argued that it has given up even trying. At its most extreme this disparity between the design of many records management systems and the needs of the individual user is most succinctly summed up in a quote made by one EDRMS user to me once that ‘making me use an EDRMS is like asking a plasterer to use a hammer’!
This clearly puts records management and the technology we rely on to implement it (whatever that technology may be) in something of a quandary. Is it really possible for it to successfully serve two equally demanding masters? Can we really hope to find ways of meeting the myriad, highly specific, highly personal demands of our user community in a way which not only pleases each individual user but also in a way which continues to meet the obligations and interests of the organisation as a whole?
Carry on as we are and I fear the answer will continue to be ‘no’; but open our eyes and ears to some radical new perspectives and it could yet be a ‘yes’. Human-Computer Interaction, or HCI is a combination of computer science, cognitive psychology, sociology, information science and design which might just represent the ‘missing piece of the puzzle’. A blog post doesn’t provide the space to explore the detail – that’ what Jay and I start to do in the RMJ paper. Here it suffices to describe it as a structured approach which puts the users first to ensure that they can interact with the system in ways which meet their needs whilst also continuing to meet the needs of the organisation. By shining a light on the behaviour, needs, opinions, tendencies and motivations of end-users it’s the first step towards achieving truly effective records management systems. After all, give somebody a tool that patently saves them time, energy and frustration and they would be foolish not to embrace it; but so too we must acknowledge that the reverse is true and that to try to make somebody use a tool that promises to only help someone (or something) else but at their own personal expense and surely we must concede that they would be a fool to use it.
The implications of such a shift in emphasis are profound, for records management as traditionally conceived is a house built from the top down determined by the needs of the organisation, and not one built from the bottom up based on the needs of its users. But it also offers some tantalising prospects: not just RM systems that users actively want to engage with, but also the possibility that we could start to use this new found knowledge of user behaviour to design and create records management systems that can actually manage records ‘automatically’ (at least in part) based on this behaviour – in a way similar to that used by Amazon et al to organise their content to aid the user experience. Desirable? Definitely. Possible? Who knows, but what this space…
Friday, 9 January 2009
New year, new predictions
It always fascinates me how the dawn of a new year seems to bring out the soothsayer in us, with all manner of commentators and observers from all walks of life taking the opportunity to predict what the year ahead will bring. Well this is a temptation that I intend to resist, but that doesn't stop me from being interested in the efforts of others.
One, in particular, has been brought to my attention my Andrew Warland, a fellow contributor to the Records Management2.0 Ning site and relates to 2009 being the year that the internet truly goes mobile
Coming after a year when data loss scandals were virtually a weekly occurrence it remains to be seen whether this hints at a solution (with less losable data being stored locally on remote devices) or a likely escalation of the problem. Hopefully the former...
I was also struck by the sentence "The brittleness of PC metaphors on mobile devices coupled with the concept of context speaks to an emergent and important trend on the horizon – smart and intuitive interfaces that predict user intent through an understanding of relationships", not least because I'm trying to find time at the moment to write a paper for the Records Management Journal entitled 'Forget Electronic Records Management, its Automated Records Management that we desperately need' which will say that this is exactly the route that the RM community also needs to be going down: using technology to intelligently assess both the information and its context and to automatically make management decisions accordingly.
One, in particular, has been brought to my attention my Andrew Warland, a fellow contributor to the Records Management2.0 Ning site and relates to 2009 being the year that the internet truly goes mobile
Coming after a year when data loss scandals were virtually a weekly occurrence it remains to be seen whether this hints at a solution (with less losable data being stored locally on remote devices) or a likely escalation of the problem. Hopefully the former...
I was also struck by the sentence "The brittleness of PC metaphors on mobile devices coupled with the concept of context speaks to an emergent and important trend on the horizon – smart and intuitive interfaces that predict user intent through an understanding of relationships", not least because I'm trying to find time at the moment to write a paper for the Records Management Journal entitled 'Forget Electronic Records Management, its Automated Records Management that we desperately need' which will say that this is exactly the route that the RM community also needs to be going down: using technology to intelligently assess both the information and its context and to automatically make management decisions accordingly.
Subscribe to:
Posts (Atom)