Starting UKSG in a state of ... informed bewilderment

"The future is already here, it's just not evenly distributed." John Naughton (Observer columnist and OU academic) opens with this striking William Gibson quote, reminding us that you only discover new things if you know where to look and are willing to pay attention. We're in a state of "informed bewilderment", with no idea how the internet revolution will pan out, so should stop trying to predict the future, and pay attention to what's already here.

Dissolving value chains

This is either an exciting, fulfilling & rewarding time ... or a traumatic experience that is likely to be destructive to some well-established businesses (even industries). The internet is "a vast global machine for springing surprises ... a phenomenal enabler of disruption." (Read Barbara von Shewick's "Internet Architecture and Innovation", Jonathan Zittrain's "The Future of the internet - and how to stop it" or John's own "A brief history of the future"). In programming terms, "disruption is a feature, not a bug". It cuts out the middlemen that have been such an enduring feature of our economy - journalists, travel agents .. and librarians, and publishers? "The net dissolves value chains" - where once journalism and classified advertising had a happy marriage, now one is helped by the internet and the other is disrupted. It's impossible to predict when open access will overtake closed access in the scholarly ecosystem, but the direction of funding etc indicates clearly where we're headed, so we need to pay attention to the existing change.

An increasingly complex ecosystem

The scholarly ecosystem has grown complex in its proliferation (of publishers, authors, institutions etc), and "for a system to be viable, it has to match the complexity of its environment." But there's not a single organism in our ecosystem able to match the complexity of our environment. Complexity is the new reality, and complex systems are intrinsically unpredictable. The banking crisis warns us of the dangers of being dependent on a system so complex that few people understand it, a system that is too complex to be modelled, too complex to be understood.

The avalanche of data in science

Librarians' functions have traditionally been determined largely by the physical aspects of materials and their housing. Value and roles were clear in the print ecosystem, whereas now, many students don't visit the physical library. Teaching, scholarship and research increasingly take place in a digital environment; librarians will need to move to where the action is ("from place to space"). The traditional information skills will need overhauling. Cornell's "Seven Steps of the Research Process" starts with encyclopedias and catalogues and only fleetingly, in step 5, refers to finding "internet resources". This already doesn't reflect how students behave. We need to adjust to new realities in science, which is becoming more data intensive. John closes by quoting Alan Kay: "The best way to predict the future is to invent it."

UKSG Conference Summary

This is the summary I put together for my SAGE colleagues. I am reposting here in case it is of use. Comments on theme omissions welcome.

The 33rd Annual UKSG Conference was held in Edinburgh a couple of weeks ago, with a varied programme and over 850 attendees. Themes gleaned from the sessions and discussions I attended are summarised below. The very active twitter stream from the conference can be found here. I was amazed at the ability of delegates to listen, assimilate and then tweet or blog all at the same time!

Social Media

An increase in focus on Social Media and new ways of sharing information and research was very evident. There was a much higher engagement and understanding of newer communication channels. Interestingly, it was not younger researchers quoted as spending more time blogging or commenting online but more established researchers who have already secured a reputation and can afford to spend time in this way. They are contributing to knowledge and growth in their field but not just in the tradition of peer-reviewed high impact journals. An interesting development would be a unique researcher identifier such as ORCID which would help tie all researcher output together.

Researchblogging is interesting, its focus is on serious posts about peer reviewed academic research. It has over 1200 blogs and has doubled in the last year. Adam Bly from Researchblogging explained that more collaborative science and the creation of new knowledge has the scientific information industry running to keep up. We’ll be contacting them to ensure posts on Sage content can be linked back to the original research.

Open Access

OA feels like it is slowly gaining traction as a publishing model. The vibe from various sessions was less if than when. Tony Hirst was emphatic on the new openness of communication channels. Why use traditional journal articles to share ideas he asked? If he or his colleagues do decide to publish an article it will be in OA jnls. However, no clear wide-ranging path for institutions to fund OA was apparent yet.

Usage

JISC Collections have secured funding to progress with the JISC usage stats portal or “make it real”. We'll be contacted for our input, the prototype involved data from Elsevier, OUP and Springer. Its aim is to present usage data in user-friendly ways for librarians and include new ways to benchmark usage. There is not clarity on confidentiality issues and was no resolution proposed on the question of the confidential nature of usage data. It’s early days.

Big Deals, Value & Pricing

An interesting session on value had Ted Bergstrom from UCSB explaining that he is securing information about big deal pricing so he can publish information about outliers in the public domain. His reason: “as citizens of the academic community, we are interested in helping librarians to understand the dynamic economic problem that they face and aiding them in negotiating effectively with large publishers. We plan to release a collection of information and analyses that will serve this purpose.” See his Big Deal Contract Project page.

As in previous years the mood seemed to be that the Big Deal bubble must burst, as it is unsustainable for many institutions, but there was no clear way forward proposed still.

Carol Tenopir talked about developing real tools for librarians to demonstrate value of their collections. She pointed us to the ARL website for more information.

Jill emery from Texas talked about patron-driven ebook and journal article acquisition stating 'the age of the article is here'. She explained they need “aggregated article access”. She wants publishers to listen as they have to purchase “Just in time not just in case'.

Quality Metrics

Frustration with the reliance on the Impact Factor and the fact it ranks journals and not articles was apparent. There is a desire to produce new ranking metrics. the Australian Research Assessment is no longer using the Impact Factor we were told.

Pete Binfield (PLOS) ran a session on how they have introduced article-level metrics - it’s worth a look if you haven’t seen. They recognise they are very much at the beginning but are very keen to do whatever they can to help users decide which content is highly valued by the community. Also, these metrics not just about evaluation but to help users filter and discover articles of value.

Pete talked about how they have work to do on working out how to measure “influence”. It’s important to demonstrate influence beyond the scientific community. This ties in with our work to show the value of social science research. How do we ensure the research we publish is credited appropriately when it influences Government policy for instance?

As Hannah Whaley says on her blog: The discussion around these issues is healthy, as is the growing volume with which librarians and researchers are willing to speak them out loud. However these key themes are notable for representing problems, not solutions. It is clear that licensing models, researcher metrics, electronic and open access still have some way to evolve to meet the growing needs and expectations of the community.

Richard "call me Dimbleby" Gedye and the Four Horses of the Research Quality Apocalypse

We're not too thin on the ground for this morning's 9am session, which suggests either that the conference dinner was a washout (it wasn't, as witnessed by my slight queasiness and faint headache) or that this morning's session is big draw (it is - a new, Question-Time-style format for UKSG).

Jim Pringle from Thomson Reuters opens the show. We're talking about the value / quality of research, and the ways in which it can be measured. Jim suggests:

• attention (citations, but also more generally)
• aggregation (researcher -> institution, article -> journal)
• relation (links, related content, metadata - lots of data stored around the institution can be useful in assessing researcher value / impact).

The Impact Factor is "foundational" in this space, and "will continue to be the touchstone for the evaluation of journals for a great deal of time to come." But the area is a "playground for mathematicians, these days". Eigenfactor, for example (network-based factors) consider the journals as a node in a network, and look at relationships between different journals. These metrics show different results, in part due to the way in which different citations are weighted. Normalised aggregates enable us to compare collections across disciplines, and are arguably more flexible / useful.

Tools are emerging that make use of metadata relationships (funders, co-authors, patents, etc) to more powerfully and accurately measure value. "But metrics are only as good as the people who use them, and can support judgements but should never be the sole ground for judgements."

COUNTER's Peter Shepherd walks us through PIRUS2, a project that is developing a standard for reporting usage at the individual article level. Journal-level metrics may not be representative of individual articles and therefore author, and citation data is not adequate for measuring some fields. The H-Index is author-centred but can be biased towards older researchers. Overall reliance on any one metric is misleading, and distorts author behaviour.

Article-level usage is becoming more relevant because:

• more journal articles in repositories
• interest from authors and funding agencies
• online usage growing in credibility e.g. PLoS reporting, Knowledge Exchange
• increased practicality - COUNTER, SUSHI

Challenges include the widespread distribution of articles - dispersed usage data to be captured. PIRUS aims to create guidelines for creating and consolidating article-level usage reports. Ongoing issues are technical (scaling previous work), organisational (underpinning business models), economic (allocating costs among stakeholders) and political (engaging stakeholders). Peter closes by quoting Joel Best - statistics are not magical, and need to be considered in context.

Alain Peyraube from CNRS picks up the baton, talking about the European Reference Index for the Humanities project. "We need appropriate tools (for measuring impact) or we are going to lose valuable grants." In the humanities in particular, such a methodology needs to address monographs, book chapters, edited volumes etc as well as journals. The ERIH steering committee took some time to set up the project, identifying which disciplines would be considered (lots of overlap with social sciences), selecting peer review as "the only practicable method of evaluation in basic research", setting up panels for each of 15 disciplines, providing guidelines and soliciting categorised lists of journals. The project suffered from considerable misunderstanding and criticism, especially from the UK scientific community, and particularly around the categorisation of journals. Both the panels and the journal lists have been revised since the project's inception.

Hugh Look takes the stage in a fabulous red blazer (Good Morning Campers!). Trying to attach numbers to things, says Hugh, is a difficult business - numeric targets are always open to abuse, and he points us to the LSE's Michael Power for a widespread analysis of this. What's the benefit of measuring - and who benefits? What are the underlying structures, and what behaviours do they engender? How strong is the link between measurements and real-world impact? We're not currently measuring performance or quality - we're setting up a structure of control, and this risks unproductive use of public money. "The managerial class are the primary beneficiaries of the measurement culture." They manage risk to the institution - but also the risk to themselves, so they use metrics to safeguard their own position / avoid bad PR, and coerce practitioners into compliance by making it part of annual reviews. There is a flight from judgement in the way that metrics are used. Managers cannot do peer review themselves, so they mistrust it.

Hugh acknowledges that measuring things can break "who you know" elitism, but both he and Peter referred to "fetishism" around statistics. Some organisations, e.g. the RAF, are stepping back from using metrics to assess research quality - a "dawning of common sense". Metrics divert attention away from "other things that are more useful to do" - "what aren't we doing?"

(Questions / floor discussion I will post separately, or I'll be verging on LiveSerials' longest posting).

Plenary session 2: Economics of Scholarly Information

Ted Bergstrom
Big Deals and the Terrible Fix

Ted started by describing librarians' “shopping problems"

• Delegation: Librarians are making choices for consumers who aren't spending their own money – 'the university' pays
• Unreliable signals: Arguments for subscriptions are fervid but not always reliable
• Complexity: Libraries face all or nothing complex deals, and when there are large numbers of titles covering 100+ disciplines, who can say what it's worth?
• Monopoly: Journals represent price inelastic demand, i.e. demand doesn't vary according to price (much like the prescription drug industry)

The first big deals entailed a publisher (roughly speaking) working out the library's current spend, multiplying by a factor, and then offering e-access to all content in addition to the print for existing subscriptions.

How effective was this? Very, since the publisher knows that the library is willing to pay at least same as they did for their paper subscriptions, and giving them additional content costs the publisher nothing. Bundling works well from a seller's perspective: demand for bundles varies less than individual titles, and deters entry to the market by making it hard to add new competing journals. And of course, by the time the deal expires, the faculty are addicted to online access so the library has to negotiate a new deal.

Ted then gave the audience an economic prescription: namely, if you want to allocate resources efficiently, use a price system where users pay for what they get out of their own money, and therefore economise, ie a pay per view model. If informed users spend their own money on downloads, demand becomes more price elastic, monopolists are forced to cut prices, and authors prefer to publish in reasonably priced journals in order to get more readers.

Does it work for academics? Ask yourself: who else is better placed to decide value of article access?

Currently there's a big discrepancy between the average cost of a non-profit and for-profit article. The competition prediction suggests that if users pay, it would drive down prices to just over cost-price. There could be limited role of central purchase, where libraries could subscribe to journals which cost no more than 1.5 times as much as the average non-profit journals and allow their users free access.

What can one library do?

• Consider dropping big deal subscriptions to overpriced journals
• Maintain subscriptions to reasonably priced journals, at zero cost to end users
• Subsidise user-pays models (not to their full cost though)

Marybeth Manning
Reset: a publisher's response to the changing economy

SPIE have taken the “unusual” step of reducing institutional subscription prices to their Digital Library by 10%, because they believed there would be sustained impact on scholarly publishing from changing publishing models and economic crises.

SPIE are a not for profit international society for optics and photonics: their Digital Library contains journals, conference proceedings and books. The subscription model is tiered (with different criteria for academic, government and corporate customers) and was originally based on print pricing.

Their objective is optimal dissemination of information, but it must be sustainable: a balance of reach and revenues. SPIE decided that their tier model was acceptable and understood, but penalised smaller institutions or large institutions with few relevant programmes. Corporations preferred price per use, and overall the cost per use for low tiers too high. Combined with environmental factors (the global economic crisis) they decided on overall price reductions on full DL subscriptions and topical segments (not including consortia or single subscriptions). They also added a 5th tier for the smallest organisations, and allowed introductory discounts to enable institutions to test interest. So far renewal levels have stayed strong, and they have good levels of new business, especially in lower tiers.

Does it apply to wider community?
SPIE wanted to shift their business model rather than have it shifted for them by factors outside their control, and believe that market growth and user demand will support reduced prices in future. They “want to be seen as part of the solution not the problem”.

But does this affect anything when big deals/mega publishers are so dominant? If libraries won't vote with their feet (or with their £) then who is to blame?

Carol Tenopir
University investments in the library: measuring the return

How does the library demonstrate its value to the university and its scholarship?
How can value be measured?

• implicit value – downloads, usage
• explicit value – testimonials, researchers' purpose in using library services
• derived values – Return on Investment (ROI)

Derived measures: A way to show that the library contributes to the income generating activity of the university ie for every £ spent on the library, the university received £ in return. One way is to look at money spent on collections and then money coming back in grants.

Grant cycle => conduct research – write articles – write reports and proposals – obtain grants – conduct research etc. Libraries have already been connected to all steps aside from obtaining grants: this is key unknown.

University of Illinois did a phase 1 study funded by Elsevier which found $4.38 grant income for each $1 invested in the library (based on the % of faculty who rated citations in proposals as important in their success x % of proposals funded/library budget). Their findings have been published in a white paper.

Phase 2 expanded the methodology from phase 1 across 8 institutions in 8 countries – does it still apply? (But remember that ROI numbers do not tell the full story)

Library ROIs fell into three distinct tiers:

• STM research institutions: up to 15.5 to 1
• Research/teaching institutions STM/Humanities/Social Sciences: up to 3.4 to 1 (most common category)
• Research and teaching all disciplines, not so much emphasis on external funding: less than 1 to 1

The project also did surveys of faculty about importance of citations and amount of reading, collected comments from faculty on use and importance of e-collections, and did interviews with university administration. They found lots of positive comments about value of e-resources in research, teaching, scholarship eg facilitates interdisciplinary work, better productivity.

Universities' administration staff also said they wanted the library to help them:

• attract/retain outstanding faculty (studies have shown a relationship between reading more library resources, publishing more, getting more grants: typical profile of 'star' faculty members)
• foster innovative research (bearing in mind that the number of articles cited will be much lower than those read)
• build research reputation of institution (some cases have shown positive relationship between library funding and amount of successful grants)
• promote seamless integration of the library with institutional research activities (some studies have shown a positive relationship between article downloads and research productivity)

Phase 3 (LibValue project) will be looking at more complex areas of library's value to teaching/learning, and social/professional activities. ARL will be involved in disseminating tools and measures. They recognise that they also need to look at new scholarly endeavours – e-science, collaborative scholarship, institutional repositories.

Conclusion

• It's possible to tie library e-collections to faculty productivity
• Libraries help generate grants income
• ROI for grants varies according to the mission and location of university
• Value can be measured in many ways

July issue of Serials now published

The Editorial Board of Serials thought it would be a good exercise to alert blog readers to the new issue of Serials which has just been published and contains many of the popular presentations from the recent conference (including T Scott Plutchak's and Melinda Kenneway's highly regarded papers). We've also included some of the breakout sessions, including Paul Harwood's briefing on the UK consortia landscape with a guide for librarians, publishers and intermediaries, and Nol Verhagen's briefing on "the licensing battlefield" from a consortium perspective.

Other papers concern the scholarly community in China, e-textbooks, the usage factors project, journal collection evaluation and much more..

Read it here (subscription or members only for current three issues) - http://uksg.metapress.com/openurl.asp?genre=issue&issn=0953-0460&volume=20&issue=2
Subscription information here: http://www.uksg.org/serials/subscribe

Let us know what you think!