Liveblogging Carol Tenopir’s keynote “Measuring the Value of the Academic Library: Return on Investment and Other Value Measures”
Carol reports that she did not participate in the Fun Run/Walk at 6:30 a.m.
We’re facing the challenge of demonstrating our value to stakeholders. Economy adds to this by pressuring budgets, combine w/ perceptions of library as gateway (Ithaka studies) we have a value gap. Amount spent vs. *perceived* value. (Perception of other roles hasn’t declined.) Wrong perceptions can become reality if we don’t address them head on.
Carol in middle of studies on ROI so that will be her focus today.
Usage logs: what people do on library systems. Picture of one segment – electronic; don’t tell you what people do with the info or the value it has to them. What did people do instead of going to the library? Carol recently joined COUNTER board.
Focus groups and surveys: to examine changes, to improve what we do.
User surveys and data: go beyond amount of use – what do people do with the info, what are outcomes. combine with other data on budget and income then you can get at ROI.
Objectives of ROI: how does library contribute to income of university. For every $ spent on library, university received X $ in return. Articulate value in terms of institutional objectives.
Study in 3 phases: 1) case study at a US University (U of Illinois) – there’s a white paper on the Elsevier website. Judy Luther was involved w/ team. ROI in grants was focus. 2) expanded Phase 1 to 8 countries, 9 universities. does the phase 1 methodology transfer? 3) propsal pending w/ IMLS to look at ROI for grants/research, teaching, student engagement – essentially go beyond grants to other ROI and quantify.
Findings from Phase 1: Not just quantitative, so need to meet with top admins to determine what they value. Found similar values at research universities, e.g., attract and retain top faculty, focus on new intellectual directions, strengthen interdisciplinary work, increase research impact. Benefit of interviews: informing admins about the library, too.
How do you calculate ROI: faculty generate income through grants, increase reputation. they use library collections in proposals. what proportion of grant’s income could be assigned to the library?
Grant cycle: conduct research -> write articles -> write reports and proposals -> obtain grants -> conduct research, etc. library has role in first three, how do we make specific connection?
Worked with an economist on methods. Need to clarify purpose of project before going out to talk to faculty: not trying to claim allocation back to library, not cost/time saving exercise. Faculty worried about money being allocated as an outcome of project.
studies on ROI for public libraries: Worth their weight by Americans for Libraries Council and Making cities stronger Urban Libraries Council. Include tools for calculation – this is a goal of Tenopir’s work.
studies on ROI for corporate libraries: Demonstrating Value and ROI by Outsell. time and money saved, revenue generated.
Data gathered during Phase 1: different types with different metnhods. e.g. surveys of faculty, office data on grant income, etc.
Model: % faculty w/ grants using citations x % grant award success rate using citations from library x $ avg grant income = $ avg grant income generated using citations from library x # grants expended / $ library budget (total not just collections) = $ grant income for each $1.00 invested in library. at Illinois this came out to 4+ : 1 ratio.
Phase 2 ongoing to see if model works, is it transferable. similarities and differences across countries and institutions.
Faculty survey – tried to keep short. combination of open-ended and quantitative questions. asked demographic data to determine differences among rank and discipline.
Value of e-resources: similar responses around the world – access from desktop is key.
Measuring up to admin values: tie between faculty with more pubs and citations have higher propensity of obtaining more grants. Faculty who publish more read more. Those who receive awards read more. Can’t claim cause and effect, but it is a picture of a successful faculty member.
References clearly important to grants. Avg # of citations: 20-46 is range.
% of citations from e-library: mode vaies 50-99%.
ROI varied from 15:1 to under1:1. varied depending on institution mission. some were teaching institutions so grants are not a good mechanism for measuring ROI.
Phase 3: will broaden focus from grants to other functional areas, e.g. teaching and learning. anticipate change: new scholarly endeavors such as e-science, IRs. challenge is to develop measures. e.g. how do you measure careers after graduation, measure prestige? and what’s the library’s role?
Conclusion so far: libraries help generate grant income. e-collections valued by faculty and bring ROI to university anywhere in world. hope to show library’s products and services help faculty be successful, students be successful, immediate and downstream income.
Final thoughts: tie what you measure to mission of university. measure outcomes not inputs. quantitative data show ROI and trends, qualitative data tell a story, multiple methods needed.
question about ILL: didn’t include in study – they were looking more at collections rather than services.
question about publishing phase 2: hoping this summer, but not sure.
question about calculator: yes, expect to make tools available with ARL.
question about how people know they’re using library systems: tried to phrase questions carefully at each institution in order to get at this. suspect they underestimated library use because people may not know, especially if the systems are good.
question pointing out that studies are exploratory and require further follow up. methodology evolved with phases. tenopir agrees.