FAQ

what is ImpactStory?

ImpactStory is an open-source, web-based tool that helps researchers explore and share the diverse impacts of all their research products—from traditional ones like journal articles, to emerging products like blog posts, datasets, and software. By helping researchers tell data-driven stories about their impacts, we're helping to build a new scholarly reward system that values and encourages web-native scholarship. We’re funded by the Alfred P. Sloan Foundation and incorporated as a nonprofit corporation.

ImpactStory delivers open metrics, with context, for diverse products:

who is it for?

how should it be used?

ImpactStory data can be:

how shouldn’t it be used?

Some of these issues relate to the early-development phase of ImpactStory, some reflect our early-understanding of altmetrics, and some are just common sense. ImpactStory reports shouldn't be used:

what do these number actually mean?

The short answer is: probably something useful, but we’re not sure what. We believe that dismissing the metrics as “buzz” is short-sited: surely people bookmark and download things for a reason. The long answer, as well as a lot more speculation on the long-term significance of tools like ImpactStory, can be found in the nascent scholarly literature on “altmetrics.”

The Altmetrics Manifesto is a good, easily-readable introduction to this literature. You can check out the shared altmetrics library on Mendeley for a growing list of relevant research.

terms of use

Due to agreements we have made with data providers, you may not scrape this website -- use the embed or download funtionality instead.

which metrics are measured?

Metrics are computed based on the following data sources (column names for CSV export are in parentheses):

where is the journal impact factor?

We do not include the Journal Impact Factor (or any similar proxy) on purpose. As has been repeatedly shown, the Impact Factor is not appropriate for judging the quality of individual research products. Individual article citations reflect much more about how useful papers actually were. Better yet are article-level metrics, as initiated by PLoS, in which we examine traces of impact beyond citation. ImpactStory broadens this approach to reflect product-level metrics, by inclusion of preprints, datasets, presentation slides, and other research output formats.

where is my other favourite metric?

We only include open metrics here, and so far only a selection of those. We welcome contributions of plugins. Write your own and tell us about it.

Not sure ImpactStory is your cup of tea? Check out these similar tools:

you're not getting all my citations!

We'd love to display citation information from Google Scholar and Thomson Reuter's Web of Science in ImpactStory, but sadly neither Google Scholar nor Web of Science allow us to do this. We're really pleased that Scopus has been more open with their data, allowing us to display their citation data on our website. PubMed and Crossref are exemplars of open data: we display their citation counts on our website, in ImpactStory widgets, and through our API. As more citation databases open up, we'll include their data as fully as we can.

Each source of citation data gathers citations in its own ways, with their own strengths and limitations. Web of Science gets citation counts by manually gathering citations from a relatively small set of "core" journals. Scopus and Google Scholar crawl a much more expansive set of publisher webpages, and Google also examines papers hosted elsewhere on the web. PubMed looks at the reference sections of papers in PubMed Central, and CrossRef by looking at the reference lists that they see. Google Scholar's scraping techniques and citation criteria are the most inclusive; the number of citations found by Google Scholar is typically the highest, though the least curated. A lot of folks have looked into the differences between citation counts from different providers, comparing Google Scholar, Scopus, and Web of Science and finding many differences; if you'd like to learn more, you might start with this article.

what are the current limitations of the system?

ImpactStory is in early development and has many limitations. Some of the ones we know about:

gathering IDs sometimes misses products

products are sometimes missing metrics

metrics sometimes have values that are too low

other

Tell us about bugs! @ImpactStory (or via email to team@impactstory.org)

is this data Open?

We’d like to make all of the data displayed by ImpactStory available under CC0. Unfortunately, the terms-of-use of most of the data sources don’t allow that. We're trying to figure out how to handle this.

An option to restrict the displayed reports to Fully Open metrics — those suitable for commercial use — is on the To Do list.

The ImpactStory software itself is fully open source under an MIT license. GitHub

who developed ImpactStory?

Concept originally hacked at the Beyond Impact Workshop, part of the Beyond Impact project funded by the Open Society Foundations (initial contributors). Here's the current team.

who funds ImpactStory?

Early development was done on personal time, plus some discretionary time while funded through DataONE (Heather Piwowar) and a UNC Royster Fellowship (Jason Priem).

In early 2012, ImpactStory was given £17,000 through the Beyond Impact project from the Open Society Foundation. As of May 2012, ImpactStory is funded through a $125k grant from the Alfred P. Sloan Foundation.

what have you learned?

how can I help?

this is so cool.

Thanks! We agree :)

You can help us. Demonstrating the value of ImpactStory is key to receiving future funding.

Buzz and testimonials will help. Tweet your reports. Blog, send email, and show off ImpactStory at your next group meeting to help spread the word.

Tell us how cool it is at @ImpactStory (or via email to team@impactstory.org) so we can consolidate the feedback.

I have a suggestion!

We want to hear it. Send it to us at @ImpactStory (or via email to team@impactstory.org).