scio11: Alt-metrics

Jason Priem, Paul Groth, Jason Hoyt, Martin Fenner

The standard scholarly communication system – journal articles – and the shadow system like around water coolers, in journal clubs, in the hallways at conferences. Now that some of that is being done online, we can track and use it as a measure of impact.

Advantages:

  • fast
  • context
  • can look at a lot more scholarly products than just journal articles

Would you put the number of times a tweet of yours has been re-tweeted on your CV? Comments from the audience: can you make the altmetrics look official? I’m tenured so I can play, but I’d put it on my cv if a very obscure tutorial video got hits on youtube. Someone who was recently getting tenured didn’t put it in her package. Another comment – my university has been around for 150 years, but it only just now started allowing patents in the tenure package – I can’t imagine ever putting even more modern things in!

Comments on papers – can you get some sort of reputation or credit for that. Metrics are relevant to the currency of scholarship – in biochemistry one thing, but in filmmaking maybe film views

no sentiment analysis in citations – infamous vs. famous, but that’s always been the case. There are semantic annotation efforts.

There’s also the fruitbat problem (very popular article on Plos because – um – interesting topic) – maybe with multiple topics

There also has to be a temporal aspect to the metrics and you could annotate that to point to the news story that lead to the interest.

Many things we could measure, which to prioritize? Hoyt would prioritize whatever could displace the JIF.

An alt metric is to make available the details to recreate the experiment.

What tools? readermeter (takes Mendeley data). Profiles (add your own papers and then see statistics on them – can make your pre or post print pdf available)

impacts at conferences? Paul – program committee membership, slides from slideshares. people who use slideshare are not the people on the program committees.

citations – set up a research center, first papers from their research were 2 years later, citations quite a bit after that. Need some other metrics that will be faster to show impact of science to science and of outreach efforts.

Doesn’t always and shouldn’t always correlate with traditional measures because we do more things with articles

Paul has done some things to look at what your network looks like and what your publishing looks like over time.

Mendeley is looking at reading time, heatmaps (parts of papers where are highlights), correlation of readership with f1000 scores.

finding interdisciplinary metrics is very hard – bcs journals and assigning disciplines isn’t straightforward – these altmetrics can help

Tags:

Comments are off for this post