The danger of using only sources with recent coverage

… is well documented. Consider, for example, the tragic case of the JHU researcher who only searched Medline 1966 forward and so missed an association between the intervention in a study and lung toxicity that had been reported in earlier literature [1-2]. In biomedicine, there is a huge emphasis on recency – and for good reason, science moves fast. The cited half life (a measure of how far back citations generally go) is way shorter in medical fields than in say, math (which is always >10 years). The engineering databases that I use most frequently go back to 1898 and 1884. I also use Web of Science and Chem Abstracts which go back to the very early 1900s (~1908).

But anyhow, Biochembelle, re-tweeted by Scicurious, pointed to an editorial from Nature Reviews Microbiology [3] that says youngsters today aren’t getting the proper baseline literature because they’re relying on PubMed and Google Scholar. They cite the subject area of bacteriophage biology – developed well before the Medline era. Some researchers in this area have created their own bibliography of articles prior to PubMed, but they are concerned about losing access to the publications as they are moved out of the library to storage.

There are like a ton of things wrong with these statements. First, have they tried Biological Abstracts? As far as I can tell it goes back to at least 1917 (my parent institution has it stored because we have the online version, BIOSIS, and we have the backfile). Second, libraries typically don’t move journal runs off site unless they have the electronic equivalent or at least until they’ve shown that there’s very little if any usage. Many scholars wish more were moved off site – they get free scanning and electronic delivery on those articles instead of having to photocopy themselves! Libraries are also buying electronic backfiles – don’t assume that just because it’s old, we don’t have it online! In fact, some pre-1923 biology texts are freely available in the Biodiversity Heritage Library.

My points in a nutshell:

  • yes, it is very dangerous to rely on incomplete resources like GoogleScholar
  • yes, it is very dangerous to only use recent information
  • if you’re at a research institution, you don’t HAVE to rely on PubMed and GoogleScholar, you have access to other resources and it’s no one's fault but your own if you don’t ask your librarian what to use

[1] McLellan, F. (2001) 1966 and all that-when is a literature search done? Lancet 358(9282),646.doi:10.1016/S0140-6736(01)05826-3

[2] Ramsay, S.(2001) Johns Hopkins takes responsibility for volunteer's death. Lancet 358(9277), 213. doi:10.1016/S0140-6736(01)05449-6

[3] Raiders of the lost articles. Nature Reviews Microbiology 8, 610. doi:10.1038/nrmicro2435

12 responses so far

  • scicurious says:

    I think these are great points, but we have to first make sure that students learn that there IS something out there other than Pubmed, as a lot of students consider that to be the major repository, and may never really look elsewhere. I've never had a PI during my training ask me about using anything except Pubmed. I didn't even know Biological Abstracts EXISTED until just now. So I think it's really good, and really important, to get knowledge of other resources out there, and I'll be checking Biological Abstracts. 🙂

    • Christina Pikas says:

      I guess depending on the school, the professor, and the librarian, this sort of stuff should be covered in an information literacy class for undergrads. There *should* be some class for beginning grad students with a librarian providing information, but I don't know how often that happens (I know it does happen some because I know librarians who teach these sessions). My grad school, for example, doesn't have *any* presentations from our librarian to our students!

  • Dirk Hanson says:

    Any research more than about 15 years old can be (and often is) done all over again, because researchers often don't search the literature any farther back than that--the Dark Ages to young scientists.

  • Coturnix says:

    This can be quite embarrassing,'s a story:

    Several years ago I was at a conference, listening to talks in a small room packed with about 200 people, the Who'S'Who of the field. A young researcher (PhD student or postdoc, can't recall now) finished his intriguing talk and started ther Q&A session. A member of the audience asked him if the same phenomenon was resolved in another species (he named a particular species). The speaker mumbled something like "I don't know, I guess that was never studied in that species...."

    The reaction of the room was....interesting. The youngsters just nodded along. But the veterans (and obsessive-compulsive readers of historical literature, like me) gasped!!!! Right there, smack in the middle of the room, sat His Majesty Biggest Name in the Field, who did his Dissertation (and thus quite a few papers) answering exactly that question in that particular species. Yikes!

    All eyes turned to His Majesty Biggest Name in the Field as he slowly raised his hand. The moderator, of course, immediately gave him a nod. To his credit, he did not say "Erm, that was my dissertation", but instead said something like "Actually, those who are familiar with the older literature may be aware that this question was answered some, some decades (laughter) ago, and the answer is this..." and then he proceeded to gently explain the findings from his Dissertation without ever mentioning it was his. It looked like a gentle lecture from an Old Guy, but it was actually a huge embarrassment.

    • Christina Pikas says:

      That's my greatest fear - to have overlooked some critical piece of the literature. Of course, I'm more likely to be criticized for citing old work!

      • Coturnix says:

        Well, it is easy to miss a piece of literature. And people are usually perfectly understanding about it. But to miss some of the founding work of the entire field you are in? And especially if that work is VERY closely related to your own narrow research question? Very embarrassing!

  • Kele says:

    How does Web of Science compare to PubMed in this respect?

    • Christina Pikas says:

      Your institution could buy several different cuts of WoS - only back to the '70s, back to '45 or way back to the early 1900s. There's also a conference database that has to be purchased separately. Web of Science does not have the MeSH headings (human-assigned subject keywords) and it might not even have abstracts before a certain point. It does have citations so you can chain back that way. They're different but complementary databases

  • Rob Knop says:

    Of course, there are some of us who are not at a research institution, but at a smaller place with a smaller library budget, or nowhere.

    Fortunately for me, in astronomy, the predominant literature database is fully free access- It should be so in every field. After all, some fraction of every grant goes to pay for access to proprietary databases (even if only through overhead). Far more efficient (and ethical for science as a whole) for the agencies to directly fund an open access literature database.

    (A similar argument could be made for page charges.)

  • Isn't it time to start digitizing the backfiles in earnest?

    Seems like these bibliographies of gems are one place to start. In some fields, they're republished to bring more attention to them, and to push them into the electronic archive.

    @Coturnix Wow!

    • Christina Pikas says:

      Well - many/most (?) publishers are digitizing backfiles. They might be sold as a separate subscription (for an ASME title, 3 subscriptions!) or as a one time purchase. The one time purchases are great for end of the year money. Of course, we can't move our Inspec backfile to a different platform and that's a real bummer.

      Another confusing thing is when we have the backfile via, say, JSTOR, and then the publisher adds their own backfile (like Science). Researchers *think* we don't have access because it says so on the Science website. Sigh. If only they would check our catalog or open url resolver.
      This is another blog post 🙂

  • Jim Thomerson says:

    Back in the 80's and 90's we did a lab in introductory biology where we took the students to the library and did an exercise using Biological Abstracts. I've heard a chemist say it is easier to have a student repeat the experiment than to search the literature. I'm a fish taxonomist, so I have to be familiar with literature back to 1758. I once published a paper in a particular journal. About a year later, a paper on the same subject was published in the same journal, and my paper was not cited.