The scientific literature: dealing with signal-to-noise

It’s become a modern-day cliche that we have to cope with total information overload nearly all the time now: the Internet has so accelerated the pace of communication and innovation that it’s all we can do to keep up with what’s going on. Furthermore, while barriers to communication have come down, standards of quality have not simultaneously been raised — the effect being that we’re constantly exposed to endless amounts of crap as well as a very few nuggets of useful information. I’m sure this has been beaten to death by other bloggers, and I certainly don’t want to contribute too much to the problem, so I will confine myself only to one narrow aspect of it: keeping up with the scientific literature in one’s chosen field.

astro-ph: peer review or not?

Most fields have something like peer-reviewed journals, which subject each new piece of work clamoring for publication to rigorous review by one or more experts looking for the flaws in said work. This is a good and necessary thing for various reasons which belong in another entry. We astronomers, as well as physicists, mathematicians and other quantitative types, also have (the astronomy subsection is called “astro-ph”, to which I’ve already been linking liberally), an online repository where people can deposit pre-prints of their work, to give timely access to it before publication — or sometimes even before peer review is finished, such as when the article is first submitted. There are also many articles which are never submitted to a peer-reviewed journal, but are simply put on the archive as public resources — or, on occasion, to circumvent the peer-review quality control process.

As a result, one starts imposing other perceptual filters: Has the work been submitted for peer review? Who’s on the article — people who you know already write well and are known for careful work? How closely related is the article to what you’re already doing? That last one pains me a bit because I’d like to be able to branch out and read other things, especially since a lot of interesting science lies at the interfaces between established areas of research. This is, I suppose, why one goes to talks as well — and can then follow up on the topic later on. There have been a lot of good talks at Stromlo lately which I’d like to post about.

How to read a journal article

Even once a paper makes it through all that, it needs to be dealt with in a timely manner. The way in which one reads papers is therefore often seriously truncated: As an undergrad, as a grad student, and even in my first couple of postdocs, I was still trying to read articles end to end. This usually ended up in my giving up in exhaustion: Most scientific articles are not particularly gripping reads, nor are they evaluated on their literary quality (though the very best papers are of course well-written). Reading them end to end can be torturous, slow, and above all cause you to process a lot more information than you really have to.

The way I find many other scientists read is as follows: read the abstract, the introduction if you must, then the conclusion. Look through the article and find the plots, ensure that you understand the main features. Then dive into the details of the paper for anything else you need to have fleshed out. If you’re familiar with the area, you can probably process a paper in 15 minutes when it would take you an hour to read it end to end. If you’ve looked up the paper because you’re going to cite it in your own work, you don’t even usually need to do that much — just locating the piece of information causing you to cite it, along with any necessary qualifiers needed for context, is enough. I know I read things very differently, and much more quickly, when I’m doing targeted searches to answer particular questions I have, rather than generally trying to learn what’s going on — another reason going to talks is helpful.

Also, be prepared to quit early if you find you aren’t getting value of the paper. I felt motivated to write this whole rant, if you want to call it that, when I picked up an astro-ph article on a theoretical model for dark energy (I won’t link it here). These tend to be the worst, because dark energy is one of the least well-motivated topics to investigate theoretically, and there are zillions of ideas of all different stripes such that observations have very little traction to distinguish them yet. I stopped reading this particular one when I saw that the author was proposing to throw out Maxwell’s theory of electromagnetism (based, at heart, on 19th-century vector calculus) in favor of an “8-component bosonic spinor” or “split octonion” which suggested new structure for the photon. What this had to do with dark energy was not easy to see, but I saw no evidence in this article of any demonstration that the proposed modified electromagnetism reduced to Maxwell’s theory in the appropriate limit, nor that the idea didn’t carry other experimental implications besides explaining dark energy that weren’t already ruled out. Maxwell’s theory is a very well-tested theory. (To be sure, there was no sign that the paper had been submitted to a refereed journal either, so I might well have just left it alone.) I gave up on it after about five minutes.

This all works best, of course, if you’re familiar with the area, so that you can gloss over a lot of the introductory material and get to the good parts quickly. I still find I try to at least skim the article end to end, or read it thoroughly and take notes if I’m unfamiliar with the area in order to learn. But that process is rarely fun. Which leads me to point out that

Reading papers isn’t the only way to stay informed

I feel this is worth re-mentioning since I’ve already said it several times above: Reading the literature alone in your office is not the only way, nor even the most effective way, to stay informed. Going to talks and asking questions of the speaker, either during the talk or afterwards, is a very good and necessary thing to do. Bothering your colleagues at tea is another way. This is probably why they have so many occasions for interaction at Mt. Stromlo, and why “increasing interaction” is always high at the top of the list of suggestions for improving various research institutes. Of course, there really isn’t that much interaction at many top-flight places I’ve been at. I’m attributing this impression to the idea that many researchers there are simply so focused that they can’t be bothered to go to tea…

When talking to others, the advantage is that everything you learn is already passed through the perceptual filter of another accomplished scientist, so that the signal-to-noise is very high. This also presents a slight downside, of course, since if you’re looking to do something really new or crazy, many people may not like it nor will they think it will work. But that seems pretty easily manageable to me by sufficiently independent-minded people, and as scientists we’re all trained to form our own opinions as much as possible anyway.


About Richard

I'm an American scientist who is building a new life in Australia. This space will contain words about science and math, but also philosophy, policy, literature, my travels, occasional rants, all sorts of things I find strange and awesome. The views expressed in this blog do not necessarily reflect the opinions of my employer at the time (currently University of Sydney), though personally, I think they should.
This entry was posted in Astronomy. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s