The importance of the literature review

I came across a recent article entitled Dismissive reviews: academe's memory hole, focusing on the importance of knowing what's been done in the past and the consequences and prevalence of easy dismissal of prior research.

With a dismissive literature review, a researcher assures the public that no one has yet studied a topic or that very little has been done on it. ... firstness claims and dismissive reviews are far more common than they have any right to be. And, when false, they can be harmful. Dismissive reviews assure readers that no other research has been conducted on a topic, ergo, there is no reason to look for it.

The author points out a bunch of examples, noting "[p]erhaps we can empathize with an impoverished Ph.D. student cutting corners to meet a dissertation deadline. In the research fields I know best, however, dismissive reviews are popular with some of the most celebrated and rewarded scholars working at the most elite institutions."

His area of research is educational testing and he notes some of the consequences of the lack of knowledge of the past:

All the aforementioned statements dismissing the research on educational testing were uttered within several years of the 2000 presidential campaign, the only national election in our country’s history in which standardized testing was a major campaign issue. Thus, while the most far-reaching federal intervention in U.S. assessment policy—contained inside the No Child Left Behind (NCLB) Act—was being considered, the most influential research advisors for both major political parties managed to convince policy makers that no research existed to help guide them in their program design. That casually, a century’s worth of relevant research was declared nonexistent. The result? The research-uninformed NCLB Act. ... With the single exception of the federal mandate, there was no aspect of the NCLB accountability initiative that had not been tried and studied before. Every one of the NCLB Act’s failings was perfectly predictable, based on decades of prior experience and research. Moreover, there were better alternatives for every characteristic of the program that had also been tried and studied thoroughly by researchers in psychology, education, and program evaluation. Yet, policy makers were made aware of none of them.

What solutions does he propose? One of his suggestions is more care when writing literature reviews to note where exactly the authors went searching for prior work. One of the things that the author notes makes finding prior research difficult is simply the massive amounts of data being produced.

In 2008, 2.5 million Ph.D.’s resided in the United States alone. They, and others like them now fill more than 9,100 journals for over 2,200 publishers in approximately 230 disciplines from 78 countries, according to Journal Citation Reports. ProQuest Dissertations and Theses—Full Text “includes 2.7 million searchable citations to dissertations and theses from around the world from 1861 to the present day….More than 70,000 new full text dissertations and theses are added to the database each year.

I've managed to find prior work in an area that my research touchs on that multiple experts in that area weren't aware of simply by spending a day or two working my way through Google Scholar search results on related keywords and then briefly skimming possibly-relevant documents.

Another of his suggestions also sounds reasonable to me:

I believe that we should redefine the meaning of “a contribution” to research. Currently, original works are considered contributions, and quality literature reviews are not. But, what of the scholar who dismisses much of the research literature as nonexistent (or no good) each time he “contributes” an original work? That scholar is subtracting more from society’s working memory than adding. That scholar’s “value added” is negative.

Tearing people down

Random links

Hayley Wickenheiser one of first female characters in NHL video game
"EA Sports started including female players in NHL 12, which featured generic female characters. But this is the first time recognizable, real-world female athletes are a part of the made-in-Canada game. ... Wickenheiser ... became the first woman to record a point in a men’s professional hockey game with the Kirkkonummen Salamat of the Finnish second division. ... While the star forward may be featured in the NHL video game, Wickenheiser doesn’t think it’s a realistic goal to integrate female players into the league." Basically she seems to prefer a separate female-only league.
Internet for those who won’t get off their asses
"The northern town of Hoshaya is planning on installing WiFi Internet access on the donkeys it uses as part of its Talmudic-era amusement village, Kfar Kedem, Israel Hayom reported on Monday." Pictures here.
Nokia's Bad Call on Smartphones
The problems of getting research to market: "More than seven years before Apple Inc. rolled out the iPhone, the Nokia team showed a phone with a color touch screen set above a single button. The device was shown locating a restaurant, playing a racing game and ordering lipstick. In the late 1990s, Nokia secretly developed another alluring product: a tablet computer with a wireless connection and touch screen—all features today of the hot-selling Apple iPad. ... Consumers never saw either device. The gadgets were casualties of a corporate culture that lavished funds on research but squandered opportunities to bring the innovations it produced to market."
Abortion and the Gender Gap Continued
"surveys show that on average men and women have fairly similar attitudes toward abortion. Some analyses of this data show that when certain demographic factors are held constant, women are actually slightly more pro-life than men."

Should economists and other experts remain politically neutral?

Some thoughts on an op-ed written by a Boston University economics professor in response to Economists for Romney, a group of professional economists basically asserting that Romney's economic plan is better than Obama's. The article also notes Paul Krugman as basically doing the same thing for the democratic party. The author asserts that this is doing a disservice to economists's duty to objectivity and makes claims in excess of the available evidence.

It also notes a failed campaign for presidency on the part of the article's author. I have a feeling that the following might have something to do with why he lost:

Having run for president on the Americans Elect platform before it was disbanded in May, I may be questioned about my own impartiality in objecting to the politicization of economics. But in my short campaign, I limited my public statements and Web postings to the policies I favor, and I almost entirely avoided any criticisms of either the president’s or former governor’s positions. I did this because I was running as an economist, not as a politician.

Like it or not running as a politician seems to be a requirement to get elected and attack ads seem to be disliked but probably effective.

Should economists have to distance themselves from stating an informed opinion? That seems unreasonable to me. Given that I spend a lot of time around computers, e.g., if someone asks me about buying a computer am I allowed to assert either "buy a Mac" or "don't buy a Mac"?

Also interesting to note what the author of the op-ed considers to be objective:

why does the U.S. save and invest next to nothing? The answer is the income effect. Every administration starting with Dwight Eisenhower’s has expanded America’s Ponzi scheme, which takes resources from young savers (including those not yet born whose current spending is zero) and gives them to old spenders. This huge intergenerational redistribution has produced an enormous increase in the absolute and relative consumption of the elderly.
This is just what the life-cycle model of saving predicts. If you take money from the young and tell them they will get it back in spades when old, and then give this money to retirees, the following will, as a matter of theory and practice, happen: The elderly will go shopping, the young won’t bother saving, national consumption will rise, and domestic investment will fall.

In the US the likelihood of voting increases with age, meaning that those most likely to benefit from the current approach are also the most likely to vote. As a result, statements like the above about social security seem likely to ensure that you won't be elected, even if the effects noted are in fact true and might result in significant worldwide economic problems in the coming years.

(I haven't seem any current counterpart to Economists for Romney from Democrat-leaning economists but, based on an Economist survey during the last US presidential election, most economists then preferred Obama over McCain.)

Pages

Subscribe to Rotundus.com RSS