As reviewer of journal manuscripts and conference papers I normally look to see if the piece before me answers the following questions:
1. WHAT-WHY: What is the scientific problem at hand and why is important?
2. WHO-WHAT-WHY: Who proposed what previous solutions and why are these inadequate or incomplete?
3. WHAT-YOUR-WHY: What is your proposed solution and why is better?
4. HOW-WHAT: How is the solution implemented and what are the benefits or practical applications?
5. PROS-CONS-WHAT: What are the possible pros and cons of your solution and what are the next areas of research?
I reject papers that conform to any of the followings:
1. expired – to me, any paper with an expired deadline is an expired paper.
2. essays – to me, any paper without experimental data is an essay.
3. diversions – to me, any paper that deviated from editorial guidelines is a diversion.
I also ponder how much of the paper is hearsay or an opinion. If there is no experimental evidence to support these, then it get rejected.
Back in 2/10/2007 in How to Read an Engineering Research Paper Dr. William G. Griswold, CS Dept, University of San Diego wrote some great guidelines for students that need to analyze engineering literature.
His guidelines are quite handy since students often struggle when asked to read and analyze research papers or submit term papers.
The trick consists in thinking as referees, not as casual readers.
1. What are motivations for the work?
For a research paper, there is an expectation that a problem has been solved that no one else has published in the literature. This problem intrinsically has two parts.
(a) What is the people problem?
The people problem is the benefits that are desired in the world at large; for example some issue of quality of life, such as saved time or increased safety.
(b) The second part is the technical problem, which is: why does not the people problem not have a trivial solution?
There is also an implication that previous solutions to the problem are inadequate. What are the previous solutions and why are they inadequate? Finally, the motivation and statement of the problem are distilled into a research question that can be addressed within the confines of this particular paper. Oftentimes, one or more of these elements are not explicitly stated, making your job more difficult.
2. What is the proposed solution?
This is also called the hypothesis or idea. There should also be an answer to the question why is the solution to the problem better than previous solutions. There should also be a discussion about how the solution is achieved (designed and implemented) or is at least achievable.
3. What is the evaluation of the proposed solution?
An idea alone is usually not adequate for publication of a research paper. This is the concrete engagement of the research question. What argument, implementation, and/or experiment makes the case for the value of the ideas? What benefits or problems are identified? Are they convincing? For work that has practical implications, you also want to ask: Is this really going to work, who would want it, what it will take to give it to them, and when might it become a reality?
4. What are the contributions?
The contributions in a paper may be many and varied. Beyond the insights on the research question, a few additional possibilities include: ideas, software, experimental techniques, or an area survey.
5. What are future directions for this research? Not only what future directions do the authors identify, but what ideas did you come up with while reading the paper?
Sometimes these may be identified as shortcomings or other critiques in the current work.
This is a legacy post, originally published in 2/10/2007.