How Much Literature is Enough?

How do I know that I have reviewed enough literature and that is where I am supposed to stop?

The short answer is that, as an academic, you never stop reviewing the literature because you need to be aware of the latest developments in your field and what your competitors are doing.

I don’t think this sufficiently answers the question though. As a PhD student aiming to submit a thesis there has to be a cutoff point somewhere; a point where you stop adding new content and consolidate what you have into a submissible form. There also needs to be some way of filtering and prioritising literature and deciding which references to follow up.

The first thing to know is that you cannot read it all. If an online search returns 10,000 results, reading at a rate of 10 per day, every day it will take you the best part of 3 years to get through them all, and that’s not counting all the references contained in the bibliographies of those articles and all the new articles published daily.

It is mathematically impossible to read everything that’s published, so you should not try. The challenge is not to read everything, but to intelligently select what to read and what not to.

 Breadth vs depth

There is a necessary trade off between breadth and depth. The broader you make your literature search the more sources there will be, but few of these will be of real use or relevance to your research. As you focus in on your specific research niche you will find fewer papers, but the relevance will tend to increase.

While it’s important to have a broad knowledge of your field, it is usually enough to have an understanding of basic principles, techniques and the kinds of research being done. It’s only when you get to the literature directly relevant to your own work that you need real depth and detail.

If you have that broad knowledge, it’s easy to go back to the literature to look up specific work (or examples of types of work for a literature review) as and when you need to.

Not everything you read needs to go into your written literature review. The literature you select should represent the best examples you can find of the point you want to make.

You should vary the level of detail you include in each section of the literature review, depending on the relevance to your own work.

 What you need to know in great depth

• Any work done by direct competitors in the field,
• Any work that you use as a theoretical basis for your own,
• Any work whose methodology you have borrowed or adapted

Depending on your research, there may be relatively few such sources, but they will be the most important to cover in your literature review. In my own PhD, there were maybe 10 or 20 articles I referred to again and again throughout my research because they were essential to my work. I would also keep an eye on any new work published by those authors, and any new articles citing them (as it was highly likely that anyone doing the same work as me would cite the same key sources).

Although new papers are frequent, major developments happen slowly

Truly groundbreaking research is rare. Although countless new papers are published daily, most of them only contribute incrementally to the body of knowledge. You can save yourself a huge amount of time by focusing on the truly influential work, because it gives you a solid foundation for understanding the incremental work that follows.

As you develop that fundamental understanding, it should become easier to read and judge the importance of the papers you read, and to identify the trends and key principles in the field. Although the process of reviewing literature for your own benefit is never complete, it is possible to write a review that outlines these key principles and trends and provides a snapshot of the current-state-of-the-art, shared challenges and open questions.

 How much is enough?

So, returning to the original question, how do you know how much is enough when reviewing the literature?  It’s probably the wrong question, because there is no absolute measure of “enough” for you to aim for. Even if there was an arbitrary goal of, say, 150 references, this would not necessarily result in a good literature review.

I think it’s better to ask yourself what it is you are trying to communicate with your literature review, so you can judge what level of detail is necessary in each section and which references best represent your point or the kind of research you are describing. Decisiveness and confidence are crucial. Rather than worrying about what the examiner wants to see, you decide what’s important to include.

This is one of the areas where a well-honed inner critic is useful; to look at each section and decide, “I don’t need so much detail here”, or “I really should include a reference to … here”, because really this is the only way to judge whether you have “enough”.

Reference