Scoping the academic literature

The first practical task in writing a literature review is scoping – a preliminary exploration of resources, including existing reviews, to assess issues of quality and planning. Scoping helps you determine which topics should be included or excluded in your literature search, and guides decisions about the structure of your review.

Make a start

Begin by searching catalogues and databases, and reading recently published work in your field.[1] Follow the trail of references. Look for previous literature reviews on related topics. Don’t confine your search to your topic, or to your academic discipline.

Record all relevant publication data for each item, as one of the greatest frustrations of academic research is being unable to use material because you lack essential citation data. For a web page, record the URL, and the date on which you accessed the page. Save a copy of significant web pages which may subsequently be altered or removed from the Internet.

Keep a record of your search strategy to avoid duplication, déjà vu, and missing out on a significant area of the literature because you thought you had already completed that search. Examiners may take an interest not only in the evidence you gather but the process used to assemble it. Evidence of a search strategy may be required as part of the thesis or dissertation.

Theologian Nijay K. Gupta offers this advice to theological students beginning the search process:

Select a dozen or so monographs and key articles … and write something like a long book review on each one. Once you have done that, ask questions like: What techniques and insights brought about some of the advancements in this area? Which perspectives and arguments have stood the test of time? Where does the most amount of disagreement lie? In the end, you will tailor your review of literature to concentrate on the specific topic of your dissertation. What kind of information would an uninitiated generalist in your field need to know to catch him or her up to understand the state of the discipline and how your research builds on, challenges, and advances forward from where the field is currently?[2]

Scour the sources

A literature review will normally focus on primary sources – original research published in books and journals. Secondary sources include critical evaluations and syntheses of original material, and are often useful in identifying and assessing primary sources. Tertiary sources usually comprise information and ideas drawn from secondary sources such as textbooks offering a broad overview of a topic.

Sources may include published and unpublished material, both print and electronic. Most review literature will probably be in the form of single-author books, chapters in edited books, and journal articles. You may also discover relevant material in books of collected articles, review articles, opinion pieces and commentaries, and grey literature (see below). You may also find relevant material in conference papers, case studies, official publications, and statistical reports.

There are various reasons why suitable resources may be difficult to find. For example:

• • You may have typed incorrect spelling in the search box.
  • Your search terms may be too broad or too narrow, in which case you should try applying search limiters or expanders.
  • Your search terms or topics may be too recent or too general to have found their way into catalogues and search engines.

On reflection, you may discover that you need to change your search strategy or revise research question. If contemplating such thoughts, it is often advisable to seek advice from seasoned researchers and academics before making significant methodological decisions.

Below are the main sources of information and materials you would access for scoping the literature.

Catalogues. Search your academic library’s catalogue for items using keywords, author names and titles of relevant books. Most catalogues now include citations from the databases to which they subscribe with search results.

Consider browsing catalogues from other institutions such as the National Library of Australia, State Library of NSW, WorldCat, the British Library, Library of Congress, the University of Sydney Library, and the libraries of theological colleges. The Library of Congress also provides links to hundreds of similar catalogues around the world.

Consider consulting the Google and Google Scholar search engines, and websites such as the Internet Archive and Academia.edu. You may also find relevant material in the catalogues of publishers and booksellers. Academics, librarians and research specialists may provide valuable information and support as you conduct your literature search.

These may be searched through the general library catalogue or in stand-alone form. Database search results often deliver enough information for you to find books and journal articles in print and/or electronic form. There may also be an abstract giving a summary of the content. For example, a list of databases accessible to patrons of Moore Theological College Library is available here.

Other popular academic databases include the British Library’s Electronic Thesis Online Service), and Proquest Dissertations & Theses (a repository of U.S. and international dissertations and theses).

Individual databases may not scan all the journals that may be of interest to you, so it is prudent to consult more than one database. To access some databases you will need to provide your designated username and password details. You may also be asked to login or register with the database provider in order to verify your student status.

If you locate too many search results, refine your search terms and/or topic. To save time, you may want to peruse journal article abstracts rather than fulltext, and locate existing literature reviews in your field (also known as “systematic reviews”). Remember to import your database references into your reference management library (e.g., Zotero) for future use.

Books. Peruse the title, subtitle, table of contents, introduction and back-cover blurb to assess a book’s relevance to your research question. The concluding paragraphs to chapters and the index may also be illuminating. Browse adjacent books on the shelf to find similar resources that may not be captured by your search terms.

Many academic books include detailed footnotes and bibliographies, which provide a potentially rich source of information about the author’s sources and related literature. Perusing these may complement your catalogue and database searches.

Journal articles. Journals, also known as serials, provide access to the most recent ideas and debates in an academic discipline, and many are now available in electronic form.

Identify and peruse the main journals in your field and others referred to in footnotes and bibliographies of key texts. Note especially peer-reviewed articles as these will be of higher quality and reliability. Check the contents page and reviews sections of journals for items of interest that may not appear in catalogue or database searches. These reviews will often indicate whether you should examine the book in detail.

See also Zetoc, a large collection of current journals and conference proceedings dating from 1993.

Due to the nature of original research, and human error, no electronic literature search can be absolutely comprehensive. Manual searching of issues of journals may reveal ideas and resources relevant to your research question that would otherwise not come to light. Awareness of the depth and breadth of your field may also inspire innovation.

Grey matter. This category of material comprises items which lack an ISBN or ISSN, such as theses and dissertations, archival material, government reports, printed music, manuscripts of sacred texts and poetry, newspaper articles, photographs, audio and video material, websites, emails, social media texts, advertisements, pamphlets and brochures, and publications by clubs and societies.

With few exceptions, such as theses, these items are not normally available through the regular bookselling and bibliographic channels.

Websites and social media. Websites offer a rich potential source of information but are generally not as reliable as books and journals. They provide opportunities for communication with professional and ministry organisations (publications, conferences, media releases, contact data). Some provide access to archives. Individual academics may maintain their own personal web pages or blog site with lists of their interests and publications, including information not yet available in catalogues and databases. Bookmarking allows you to keep track of your web sources.

Social media may also be harnessed for scoping purposes. Social bookmarking websites are sites on which Internet users share their web pages, articles, blog posts, images, and videos. Examples include Twitter, Pinterest, StumbleUpon, Digg and reddit. These offer potential for sharing content and links to resources (e.g., you might follow a well-known author on Twitter who regularly posts links to new articles and book reviews relevant to your research).

Apply the CRAAP test

Not all information is good information. You should exercise caution when using any online material, especially websites. [3] The CRAAP test, developed by librarians at California State University-Chico to assess websites, is a checklist you can use to evaluate the quality of online academic sources, and to some degree other kinds of academic source.[4]

Currency – the timeliness of the information

• • When was the information published or posted?
  • When was it last revised or updated?
  • Does your topic require current information, or will older sources work as well?
  • Are the internet links functional?
  • Does the website’s copyright date match the currency of the content?

Relevance – the importance of the information for your needs

• • Does the information relate to your topic or answer your question?
  • Who is the intended audience?
  • Is the information at an appropriate level (i.e., not too elementary or advanced)?
  • Did you examine a variety of sources before determining this is one you will use?

Authority – the source of the information

• • Who is the author/publisher/source/sponsor?
  • What are the author’s background and organisational affiliations?
  • What are the author’s qualifications to write on the topic?
  • Is there contact information, such as a publisher’s address or email address?
  • What is the publisher’s interest, if any, in this information?
  • If the material is a journal article, is it peer-reviewed?
  • Does the URL reveal anything about the author or source (e.g., .com .edu .gov)?
  • To help answer Authority questions, check out a website’s About section

Accuracy – the reliability, truthfulness and correctness of the content

• • Where does the information come from?
  • Is the author consistent in the application of methodology and methods?
  • Is the information fact or opinion?
  • Does the author acknowledge his/her bias?
  • Does the author resort to emotive language to seek to persuade?
  • Is the information supported by a reference list citing credible evidence?
  • Are the author’s conclusions reasonable and compelling?
  • Has the information been reviewed or refereed?
  • Can you verify any of the information in another source or from personal knowledge?
  • Are there spelling, grammatical or typographical errors in the text?

Purpose – the reason the information exists

• • What is the purpose of the information? Is it to inform, teach, sell, entertain or persuade?
  • Do the authors/sponsors make their intentions or purpose clear?
  • Is the information fact, opinion or propaganda?
  • Does the point of view appear objective and impartial?
  • Are there political, ideological, cultural, religious, institutional or personal biases?

When to stop searching

For many researchers, the literature search is one of the most enjoyable experiences of the research cycle. But it can also be a harrowing task. In Visualizing Research, Carol Gray and Julian Malins offer this advice:

In searching for information, be prepared to be simultaneously depressed and excited – depressed because you cannot find anything to match your needs exactly, and excited because this means that your line of inquiry could be unusual or even unique. Be prepared to step out of both your subject area … and even your discipline.[5]

On the other hand, if you feel overwhelmed by the amount of material you have identified as relevant, you may need to narrow the focus of your research.

You will always find more articles for your research if you keep looking. It is important to know when to stop collecting new material and move on to writing. Your aim is to “build an argument, not a library.”[6] It is usually prudent to begin summarising and analysing material while collecting, and remain open to discovering new material as your project progresses through the writing stages.

---

Rod Benson is an ordained Baptist minister presently working as Research Support Officer at Moore Theological College, Sydney. He enjoys preaching, cooking, snorkeling, and reading a good book. 

---

References

[1] On choosing search terms see David E. Gray, Doing Research in the Real World (Thousand Oaks, CA: Sage, 2014), 101f; and Zina O’Leary, The Essential Guide to Doing Your Research Project (London: Sage, 2014), 91-94.

[2] Nijay K. Gupta, Prepare, Succeed, Advance: A Guidebook for Getting a PhD in Biblical Studies and Beyond (Eugene, OR: Pickwick Publications, 2011), 72 (original emphasis).

[3] For more information on evaluating online sources see Diana Ridley, The Literature Review: A step by step guide for students (Thousand Oaks, CA: Sage, 2012), 47f.

[4] The information here is adapted from material created by Rebecca Hill Renirie and published at https://libguides.cmich.edu/web_research/craap. There are various other methods, such as RADAR – Relevance, Authority, Date, Appearance, and Reason for writing.

[5] Carole Gray & Julian Malins, Visualizing Research: A Guide to the Research Process in Art and Design (Burlington, VT: Ashgate, 2004), 43.

[6] Kjell Erik Rudestam & Rae R. Newton, Surviving Your Dissertation: A Comprehensive Guide to Content and Process (Sage, 1992), 49.

Image source: NASA/imgur