Evaluating Search Systems through Field Codes and Classifications: The Case of the First Published Swedish Nursing Study

The aim of the study was to assess and compare established search systems and approaches by using the search goal of identifying the first (as in oldest) nursing-related document, with reference to the first Swedish-affiliated document, in Scopus, Web of Science, and PubMed (Medline). In doing so, the objective was to provide concrete examples and illustrate how search systems and field code searches versus classification-based searches differ. The Scopus database, Web of Science database aggregator, and the PubMed (Medline) search engine were used for evaluations. Two different search strategies were compared in each database: one guided by field codes (i.e., text-based), and one guided by pre-existing categorizations within database infrastructures. Findings highlight several factors that are important to consider when formulating and executing a search strategy. The findings illustrate important aspects of search systems and search approaches, namely the publication year, delimitations related to a country or region, and features related to a subject or research area. The findings also highlight the importance of prioritizing between different ideals in retrieval. For example, is it more important to reach the oldest records (publication year), to identify as many records as possible (exhaustiveness), or to make sure they are as subject-relevant as possible (e.g., in terms of authors belonging to the same field)? Secondly, researchers using bibliometric methods to analyze research literature should be more transparent when reporting searches, as different search systems and search approaches yield varied numbers and quality of records. While most bibliometric researchers strive to combine different approaches, this is not always made clear in reporting.