Evaluating the Consistency and Quality of Search Strategies and Methodology in Cochrane Urology Group Systematic Reviews

Evaluating the Consistency and Quality of Search Strategies and Methodology in Cochrane Urology Group Systematic Reviews

(1) Method, Analysis, and Finding (Research): Sixty-five published Urology Group Cochrane Systematic Reviews (CSR) and Protocols (CSP) were downloaded from the Cochrane Library on September 15, 2016. A data form was created using Qualtrics consisting of several sections including expanded PRESS (Peer-Review of Electronic Search Strategies) elements, database selection, search methodology reporting, & consistency. Four librarians, including two Cochrane Urology Assistant Information Specialists, evaluated 20 CSR/Ps each, allowing overlap to verify data extraction reliability. Librarian reviewers noted high variability in CSR/P search methods including selection of databases searched, full search strategies reporting, and use of publication filters, clinical trial registries, conference proceedings, and journal TOCs. Within search strategies, common errors included wrong line numbers, misspellings, inappropriate syntax, and duplicated lines. (2) Description: Cochrane Reviews are frequently viewed as the "gold standard" of systematic reviews; therefore the CSR/Ps should have high quality search methodology. This retrospective analysis represents a historical overview of search method reporting in CSR/Ps from the Cochrane Urology Group. Our results demonstrate a need for improved standardization including complete documentation of all strategies, more consistent database selection, less variation in strategy construction, increased use of peer-review and inclusion of the PRISMA Flow Diagram, and tighter adherence to the Cochrane Handbook/MECIR Standards. (3) Discussion: The range of databases searched was extremely wide with only MEDLINE and a Cochrane database seen consistently. It was rare to see search strategies provided for all (or even most) databases, with higher inclusivity in more recent reviews. Many reviews were limited to term listings in the methods while others provided one strategy as an example. Publication type filtering was extremely
varied, with alterations to standard Cochrane filters common. Most reviews included reference list searching and author contacts, but had little consensus regarding trial registries, conference proceedings, and journal table of contents.

Tran, Clara Y.; Lyon, Jennifer A.; Price, Carrie; Saragossi, Jamie

Chinese American Librarians Association

2018-06-24

Creative Commons Attribution 4.0

application/pdf

eng

Presentation files; Text

Tran, C. Y.; Lyon, J. A.; Price, C. & Saragossi, J. (2018). Evaluating the consistency and quality of search strategies and methodology in cochrane urology group systematic reviews. The Chinese American Librarian Association (CALA) 2018 Annual Conference Poster Session. New Orleans, LA, June 24, 2018.