Colonoscopy quality is measured by the degree in which the examination increases the likelihood of obtaining adequate results on health. Our aim was to develop an instrument for evaluating the quality of screening colonoscopies, taking into account the performance of endoscopists and endoscopy units.
Materials and methodsMixed methodology was employed. The first stage (qualitative) consisted of a Medline search, from which a group of experts developed the quality score items. The second stage (quantitative) utilized a modified Delphi technique to reach consensus (3 rounds). We evaluated the psychometric properties of the instrument (reliability and construct validity) in elective screening colonoscopies (in patients ≥ 50 years of age), performed within the January–April 2017 time frame.
ResultsA final instrument with 8 items was produced: (1) the Boston Bowel Preparation Scale score; (2) cecal intubation rate; (3) colonoscopy withdrawal time; (4) image documentation; (5) adenoma detection rate; (6) endoscopic surveillance planning; (7) perforation rate, and (8) continuous improvement programs. The instrument was evaluated in 323 colonoscopies performed by 31 endoscopists and found to be one-dimensional and reliable (Cronbach’s alpha 0.76). Performance was compared between endoscopists (center 1) and an expert endoscopist from another center (center 2): Boston Bowel Preparation Scale score 8.3 vs. 7.36 (P < .001), cecal intubation rate 93.5 vs. 96%, colonoscopy withdrawal time 14.8 vs. 8.4 min (P < .001), and adenoma detection rate 34 vs. 52.2% (P < .001), respectively.
ConclusionThe Colonoscopy Quality Score is a reliable and valid instrument for evaluating screening colonoscopy quality. Its results could be adapted to the usual endoscopic report to adjust monitorization frequency post-colonoscopy.
La calidad de la colonoscopía se mide por el grado en el que el procedimiento eleva la probabilidad de obtener resultados adecuados sobre la salud. Nuestro objetivo fue desarrollar un instrumento para evaluar la calidad de las colonoscopías de tamizaje, considerando el desempeño de los endoscopistas y las unidades endoscópicas.
Materiales y métodosSe empleó una metodología mixta. La primera etapa (cualitativa) consistió en una búsqueda en MEDLINE, a partir de la cual un grupo de expertos desarrolló los ítems de la escala de calidad. La segunda etapa (cuantitativa) utilizó una técnica Delphi modificada hasta llegar a consenso (3 rondas) y evaluamos las propiedades psicométricas del instrumento (confiabilidad y validez de constructos) en colonoscopías de tamizaje electivo (en pacientes ≥ 50 años), entre enero y abril del 2017.
ResultadosSe generó un instrumento final con 8 ítems: (1) puntaje de la Escala de Preparación Intestinal de Boston; (2) tasa de intubación cecal; (3) tiempo de retirada de colonoscopía; (4) documentación en imágenes; (5) tasa de detección de adenomas; (6) planeación de vigilancia endoscópica; (7) tasa de perforación, y (8) programas de mejora continua. El instrumento fue evaluado en 323 colonoscopías realizadas por 31 endoscopistas y fue considerada unidimensional y confiable (alfa de Cronbach 0.76). Se comparó el desempeño entre un grupo de endoscopistas (centro 1) y un endoscopista experto de otro centro (centro 2): puntaje Escala de Preparación Intestinal de Boston 8.3 vs. 7.36 (p < 0.001), tasa de intubación cecal 93.5 vs. 96%, tiempo de retirada de colonoscopia 14.8 vs. 8.4 min (p < 0.001) y tasa de detección de adenomas 34 vs. 52.2% (p < 0.001), respectivamente.
ConclusiónLa Escala de Calidad en Colonoscopía es un instrumento válido y confiable para evaluar la calidad de las colonoscopías de tamizaje. Sus resultados podrían ser adaptados al reporte endoscópico usual para ajustar la frecuencia de monitoreo, poscolonoscopia.
Colonoscopy is an endoscopic procedure that uses a charge-coupled device camera or a fiber optic camera on a flexible tube, enabling visualization from the anus to the ileum. The first complete colonoscopy was performed at the end of the 1960s, and since then, has become an invaluable diagnostic and therapeutic tool1. However, colonoscopies are not exempt from risk and require continuous efforts to improve safety and outcomes. It should always be performed, with respect to quality criteria (QC), completion, and safety.
Quality (from the Latin qualitas, -atis) is defined as “characteristics or a conjunct of characteristics inherent to something, permitting its value judgment”. In the endoscopic field, quality of care is the degree to which health services for individuals and populations increase the likelihood of desired health outcomes, in a manner consistent with current professional knowledge2. Quality may be affected before, during, and after the procedure. If the endoscopist meets the quality and performance criteria, he/she can continuously improve performance and safety.
Because there are numerous independent centers, and even centers with a single endoscopist, obtaining excellent, homogeneous colonoscopy results is difficult. In 1999, an audit of 68 endoscopy units in the United Kingdom (UK), involving about 9,000 colonoscopies, showed an overall cecal intubation rate (CIR) of 76.9% and a colon perforation rate of 1:769 cases. Those numbers were inadequate and below expectations. In fact, two well-known colonoscopy QC were unfulfilled3. Those results required improvement, to be able to start a national colorectal screening program based on the fecal immunochemical test, fecal occult blood test, and colonoscopy. The methodology of the joint advisory group transformed the colonoscopy practice in the UK, with the government investing enormously in it, resulting in the creation of a network of regional and national training centers. In 2011, a new national audit including 20,000 colonoscopies at 300 endoscopy units showed an increase in the CIR, reaching 95.8%, and perforations were 1:2,5104. In addition, 3 national centers and 7 regional centers grew to 23. Those results were made possible by establishing quality items to assess and record the data of each unit and the performance of a single endoscopist. In another experience in the same field at the Mayo Clinic (Rochester, Minnesota), permanent evaluation of their endoscopists per QC was implemented, through the Mayo Colonoscopy Skills Assessment Tool. As a result, their numbers improved, with permanent feedback of the overall endoscopy results, as well as those for single endoscopists5.
There is always room for improvement and minimum quality standards should be established. High-quality colonoscopy should be patient-centered, evidence-based, cost-effective, and as easy as possible to audit. Several groups have been working on establishing such QC and permanent assessment. The list of the criteria has grown, and more than 100 items are available6–9. The latest efforts have been made by the American College of Gastroenterology (ACG) and the American Society for Gastrointestinal Endoscopy (ASGE), in their joint publication of the comprehensive “Quality Indicators for Colonoscopy”, with grades of recommendation9.
Said experience makes the improvement of quality markers a realistic task, clarifying their relevance and significance. The quality markers (items, indicators) are separated, according to the time at which they occur, into pre-procedure, intra-procedure, and post-procedure markers. Items recognized as the most important are major complication rates, colonic cleansing, interval colorectal cancer (CRC) rates, and the adenoma detection rate (ADR)10–14. Adequate documentation and equipment reprocessing are also being considered.
All colonoscopy units should implement quality assessment measures. However, despite the vast amount of quality indicators, there is no consensus on which items should be considered for evaluation. In addition, many colonoscopy units do not have a registry of the procedure, limiting the opportunity of assessment and comparison between endoscopists and colonoscopy units.
Our study aimed to develop a tool for assessing screening colonoscopy quality, considering the performance of endoscopists and endoscopy units, to identify the most important quality indicators. We also included a scoring system for defining whether the endoscopist’s performance was excellent (green light); involved certain difficulties (yellow light) that implied a modified control follow-up period; or unsatisfactory (red light), requiring rescheduling. In a prospective cohort of colonoscopies, we piloted the psychometric properties as well, including the validity and reliability of the instrument.
Materials and methodsDesign and pilot studyWe employed a mixed methodology that included 3 stages: (1) item proposal, (2) a Delphi technique to identify consensus, and (3) a pilot study.
A qualitative approach was used in the first stage, to identify the aspects related to quality in colonoscopy, conducting a MEDLINE search that included the following MeSH terms: colonoscopy, endoscopy, quality, surveys and questionnaires, mass screening, and quality control.
In the second stage, a committee of 6 experts created items addressing the most relevant aspects of quality in colonoscopy that emerged from the first stage results. We performed a 3-round Delphi technique, with a panel of 42 Latin American experts in colonoscopy from 9 different countries (Argentina, Brazil, Colombia, Chile, Ecuador, Mexico, Peru, Uruguay, and Venezuela)15. The respondents were asked to rate the relevance of the items identified in the first stage, through a Likert-type scale, in which 1 = not relevant, 2 = relatively relevant, 3 = uncertain, 4 = importantly relevant, and 5 = the most relevant. The survey was administered online, and all the items with a mean value ≥4 were considered important. Based on those results and comments from the panel, a second round was carried out, with the same Likert-type scale, after which the items obtained were rated in a final third round, according to the degree of agreement (Likert scale, in which 1 = strongly disagree, 2 = disagree, 3 = undecided, 4 = agree, 5 = completely agree).
The third stage was performed to evaluate and refine the instrument. We piloted our quality score at the endoscopy unit of a university hospital in Santiago, Chile. We included all the screening colonoscopies for colorectal cancer performed on ambulatory patients 50 years old or older, within the time frame of January 2017 to April 2017. We recorded the sociodemographic characteristics of the endoscopists, including age, position (resident/staff), years of experience, and specialty (gastroenterologist/surgeon). Regarding the procedure, we recorded cleansing, according to the Boston Bowel Preparation Scale (BBPS), the CIR, withdrawal time, polyp extraction, and complications16. The information was processed, and the ADR was calculated, based on the biopsy results from all the extracted polyps. In addition, we compared those results with the performance of an expert (more than 40 years of experience) from an external center.
Subjects and procedureAll the Delphi panel opinions were confidential. The information regarding the colonoscopies was part of a self-assessment program of the endoscopy unit.
Statistical analysisTo perform the quantitative analyses, the item response categories were numerically coded from 1 to 3 (1: bad, 2: fair, 3: good). The cut-off values of each item were defined from the evidence obtained in the MEDLINE search, and when there was no evidence, the cut-off values were proposed by the expert group.
To evaluate the construct validity of the questionnaire, we used an exploratory factor analysis, followed by a Varimax rotation. The factors were chosen, using the following 2 criteria: (1) the Kaiser-Guttman criterion, in which all factors with an eigenvalue >1 were included17, and (2) the Cattell criterion, in which the inflection point of the scree plot curve was the cut-off, and all the factors mentioned above were accepted18. Considering the ordinal scale of the scores, the matrix was constructed, based on polychoric correlations19. We calculated Cronbach’s alpha to test internal consistency20. The statistical analysis was performed utilizing STATA 14 software (Stata Statistical Software: Release 14. StataCorp LP, College Station, TX, USA).
Ethical considerationsNo animals or humans were used as study subjects for procedures or interventions. Only the clinical records of patients were accessed, and we recorded the information anonymously. Likewise, we protected the identity of the endoscopists. Because the information was used retrospectively and recorded anonymously, we requested a waiver of informed consent. The study was reviewed and approved by the Ethics Committee at the Faculty of Medicine of the Pontificia Universidad Católica de Chile (ID 170505002).
ResultsDelphi panel resultsAfter obtaining 94 first items from the experts, a group of 42 Latin American endoscopists from 9 countries assessed the degree of importance in the first and second Delphi rounds (response rate 66.2%). We refined the instrument to 35 items and 23 endoscopists then evaluated it in a final round (degree of agreement and critical importance) (Fig. 1). A final 8-item instrument was obtained to evaluate colonoscopy quality (Table 1).
A scoring system based on an 8-item instrument with 3 possible responses: good, fair, and bad; the Colonoscopy Quality Score.
| Items | Evaluation | ||
|---|---|---|---|
| Good (3 points) | Fair (2 points) | Bad (1 point) | |
| 1. Cleansing assessment by the endoscopist, utilizing the Boston Bowel Preparation Scale, with more than 85% of patients reaching a good score | 8−9 | 6−7 | 5 or less |
| 2. Cecal intubation rate in CRC screening programs (in patients ≥50 years of age) | More than 95% | 95−90% | Less than 90% |
| 3. Colonoscopy withdrawal time | More than 9 min, spending more time in the ascending colon, as well as the descending colon, including a double check of the ascending colon | 6−9 min | Less than 6 min |
| 4. Colonoscopy report | Image documentation of all sites mentioned as evaluated, with emphasis on all relevant lesions | Image documentation of almost all sites mentioned as evaluated, including all relevant lesions | No image documentation of relevant lesions |
| 5. Adenoma detection rate in colonoscopies for CRC primary screening (in patients ≥50 years of age) | >30% in men | 25−30% in men | 25% in men |
| >20% in women | 15−20% in women | <15% in women | |
| 6. Surveillance program planned by the endoscopist, after adenoma or serrated polyp detection or cancer resection | Plans a colorectal cancer surveillance program | Only performs surveillance programs in patients with previously resected colon cancer | Does not perform a CRC surveillance in either case |
| 7. Perforation rate in diagnostic colonoscopies (per mille ‰) | <0.1% (1:1,000 cases) | 0.1−0.2% | >0.2% (1:500 cases) |
| 8. Continuous improvement program at the endoscopy unit | Continuous quality improvement programs, with >50% of endoscopists participating | Has continuous quality improvement programs, but <50% of endoscopists participating | Does not have continuous quality improvement programs |
CRC: colorectal cancer.
We assessed the instrument in 323 colonoscopies, performed by 31 endoscopists from a single university center in Santiago, Chile. Twenty-nine (93.5%) were staff and 4 (6.5%) were fellows. Most participants were gastroenterologists (61.3%), and all the others belonged to the surgical coloproctology department (38.7%).
A factor analysis was conducted to evaluate construct validity. Items 5 and 7 were excluded due to null variability. That analysis showed significant and positive correlations above 0.3 and below 0.9 in all the items, except for item 1 (which had negative correlations). After item extraction, the first factor had an eigenvalue above 1 (3.65), which was 3.9-times greater than the second factor (Fig. 2), explaining the 64.5% variance in the survey scores. All factorial loads were high (>0.4), and the representation in more than one factor (cross-loads) was low (Table 2); however, the loading of item 1 was negative. On the other hand, we obtained good reliability with a global Cronbach’s alpha of 0.76.
Factorial loads of the Colonoscopy Quality Score items.
| Items | Factorial load |
|---|---|
| 1. The Boston Bowel Preparation Scale score | −0.55 |
| 2. Cecal intubation rate | 0.72 |
| 3. Withdrawal time | 0.71 |
| 4. Image documentation | 0.95 |
| 6. Surveillance planned by the endoscopist | 0.70 |
| 8. Continuous improvement programs | 0.97 |
The endoscopists achieved a mean Colonoscopy Quality Score (CoQS) of 20.4 ± 2.5 (85% of the maximum score). We also compared the performance between the endoscopists from center 1 and an expert from a different center (center 2), with 42 years of experience performing colonoscopies. Center 1 had a higher BBPS score (8.3 vs 7.36, p < 0.001) and withdrawal time (14.8 vs 8.4 min, p < 0.001) and a lower ADR (34% vs 52.2%, p < 0.001). Additional results from the comparison are shown in Table 3.
Comparison of the performance of 31 endoscopists and an expert endoscopist.
| Variables | Center 1 | Center 2 | p Value |
|---|---|---|---|
| 31 endoscopists | Expert endoscopist | ||
| Number of colonoscopies | 323 | 278 | |
| Patients | |||
| Median age (years) | 64.7 | 63 | 0.089 |
| Patient sex (% female) | 64.1 | 51.1 | 0.001 |
| Mean BBPS score | 8.3 | 7.4 | <0.001 |
| Cecal intubation rate (%) | 93.4 | 96 | 0.1632 |
| Mean withdrawal time (min) | 14.8 | 8.4 | <0.001 |
| ADR (%) | 34.1 | 52.2 | <0.001 |
ADR: adenoma detection rate; BBPS: Boston Bowel Preparation Scale.
Quality in performing screening colonoscopies appears to be a cornerstone of safety in clinical practice and given that CRC is the third leading cancer worldwide, colonoscopy results must be thorough and reliable21.
Our study aimed to develop an instrument to assess quality before, during, and after the procedure. After 3 Delphi rounds, we obtained an 8-item instrument. The representation of Latin American endoscopists involved in the item development was remarkable and included 9 Latin American countries. Interestingly, one item was the BBPS, a well-known parameter for assessing quality that is dependent on the endoscopy unit. In fact, incomplete bowel cleansing decreases the detection of lesions, forcing a repeat surveillance colonoscopy one year later22,23.
Other items identified as endoscopist-dependent quality indicators are CIR, withdrawal time, the ADR, and the colonoscopy report. An adequate CIR and withdrawal time ensure careful mucosal inspection, which is essential for effective CRC prevention and reduced cancer mortality. A high ADR is also essential for recommending safe intervals between screening and surveillance examinations13. The colonoscopy report must be adequately described to certify the proper exploration of the colon and communicate all findings and interventions performed on the patient.
Regarding the perforation rate (in CRC surveillance) and continuous quality improvement programs, we believe those points reflect the endoscopic skills and training of the endoscopist. Thus, a low score on those items could reflect the need for urgent re-training.
The psychometric analysis showed a unidimensional and reliable instrument (Cronbach’s alpha of 0.76) and the quantitative phase (including 3 Delphi rounds) ensured adequate content validity, with several items reflecting all the aspects involved in quality (the endoscopy unit, endoscopist, equipment, follow-up, and training). In addition, the results from the pilot phase showed good performance of the endoscopy unit (center 1), obtaining a mean CoQS of 20.4 ± 2.5 (85% of the maximum score). The evaluation in the clear majority of endoscopists was fair or higher in each item. Furthermore, the items of the instrument discriminated the performance between the endoscopy unit (center 1) and an expert from a different center (center 2). Even with a lower BBPS and withdrawal time, the expert endoscopist obtained a higher ADR (34.1% vs 52.2%), most likely due to his expertise (more than 40,000 procedures performed). Because long-term application is needed to assess the ADR and perforation rate, we excluded items 5 and 7 from the factorial analysis.
Many authors have attempted to establish QC in the endoscopist. Ekkelenkamp et al. proposed 16 items, but the complexity of their instrument makes it difficult to use in clinical practice24 and it has not been validated in a prospective cohort. On the other hand, the European Society of Gastrointestinal Endoscopy (ESGE), the ASGE, the ACG, and the American Gastroenterological Association (AGA) have published at least 3 position statements and recommendations7,9,25. However, those recommendations are difficult to assess in clinical practice due to their length and complexity. Application of the CoQS took no longer than 2 min per endoscopist and the results were correctly received, showing it to be a simple and useful tool for assessing screening colonoscopy quality. Furthermore, it constitutes the first instrument validated in a prospective cohort in that field.
One of the limitations of our study was the difficulty in evaluating all the dimensions of the instrument, given its validation at a single endoscopy unit. Hence, the items of BBPS and continuous quality improvement programs could not be adequately represented in the factor analysis. As a future challenge, we will validate the CoQS instrument in a multicenter prospective cohort, to explore the additional dimensions of the scale. Finally, we hope that the CoQS will be a useful tool in identifying the need to re-train endoscopists and that it can be implemented in endoscopy units worldwide.
ConclusionsThe CoQS is a useful questionnaire for evaluating screening colonoscopy quality, considering the performance of endoscopists and endoscopy units. Its results could be helpful in identifying the need to re-train endoscopists.
Financial disclosureThe authors received no specific funding for this work.
Conflict of interestThe authors declare that there is no conflict of interest regarding the publication of this article.
The present study is a collaborative work involving 42 endoscopists from 9 countries, two of whom work in the United States. We appreciate the interest and participation of the following colleagues: Juan Pablo Allones, Luis Caro, Cecilio Cerisoli, Eduardo Coghlan, Mariano Higa, Fernando Martin, Diego Murature (Argentina); Simone Guaraldi, Fauze Maluf-Filho (Brazil); Carlos Bustos, Gustavo Delgado, Fernando Fluxa, Eduardo Maiza, Alex Navarro, Arnoldo Riquelme, Antonio Rollan, Gonzalo Ross, Carlos Rueda, Marcela Saenz, Roque Saenz, Oscar Varas, María Teresa Vergara (Chile); Alberto Kabalan (Colombia); Jaysoom Abarca, Ivonne Orellana, Carlos Robles, María de los Ángeles Silva (Ecuador); Sergio Sobrino (Mexico); Carla Celestino, Nancy Machaca, Juan Torreblanca (Peru); Eduardo Fenocchi, Nicolás González, Horacio Gutiérrez, Javier San Martín, Claudia Stefanoli, Daniel Taullard, Asadur Tchekmedyian (Uruguay); Alberto José Baptista, Maribel Lizarzabal (Venezuela), Andres Gelrud (Venezuela/USA), Francisco Ramirez (Peru/USA).
Please cite this article as: Sáenz-Fuenzalida R, Riquelme-Pérez A, Díaz-Piga LA, García-Rocha X, Fuentes-López E, Arnold-Álvarez J, et al. El desafío de cuantificar la calidad de la colonoscopia de tamizaje: el desarrollo y las propiedades psicométricas de la Escala de Calidad en Colonoscopia. Rev Gastroenterol Méx. 2022;87:297–304.
