Abstract
Background
Although sealants are an established and recommended caries-preventive treatment, many children still fail to receive them. In addition, research has shown that existing measures underestimate care by overlooking the sealable potential of teeth before evaluating care. To address this, the authors designed and evaluated 3 novel dental electronic health record–based clinical quality measures that evaluate sealant care only after assessing the sealable potential of teeth.
Methods
Measure I recorded the proportion of patients with sealable teeth who received sealants. Measure II recorded the proportion of patients who had at least 1 of their sealable teeth sealed. Measure III recorded the proportion of patients who received sealant on all of their sealable teeth.
Results
On average, 48.1% of 6- through 9-year-old children received 1 or more sealants compared with 32.4% of 10- through 14-year-olds (measure I). The average measure score decreased for patients who received sealants for at least 1 of their sealable teeth (measure II) (43.2% for 6- through 9-year-olds and 28.4% for 10- through 14-year-olds). Fewer children received sealants on all eligible teeth (measure III) (35.5% of 6- through 9-year-olds and 21% of 10- through 14-year-olds received sealant on all eligible teeth). Among the 48.5% who were at elevated caries risk, the sealant rates were higher across all 3 measures.
Conclusions
A valid and actionable practice-based sealant electronic measure that evaluates sealant treatment among the eligible population, both at the patient level and the tooth level, has been developed.
Practical Implications
The measure developed in this work provides practices with patient-centered and actionable sealant quality measures that aim to improve oral health outcomes.
Key Words
Abbreviation Key:
CDT (Current Dental Terminology), DQA (Dental Quality Alliance), EHR (Electronic health record), E-measure (Electronic measure), OHA (Oregon Health Authority), PRR (Preventive resin restoration), SQL (Structured Query Language)
Children’s oral health.
Despite significant local and national efforts, it remains “one of the greatest unmet treatment needs among children.”
- Griffin S.O.
- Wei L.
- Gooch B.F.
- Weno K.
- Espinoza L.
The use of sealants has been proven effective in caries prevention,
- Beauchamp J.
- Caufield P.W.
- Crall J.J.
- et al.
Evidence-based clinical recommendations for the use of pit-and-fissure sealants: a report of the American Dental Association Council on Scientific Affairs.
,
,
- Ahovuo-Saloranta A.
- Forss H.
- Hiiri A.
- Nordblad A.
- Mäkelä M.
,
- Wright J.T.
- Crall J.J.
- Fontana M.
- et al.
,
- Mejàre I.
- Lingström P.
- Petersson L.G.
- et al.
,
- Ney J.P.
- van der Goes D.N.
- Chi D.L.
,
- Locker D.
- Jokovic A.
- Kay E.
,
- Fleisch A.F.
- Sheffield P.E.
- Chinn C.
- Edelstein B.L.
- Landrigan P.J.
,
- Griffin S.O.
- Naavaal S.
- Scherrer C.
- Patel M.
- Chattopadhyay S.
Evaluation of school-based dental sealant programs: an updated community guide systematic economic review.
and increasing sealant use in children is perceived as a national health goal.
Healthy People 2020: 2020 topics and objectives: oral health: overview.
Several programs have been launched toward achieving this goal, and consequently Healthy People 2020 progress reviews show an increase in sealant use from 25.5% through 28.1% for 6- through 9-year-olds and an increase from 19.9% through 21.9% for 13- through 15-year-olds.
Healthy People 2020: 2020 topics and objectives: oral health: objectives.
Medicare: A strategy for quality assurance: Sources and Methods, vol II.
Clinical quality measures are tools to help assess what level of health care quality we achieve. There is general agreement that to improve quality, measurement is the initial step. In our previous study, we evaluated sealant placement rates in dental practices, using specifications defined by the Dental Quality Alliance (DQA) and Oregon Health Authority (OHA), and we discovered that, at the practice level, both the OHA and DQA measures underestimated the care delivered. These dental quality measures are examples of process measures that report whether patients received recommended care. The DQA and OHA measures were originally designed for use by states and insurance plans that have access to claims data.
- Kumar S.V.
- Bangar S.
- Neumann A.
- et al.
When assessing the validity of these measures across 4 dental institutions, we identified measurement gaps that could be bridged by using data from the dental electronic health records (EHRs). We discovered that, at the practice level, both the OHA and DQA measures underestimated the care delivered. For example, approximately 67% of children who did not receive sealants had permanent first or second molars that were not sealable; that is, molars that had not erupted or were carious or had been previously sealed, restored, or extracted.
- Kumar S.V.
- Bangar S.
- Neumann A.
- et al.
The reason behind their inclusion could be that these metrics are restricted to using only the Current Dental Terminology (CDT) procedure codes documented in claims-based data to track care.
Code on Dental Procedures and Nomenclature (CDT) 2018.
The high-feasibility and low-cost implementation associated with using claims data is offset by its limited capture of clinical information and the lack of standardized and widely adopted diagnosis codes, making it difficult to precisely measure oral health outcomes and quality of life.
- Herndon J.B.
- Crall J.J.
- Aravamudhan K.
- et al.
Despite these concerns, federal and state programs and associations such as the Children’s Health Insurance Program,
Medicaid/CHIP Health Care Quality Measures technical assistance brief: guidance for calculating the dental sealant measure in the child core set.
the California Dental Association, and Texas Medicaid have adopted these measures in their performance indicator tools.
Oral health improvement program.
,
CDA community oral health resources.
- Herndon J.B.
- Tomar S.L.
- Catalanotto F.A.
- et al.
,
- Curtis J.R.
- Sathitratanacheewin S.
- Starks H.
- et al.
thus providing a more accurate and reliable assessment of a patient’s caries risk status. For example, through the EHR we are able to identify the number of caries-free molars eligible for sealants in children, an area in which claims data are limited.
The purpose of our study is to develop and validate a contemporary and usable electronic measure (e-measure), from EHR data, for all dental practices with EHR capabilities, to evaluate the proportion of children and adolescents who appropriately received a dental sealant on a permanent molar.
Methods
- Stark P.C.
- Kalenderian E.
- White J.M.
Consortium for Oral Health-Related Informatics: improving dental research, education, and treatment.
and use the axiUm EHR platform (Exan) and standardized diagnostic dental terminology (Systematized Nomenclature of Dentistry-Dental Diagnostic System [SNO-DDS]).
- Kalenderian E.
- Ramoni R.L.
- White J.M.
- et al.
Each participating institution obtained institutional review board approval before measure execution.
Designing the e-measure
- Kumar S.V.
- Bangar S.
- Neumann A.
- et al.
These gaps occurred because of the inherent gaps in the design of the claims-based measures to correctly characterize patients with teeth eligible for sealant treatment. For example, in the aforementioned study, in 43% of the patients who failed to meet the measure score numerator their first or second molars were unerupted. Therefore, we concluded that to estimate the true proportion of sealant placements within a given period, the measure first had to consider tooth eligibility before assessing sealant care. Using the American Dental Association’s evidence-based guidelines, a sealable tooth is a tooth likely to become carious (that is, deep pits and fissures) and have elevated risk.
- Naaman R.
- El-Housseiny A.A.
- Alamoudi N.
To accomplish this, we revised our electronic measures by first identifying patients with 1 or more “sealable teeth.” The term “sealable teeth” was introduced to define permanent molars eligible for sealant treatment, thereby excluding from our evaluation teeth that were unerupted, carious, restored, or previously sealed. In addition, we made a change to our numerator criteria by including the preventive resin restoration (PRR) procedure (CDT code D1352) in addition to the sealant procedure (CDT code D1351).
Code on Dental Procedures and Nomenclature (CDT) 2018.
PRR is similar to the sealant procedure, and both are treatments for enamel lesions. Using the same format detailed in our previous study, we evaluated our electronic measure by caries risk status, thereby running the measures both with and without the elevated caries risk constraint.
- ▪
caries diagnosis with planned, in progress, or completed restorative, endodontic, fixed prosthetic, or surgical procedures;
- ▪
caries risk as documented in a caries risk assessment form;
- ▪
caries risk–diagnostic CDT codes;
- ▪
caries risk diagnosis and treatment pair, using a listing of caries risk diagnoses with concomitant preventive procedures.
Study Population
Code on Dental Procedures and Nomenclature (CDT) 2018.
The age criterion for our study population was applied as of the last day of the reporting year. For the 6- through 9-year age group, only teeth nos. 3, 14, 19, and 30 were used to calculate the measure scores; for the 10- through 14-year age group, only teeth nos. 2, 15, 18, and 31 were used. To carry out the study objective, we designed 3 levels of quality measures to evaluate the sealant placement in an ordered form (Table 1):
- ▪
Level 1 (measure I score: sealant on any tooth). This measure recorded the proportion of patients who completed an evaluation (either CDT code D0120 or D0150) and who received a sealant (CDT code D1351) or a PRR (CDT code D1352) on a permanent first or second molar.
- ▪
Level 2 (measure II score: sealant on at least 1 sealable tooth). We restricted this measure to patients with eligible sealable teeth. Here the measure looked at the sealable-sealed combination across the 4 index teeth for both age groups. Patients with at least 1 sealable-sealed pairing were included in the numerator. This implied that only patients with at least 1 of their sealable teeth sealed were included in the measure.
- ▪
Level 3 (measure III score: sealants on all sealable teeth). This measure is similar to the measure II score but with more exacting criteria. In this case, only patients who had all of their sealable teeth sealed were included in the numerator. The purpose of this measure was to accurately estimate the proportion of patients who had all their sealable teeth sealed in the reporting year.
Table 1Demographic characteristics of the study population.
EHR data extraction and data analysis
Step 1: Generating the Sampling Frame, Using the EHR
Because all the participating institutions were axiUm EHR users, we developed a Structured Query Language (SQL) script for all sites. The SQL queries generated a list of patients eligible to be included in both the denominator as well as the numerator. Each site tested the SQL query before the implementation.
Step 2: Measure Validation
To validate the query performance, we first ran the measure for a specific reporting period (2015) and age group (6-9 years) across the sites and compared performance with a manual review of charts.
Step 2a: Sample size estimation
The number of charts needed for manual review was calculated, using the sample size formula for estimating a proportion. The sample size calculation used the following initial values: the measure score proportion estimate, the standard margin for error (0.05), and the 2-tailed z score (z, 1.96) value.
Step 2b: Validation of the automated query
- Kumar S.V.
- Bangar S.
- Neumann A.
- et al.
,
- Bhardwaj A.
- Ramoni R.
- Kalenderian E.
- et al.
,
- Neumann A.
- Kalenderian E.
- Ramoni R.
- et al.
,
- Neumann A.
- Kumar S.
- Bangar S.
- et al.
,
- Neumann A.
- Obadan-Udoh E.
- Bangar S.
- et al.
we trained and calibrated 2 reviewers from each site to audit sampled charts. The first 50 charts from a randomized list of patients were analyzed by each of the reviewers for interrater reliability to establish calibration. Only when the 2 reviewers achieved a κ correlation coefficient greater than 0.80 (κ = 0.80) were they allowed to individually review the remaining charts.
Sensitivity, specificity, positive predictive value, and negative predictive value of the automated query were calculated, using the manual review as the reference standard. Once validated, we executed the measures across 4 consecutive years at all sites.
Statistical analysis
- Thissen D.
- Steinberg L.
- Kuang D.
Results
For the sealant measure without elevated caries risk, the site 2 κ value (κ = 38.2%) and positive and negative predictive values were found to be lower than for other sites. We deduced the reason for this low validity to be an error in the script, leading to incorrect inclusion of patients with unerupted molars. We rectified this error in our elevated caries risk iteration by updating the script.
Table 2Differences in measure scores across age.
Table 3Comparison of sealant measure scores by caries risk status.
Discussion
- Kumar S.V.
- Bangar S.
- Neumann A.
- et al.
We have attempted to overcome this limitation by developing an algorithm that uses EHR data to include patients with 1 or more sealable teeth and by excluding patients with restored, previously sealed, carious, or missing permanent first or second molars. To arrive at a comprehensive preventive process of care performance, we included both pit-and-fissure sealants as well as PRR as our numerator criteria. We also developed 3 perspectives by which to assess the performance of a dental practice. First, we report the percentage of children who received a sealant or PRR on any tooth. On testing, we encountered patients without sealable teeth receiving sealant retreatment. To account for these oddities, the second measure was designed to determine the percentage of children who received a sealant or PRR on 1 or more sealable teeth. We found minor differences in scores for these 2 measures. Last, the third measure determined the percentage of children who received a sealant on all of their sealable teeth. We found that practices performed lower with more stringent criteria when measuring if all sealable teeth received a sealant.
- Wright J.T.
- Crall J.J.
- Fontana M.
- et al.
A 2016 systematic review reported that children and adolescents who received sealants on noncavitated occlusal surfaces experienced fewer new carious lesions as compared with those without sealants.
- Wright J.T.
- Tampi M.P.
- Graham L.
- et al.
Evidence-based clinical recommendations for the use of pit-and-fissure sealants, released by the American Dental Association in 2008, were designed to aid practitioners in the development of a relevant sealant treatment plan.
- Beauchamp J.
- Caufield P.W.
- Crall J.J.
- et al.
Evidence-based clinical recommendations for the use of pit-and-fissure sealants: a report of the American Dental Association Council on Scientific Affairs.
However, despite these guidelines, studies have found that established clinical practice attitudes hamper the translation of these guidelines to everyday practice.
- O’Donnell J.A.
- Modesto A.
- Oakley M.
- Polk D.E.
- Valappil B.
- Spallek H.
,
- Govindaiah S.
- Bhoopathi V.
,
- Polk D.E.
- Weyant R.J.
- Shah N.H.
- Fellows J.L.
- Pihlstrom D.J.
- Frantsve-Hawley J.
Our results show that barriers exist in following evidence-based guidelines, and that they need to be identified and eliminated to allow for more timely execution of preventive dental care.
- Naaman R.
- El-Housseiny A.A.
- Alamoudi N.
Other reasons for variability in sealant placement may be due to educational pressures (in academic sites), treatment planning philosophies, and the institutional culture and environment. Sealants are indicated for retentive pits and fissures likely to become carious. Caries risk is a factor in the determination of whether a retentive pit and fissure is likely to become carious and whether a sealant is indicated. We therefore developed these measures stratified by caries risk.
- Bhardwaj A.
- Ramoni R.
- Kalenderian E.
- et al.
,
- Neumann A.
- Kalenderian E.
- Ramoni R.
- et al.
,
- Neumann A.
- Kumar S.
- Bangar S.
- et al.
,
- Neumann A.
- Obadan-Udoh E.
- Bangar S.
- et al.
In 2012, the DQA similarly proposed an e-measure for sealants for 6- through 9-year-old children.
Pediatric Oral Healthcare Exploring the Feasibility for E-Measures.
We hope that with the increased adoption of EHRs in practice and the collection of key data such as structured diagnoses,
- Kalenderian E.
- Ramoni R.L.
- White J.M.
- et al.
dental clinics will be prompted to systematically measure the quality of care provided. Although our study was conducted in institutions that use the same EHR, the measures can be implemented in other EHRs. We envision these types of practice-level quality measures aiding everyday providers to improve quality.
- Kalenderian E.
- Ramoni R.
- Spallek H.
- White J.
- Walji M.
It would allow for regular measuring, adjusting, and remeasuring of any given area for improvement. We recognize that the use of HER-based measures may pose a cost for some practices and that dentistry must strike a balance between the accuracy and usefulness of its measures. This flexibility in measurement will allow for a wider range of dental practices to evaluate their sealant placement rates, but additional study is required to fully understand the costs and benefits of EHR measure use. We are also aware that there is a separate debate on how quality measures should be used, and if they should be tied to reimbursements or incentives.
- Obadan-Udoh E.M.
- Calvo J.M.
- Panwar S.
- et al.
There may be other unintended consequences after these measures are implemented in practice that need further study.
Our e-measure queried structured data and was able to capture existing and past dental experiences by harnessing the ability of EHR databases to record real-time care. We were also able to assess sealant placement comprehensively by measuring 3 ways to recognize the level of care. We again showed that data from EHRs can be used to accurately assess whether children received sealants, and, if so, on most or all eligible teeth.
Limitations
We recognize that the proposed measures cannot be used in clinics or other sealant-based programs that do not use an EHR. Our measure was also developed in the axiUm EHR, and we do not have experience yet in implementing the measure in other EHRs. Our e-measure operated on the assumption that the standards of data documentation were also upheld. This measure was also designed to be reported annually. Therefore, we did not adjust the measure numerator on the basis of a qualifying examination date. So, it is possible that children who were seen later in the reporting year would have had less time to be included in the numerator than those seen earlier in the same reporting year. In addition, our study design looked at cross-sectional data rather than longitudinal data to measure sealant placement. It would be informative to observe how each patient from the ages of 6 through 14 differs in caries risk, sealant eligibility, and other associated factors influencing preventive care by using longitudinal data spanning multiple years. Last, this measure does not consider substandard recall, excludes restored teeth, and includes only unrestored teeth that have been adequately sealed. But these limitations apply to all measures that exclude teeth that have been restored or are carious, and this measure was not designed to show substandard recall care.
Conclusions
We developed and validated a practice-based sealant e-measure that assesses sealant treatment in the eligible population, both at the patient level and the tooth level. Although the purpose of this study was to develop and assess the e-measure, the results also show that sealant use among the 3 sites in this study was lower than desired given the overwhelming evidence on the benefits of sealants for preventing caries, implying that further quality improvement is needed.
References
Children’s oral health.
()
Vital signs: dental sealant use and untreated tooth decay among US school-aged children.
MMWR Morb Mortal Wkly Rep. 2016; 65: 1141-1145
Evidence-based clinical recommendations for the use of pit-and-fissure sealants: a report of the American Dental Association Council on Scientific Affairs.
JADA. 2008; 139: 257-268
Timetable for oral prevention in childhood: a current opinion.
Prog Orthod. 2015; 16: 27
Pit and fissure sealants versus fluoride varnishes for preventing dental decay in the permanent teeth of children and adolescents.
Cochrane Database Syst Rev. 2016; 1: CD003067
Evidence-based clinical practice guideline for the use of pit-and-fissure sealants: a report of the American Dental Association and the American Academy of Pediatric Dentistry.
JADA. 2016; 147: 672-682.e12
Caries-preventive effect of fissure sealants: a systematic review.
Acta Odontol Scand. 2003; 61: 321-330
Economic modeling of sealing primary molars using a “value of information” approach.
J Dent Res. 2014; 93: 876-881
Prevention, part 8: the use of pit and fissure sealants in preventing caries in the permanent dentition of children.
Br Dent J. 2003; 195: 375-378
Bisphenol A and related compounds in dental materials.
Pediatrics. 2010; 126: 760-768
Evaluation of school-based dental sealant programs: an updated community guide systematic economic review.
Am J Prev Med. 2017; 52: 407-415
Healthy People 2020: 2020 topics and objectives: oral health: overview.
US Department of Health and Human Services,
Washington, DC2013 ()Healthy People 2020: 2020 topics and objectives: oral health: objectives.
()
Medicare: A strategy for quality assurance: Sources and Methods, vol II.
National Academies Press,
Washington, DC1990Assessing the validity of existing dental sealant quality measures.
JADA. 2018; 149: 756-764.e1
Code on Dental Procedures and Nomenclature (CDT) 2018.
()
Developing and testing pediatric oral healthcare quality measures.
J Public Health Dent. 2015; 75: 191-201
Medicaid/CHIP Health Care Quality Measures technical assistance brief: guidance for calculating the dental sealant measure in the child core set.
()
Oral health improvement program.
()
CDA community oral health resources.
()
Measuring quality of dental care: caries prevention services for children.
JADA. 2015; 146: 581-591
Using electronic health records for quality measurement and accountability in care of the seriously ill: opportunities and challenges.
J Palliat Med. 2018; 21: S52-S60
Consortium for Oral Health-Related Informatics: improving dental research, education, and treatment.
J Dent Educ. 2010; 74: 1051-1065
The development of a dental diagnostic terminology.
J Dent Educ. 2011; 75: 68-76
The use of pit and fissure sealants: a literature review.
Dent J (Basel). 2017; 5: 34
Measuring up: implementing a dental quality measure in the electronic health record context.
JADA. 2016; 147: 35-40
Evaluating quality of dental care among patients with diabetes: adaptation and testing of a dental quality measure in electronic health records.
JADA. 2017; 148: 634-643.e1
Tobacco screening and cessation efforts by dental providers: a quality measure evaluation.
J Public Health Dent. 2019; 79: 93-101
Number of pregnant women at four dental clinics and the care they received: a dental quality emeasure evaluation.
J Dent Educ. 2019; 83: 1158-1165
Interrater reliability: the kappa statistic.
Biochem Med (Zagreb). 2012; 22: 276-282
Quick and easy implementation of the Benjamini-Hochberg procedure for controlling the false positive rate in multiple comparisons.
J Educ Behav Stat. 2002; 27: 77-83
Sealants for preventing and arresting pit-and-fissure occlusal caries in primary and permanent molars.
Pediatr Dent. 2016; 38: 282-308
Sealants and dental caries: insight into dentists’ behaviors regarding implementation of clinical practice recommendations.
JADA. 2013; 144: e24-e30
Dentists’ levels of evidence-based clinical knowledge and attitudes about using pit-and-fissure sealants.
JADA. 2014; 145: 849-855
Barriers to sealant guideline implementation within a multi-site managed care dental practice.
BMC Oral Health. 2018; 18: 17
Pediatric Oral Healthcare Exploring the Feasibility for E-Measures.
American Dental Association on behalf of the Dental Quality Alliance,
Chicago, IL2012: 31Quality measures everywhere: the case for parsimony.
JADA. 2018; 149: 322-326
Unintended consequences and challenges of quality measurements in dentistry.
BMC Oral Health. 2019; 19: 38
Biography
Ms. Kumar is a graduate research assistant, Department of Diagnostic and Biomedical Sciences, School of Dentistry, University of Texas Health Science Center at Houston, Houston, TX.
Biography
Dr. Yansane is an assistant professor, Department of Preventive and Restorative Dental Sciences, School of Dentistry, University of California, San Francisco, San Francisco, CA.
Biography
Dr. Neumann is an associate professor, Department of General Practice and Dental Public Health, School of Dentistry, University of Texas Health Science Center at Houston, Houston, TX.
Biography
Dr. Johnson is a professor, School of Biomedical Informatics, University of Texas Health Science Center at Houston, Houston, TX.
Biography
Dr. Olson is a professor and the chair, pediatric dentistry, School of Dentistry, University of Texas Health Science Center at Houston, Houston, TX.
Biography
Ms. Bangar is a research associate, Department of Diagnostic and Biomedical Sciences, School of Dentistry, University of Texas Health Science Center at Houston, Houston, TX.
Biography
Mr. Kookal is a clinical informatics research data warehouse systems analyst, technology services and informatics, School of Dentistry, University of Texas Health Science Center at Houston, Houston, TX.
Biography
Dr. Kim is an instructor, Department of Restorative Dentistry and Biomaterials Sciences, Harvard School of Dental Medicine, Harvard University, Cambridge, MA.
Biography
Dr. Obadan-Udoh is an assistant professor, Department of Preventive and Restorative Dental Sciences, School of Dentistry, University of California, San Francisco, San Francisco, CA.
Biography
Dr. Mertz is a professor, Department of Preventive and Restorative Dental Sciences, School of Dentistry, University of California, San Francisco, San Francisco, CA
Biography
Ms. Simmons is the chief operating officer, Willamette Dental Group, Portland, OR.
Biography
Ms. Mullins is the vice president of operations, Willamette Dental Group, Portland, OR.
Biography
Dr. White is a professor, Department of Preventive and Restorative Dental Sciences, School of Dentistry, University of California, San Francisco, San Francisco, CA.
Biography
Dr. Kalenderian is a lecturer, Harvard School of Dental Medicine, Harvard Univeristy, Cambrige, MA, and a professor, Department of Preventive and Restorative Dental Sciences, School of Dentistry, University of California, San Francisco, San Francisco, CA.
Biography
Dr. Walji is an associate dean for technology services and informatics, and a professor, Department of Diagnostic and Biomedical Sciences, University of Texas School of Dentistry at Houston, 7500 Cambridge St, SOD 4184, Houston, TX 77054.
Article Info
Footnotes
Research reported in this article was supported by award R01DE024166 from the National Institute of Dental and Craniofacial Research, National Institutes of Health. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Disclosure. None of the authors reported any disclosures.
Drs. Kumar and Yansane contributed equally to this article.
Drs. Kalenderian and Walji are co-last authors.
Identification
DOI: https://doi.org/10.1016/j.adaj.2020.06.015
Copyright
© 2020 American Dental Association.
User License
Creative Commons Attribution (CC BY 4.0) |
ScienceDirect
Access this article on ScienceDirect