Detecting Developmental-Behavioral Problems in Primary Care
The 20,941 families participating in screening had elevated psychosocial risk factors and were disproportionately poor, ethnic minorities, non-English speaking, and had lower than average high school graduation rates (compared with U.S. Census Bureau population parameters, www.census.gov, accessed March 2014). Of parents, only 69% had completed high school (compared with 84% nationally); 30% had incomes below poverty guidelines (versus 24% nationally); 34% were non-English speaking (compared with 12% nationally); 31% were Latino (compared with 17% nationally); and 50% were White (compared with 70% nationally). The incidence for other ethnicities was similar to Census Bureau data: 13% were African American and 6% were Asian, American Indian, or Pacific Islanders.
Figure 1 shows the frequency and type of screening test used across the 0- to 8-year age range. Age in months is shown in the 0- to 3-year age range, and age is shown annually thereafter. PEDS was used at 100% of all visits (N = 20,941), followed by the PEDS:DM at 41% of visits (N = 8,657) and the M-CHAT at 21% of visits (N = 4,476). Use of the M-CHAT spiked around 18 months of age and remained high in the months surrounding 24 months of age, suggesting compliance with recommendations for autism-focused screens at these specific ages. The decreasing numbers of children screened after 3 years of age reflects the known and dwindling trajectory of attendance at well-child visits with older children (Selden, 2006).
(Enlarge Image)
Figure 1.
Frequency and types of screening test used by age. PEDS = Parents' Evaluation of Developmental Status; PEDS:DM = Parents' Evaluation of Developmental Status: Developmental Milestones; M-CHAT = Modified Checklist of Autism in Toddlers.
Given elevated psychosocial risk factors in the study sample, it is not surprising that screening test failure rates were higher than the expected prevalence of 13% to 16% (Boyle et al., 2011). Problematic performance on one or more of the three screens was 22% (N = 4,629/20,941). Of families completing PEDS, 17% were at high or moderate risk for developmental and/or mental health problems (N = 3,705/20,941). Of those to whom the PEDS:DM was administered, 17% had two or more unmet milestones (N = 1,468/8,657), and 8% of those to whom the M-CHAT was administered received failing scores (N = 399/4,476).
Figure 2 shows the percentage of children with high/moderate risk scores on PEDS, two or more milestones unmet on the PEDS:DM, and failing scores on the M-CHAT. Visible is the known and predictable increase in delays as children's age increases (Newachek et al., 1998). Children 3 years and older were twice as likely to perform poorly on screens than were children in the birth through 2-year age range (odds ratio [OR] = 2.3, 95% confidence interval [CI] = 2.12–2.43, p < .0001). Thus it is particularly unfortunate that attendance at well-child visits drops precipitously at age 3 years and beyond (as shown in Figure 1).
(Enlarge Image)
Figure 2.
Percentage of problematic performance by age and screening test. PEDS = Parents' Evaluation of Developmental Status; PEDS:DM = Parents' Evaluation of Developmental Status: Developmental Milestones; M-CHAT = Modified Checklist of Autism in Toddlers.
Most children (66%; N = 13,859) were screened at ages coinciding with the AAP's periodicity schedule (±1 month), that is, at approximately 2, 4, 6, 9, 12, 15, 18, 24, 36, 48, 60, 72, and 84 months. Nevertheless, 34% (N = 7,082) were screened between well-child visit ages, indicating substantial use of opportunistic screening/surveillance. Interestingly, children screened outside the well-child visit schedule were 1.5 times more likely to perform poorly on one or more screening tests (OR=1.6, 95% CI = 1.50–1.67, p < .0001).
Via clinic visits and interviews, the following approaches to the use of online screening were found. Staffing patterns unique to each approach are also described.
Parent Portal. Parent portals have a number of advantages. Families can complete screens prior to the visit but do not see results. Rather, the results are sent directly to clinicians, which gives them advance notice about whether referrals are needed, along with information about the type of anticipatory guidance and promotion of development that should be provided. Use of parent portals enables care providers to prepare for patient visits by obtaining brochures regarding referral resources and materials that provide parenting guidance, such as specific handouts of topics of interest to families; it also gives care providers time to practice how to explain the results, if necessary.
The PEDS Online parent portal through which parents can complete screens prior to encounters was used by 24 of the 79 clinics that collectively screened 2,086 children (10% of the 20,941). Of the 24 clinics, 15 were private practices and 5 were community or public health clinics. Parents accessing the portal were more likely to be English-speaking (OR = 3.2, 95% CI = 1.61–6.32, p < .0001) Otherwise there were no differences in presence or absence of portal use due to parents' level of education or poverty level. Of the 24 clinics, 20 clinics used PEDS Online with 56% to 100% of families, while the remaining 4 clinics used PEDS Online less than 35% of the time.
Approaches to Portal use. Clinics encouraged families to use the portal in two different ways:
Uptake on Portal use. Of the two approaches, the clinics with the highest uptake on portal usage were those providing computers in the waiting room (56%, N = 1169/2086). The four clinics with lower rates of portal usage (44%, N = 917/2,086) used the appointment reminder approach.
Staffing Patterns and Portal use. Clinics with computers in waiting rooms often had waiting-room attendants (usually gap year students or retirees paid close to minimum wage) to help parents use computers, probe literacy (by asking if parents preferred assistance), and interview families with limited literacy (e.g., by reading questions aloud). Many of the waiting room attendants were also charged with entertaining children, modeling appropriate adult-child interactions (such as talking to children about their activities), and in many cases implementing Reach and Read, that is, by reading children's books aloud so that parents could complete screens undisturbed.
Interview. Seven clinics administered PEDS Online exclusively by interview to 16% of all patients in the study sample (N = 3,313/20,941). Clinic types included emergency department/crisis call centers, private practices, and community/public health centers. Interview administrations were more common when families did not speak English (OR = 1.8, 95% CI = 1.01–3.22, p < .05) or had elevated poverty levels (OR = 2.0, 95% CI = 1.11–3.65, p < .02).
Approaches and Staffing Patterns. Some clinicians, once entering the examination room, preferred to begin with PEDS Online screens by interview, as an opening to the encounter. In these cases providers gave live interviews while recording responses onto the PEDS Online Web site.
For clinics working with non–English-speaking families, the presence of bilingual (most often English-Spanish speaking) staff was common. Bilingual staff were varied in professional backgrounds and included physicians, skilled nurses, medical technicians, and receptionists.
In some clinics, families were scheduled for well-child visit appointments on days when bilingual staff were present. Group well-child visits were reported by one clinic that had limited bilingual staff via a "Spanish (other) Language Day."
When bilingual staff were not available or when clinics cared for families who spoke neither English nor Spanish, interpretive/translation services were used. Telephone services were the most common method; personnel were provided written translations of PEDS Online measures and then engaged in a three-way call with families and clinic staff, providing back-translation into English. Clinic staff (who were varied in terms of professional backgrounds but most often included skilled nurses) then entered parents' responses into PEDS Online.
Paper-pencil in Waiting or Examination Rooms Together With PEDS Online. The balance of practices (48/79), whether interviewing only a portion of families or making partial use of the parent portal, administered screens by asking parents to complete printed copies of screens in waiting rooms (74% of all families, N = 15,548/20,941). After parents completed measures, responses were entered by clinic staff into PEDS Online to obtain results.
Approaches and Staffing Patterns. In some clinics, clipboards were pre-prepared with printouts of PEDS Online screens, often together with other surveys (such as a checklist of Bright Futures topics, screens for parental depression/psychosocial risk, and the 5-2-1-0 obesity checklist). Receptionists were often charged with disseminating clipboards/measures at check-in.
In several large clinics, receptionists were consumed with patient registration and check-out, and so the medical technician station served as the point for disseminating clipboards/measures (in many clinics, several families were in the medical technician's room at the same time and thus had at least some time to complete measures while they waited).
Next, skilled nurses positioned at (much more private) nursing stations entered parents' responses into PEDS Online, offering an interview if forms were incomplete or if evidence of limited literacy was present (e.g., no words written on the PEDS Response Form and skipped questions on other measures). Nurses also often clarified parents' comments and answers to items and pasted final results into EHRs or printed them out for paper charts.
Measures were sometimes (more frequently in clinics) completed in examination rooms, in which case nurses or other staff accompanied families into examination rooms, helped them finish partially completed forms, gathered responses by interview or on paper, entered results into PEDS Online, and also gathered vital health information. In these cases, computers in the examination room were often available (or were brought in by staff), EHRs were in use, and staff often left the EHR age-specific encounter form open alongside the PEDS Online Web site showing results. In clinics without EHRs, staff entering examination rooms, most often nurses, ensured that paper and pencil forms were complete, and then left the examination room with screening test forms to enter information and to print out results to attach to paper charts.
Hands-on Screening. Some providers preferred to directly administer PEDS:DM items to children. Although this approach was rare, teaching hospital clinics and practices working with medical and nursing students reported using this approach to ensure that trainees mastered skills in behavior management and learned critical milestones. Although not represented in this study, by report, many early intervention intake services, developmental-behavioral clinics, and subspecialty clinics also used hands-on administration for training purposes, to gain more detailed insight into children's challenges, and to meet program/measurement requirements.
Gated Screening. A few practices used a gated screening process wherein PEDS Online was used as the front-line series of screens. Children performing poorly were then seen by a nurse practitioner or developmental specialist for additional assessment. Measures included the Brigance Screens (Brigance, 2014) and the PEDS:DM Assessment Level (Glascoe & Robertshaw, 2007). A few sites used the Denver-II, although this measure has limited accuracy and should be avoided, especially when clinics train young professionals and should set an evidence-based example (Glascoe et al., 1992).
Results
Demographics
The 20,941 families participating in screening had elevated psychosocial risk factors and were disproportionately poor, ethnic minorities, non-English speaking, and had lower than average high school graduation rates (compared with U.S. Census Bureau population parameters, www.census.gov, accessed March 2014). Of parents, only 69% had completed high school (compared with 84% nationally); 30% had incomes below poverty guidelines (versus 24% nationally); 34% were non-English speaking (compared with 12% nationally); 31% were Latino (compared with 17% nationally); and 50% were White (compared with 70% nationally). The incidence for other ethnicities was similar to Census Bureau data: 13% were African American and 6% were Asian, American Indian, or Pacific Islanders.
Screens Administered and Age of Administration
Figure 1 shows the frequency and type of screening test used across the 0- to 8-year age range. Age in months is shown in the 0- to 3-year age range, and age is shown annually thereafter. PEDS was used at 100% of all visits (N = 20,941), followed by the PEDS:DM at 41% of visits (N = 8,657) and the M-CHAT at 21% of visits (N = 4,476). Use of the M-CHAT spiked around 18 months of age and remained high in the months surrounding 24 months of age, suggesting compliance with recommendations for autism-focused screens at these specific ages. The decreasing numbers of children screened after 3 years of age reflects the known and dwindling trajectory of attendance at well-child visits with older children (Selden, 2006).
(Enlarge Image)
Figure 1.
Frequency and types of screening test used by age. PEDS = Parents' Evaluation of Developmental Status; PEDS:DM = Parents' Evaluation of Developmental Status: Developmental Milestones; M-CHAT = Modified Checklist of Autism in Toddlers.
Performance Across the Birth to 8-Year Age Span
Given elevated psychosocial risk factors in the study sample, it is not surprising that screening test failure rates were higher than the expected prevalence of 13% to 16% (Boyle et al., 2011). Problematic performance on one or more of the three screens was 22% (N = 4,629/20,941). Of families completing PEDS, 17% were at high or moderate risk for developmental and/or mental health problems (N = 3,705/20,941). Of those to whom the PEDS:DM was administered, 17% had two or more unmet milestones (N = 1,468/8,657), and 8% of those to whom the M-CHAT was administered received failing scores (N = 399/4,476).
Figure 2 shows the percentage of children with high/moderate risk scores on PEDS, two or more milestones unmet on the PEDS:DM, and failing scores on the M-CHAT. Visible is the known and predictable increase in delays as children's age increases (Newachek et al., 1998). Children 3 years and older were twice as likely to perform poorly on screens than were children in the birth through 2-year age range (odds ratio [OR] = 2.3, 95% confidence interval [CI] = 2.12–2.43, p < .0001). Thus it is particularly unfortunate that attendance at well-child visits drops precipitously at age 3 years and beyond (as shown in Figure 1).
(Enlarge Image)
Figure 2.
Percentage of problematic performance by age and screening test. PEDS = Parents' Evaluation of Developmental Status; PEDS:DM = Parents' Evaluation of Developmental Status: Developmental Milestones; M-CHAT = Modified Checklist of Autism in Toddlers.
Risk of Children Screened Outside the Well-child Visit Schedule
Most children (66%; N = 13,859) were screened at ages coinciding with the AAP's periodicity schedule (±1 month), that is, at approximately 2, 4, 6, 9, 12, 15, 18, 24, 36, 48, 60, 72, and 84 months. Nevertheless, 34% (N = 7,082) were screened between well-child visit ages, indicating substantial use of opportunistic screening/surveillance. Interestingly, children screened outside the well-child visit schedule were 1.5 times more likely to perform poorly on one or more screening tests (OR=1.6, 95% CI = 1.50–1.67, p < .0001).
Implementation of Online Screening by Clinics
Via clinic visits and interviews, the following approaches to the use of online screening were found. Staffing patterns unique to each approach are also described.
Parent Portal. Parent portals have a number of advantages. Families can complete screens prior to the visit but do not see results. Rather, the results are sent directly to clinicians, which gives them advance notice about whether referrals are needed, along with information about the type of anticipatory guidance and promotion of development that should be provided. Use of parent portals enables care providers to prepare for patient visits by obtaining brochures regarding referral resources and materials that provide parenting guidance, such as specific handouts of topics of interest to families; it also gives care providers time to practice how to explain the results, if necessary.
The PEDS Online parent portal through which parents can complete screens prior to encounters was used by 24 of the 79 clinics that collectively screened 2,086 children (10% of the 20,941). Of the 24 clinics, 15 were private practices and 5 were community or public health clinics. Parents accessing the portal were more likely to be English-speaking (OR = 3.2, 95% CI = 1.61–6.32, p < .0001) Otherwise there were no differences in presence or absence of portal use due to parents' level of education or poverty level. Of the 24 clinics, 20 clinics used PEDS Online with 56% to 100% of families, while the remaining 4 clinics used PEDS Online less than 35% of the time.
Approaches to Portal use. Clinics encouraged families to use the portal in two different ways:
Having a waiting room computer kiosk or providing parents with tablet computers at check-in so that families could complete screens in the waiting room on the day of the encounter; or
Giving parents an appointment reminder card, including information on how to log in to the PEDS Online Web site together with a request to complete screens before the next scheduled visit.
Uptake on Portal use. Of the two approaches, the clinics with the highest uptake on portal usage were those providing computers in the waiting room (56%, N = 1169/2086). The four clinics with lower rates of portal usage (44%, N = 917/2,086) used the appointment reminder approach.
Staffing Patterns and Portal use. Clinics with computers in waiting rooms often had waiting-room attendants (usually gap year students or retirees paid close to minimum wage) to help parents use computers, probe literacy (by asking if parents preferred assistance), and interview families with limited literacy (e.g., by reading questions aloud). Many of the waiting room attendants were also charged with entertaining children, modeling appropriate adult-child interactions (such as talking to children about their activities), and in many cases implementing Reach and Read, that is, by reading children's books aloud so that parents could complete screens undisturbed.
Interview. Seven clinics administered PEDS Online exclusively by interview to 16% of all patients in the study sample (N = 3,313/20,941). Clinic types included emergency department/crisis call centers, private practices, and community/public health centers. Interview administrations were more common when families did not speak English (OR = 1.8, 95% CI = 1.01–3.22, p < .05) or had elevated poverty levels (OR = 2.0, 95% CI = 1.11–3.65, p < .02).
Approaches and Staffing Patterns. Some clinicians, once entering the examination room, preferred to begin with PEDS Online screens by interview, as an opening to the encounter. In these cases providers gave live interviews while recording responses onto the PEDS Online Web site.
For clinics working with non–English-speaking families, the presence of bilingual (most often English-Spanish speaking) staff was common. Bilingual staff were varied in professional backgrounds and included physicians, skilled nurses, medical technicians, and receptionists.
In some clinics, families were scheduled for well-child visit appointments on days when bilingual staff were present. Group well-child visits were reported by one clinic that had limited bilingual staff via a "Spanish (other) Language Day."
When bilingual staff were not available or when clinics cared for families who spoke neither English nor Spanish, interpretive/translation services were used. Telephone services were the most common method; personnel were provided written translations of PEDS Online measures and then engaged in a three-way call with families and clinic staff, providing back-translation into English. Clinic staff (who were varied in terms of professional backgrounds but most often included skilled nurses) then entered parents' responses into PEDS Online.
Paper-pencil in Waiting or Examination Rooms Together With PEDS Online. The balance of practices (48/79), whether interviewing only a portion of families or making partial use of the parent portal, administered screens by asking parents to complete printed copies of screens in waiting rooms (74% of all families, N = 15,548/20,941). After parents completed measures, responses were entered by clinic staff into PEDS Online to obtain results.
Approaches and Staffing Patterns. In some clinics, clipboards were pre-prepared with printouts of PEDS Online screens, often together with other surveys (such as a checklist of Bright Futures topics, screens for parental depression/psychosocial risk, and the 5-2-1-0 obesity checklist). Receptionists were often charged with disseminating clipboards/measures at check-in.
In several large clinics, receptionists were consumed with patient registration and check-out, and so the medical technician station served as the point for disseminating clipboards/measures (in many clinics, several families were in the medical technician's room at the same time and thus had at least some time to complete measures while they waited).
Next, skilled nurses positioned at (much more private) nursing stations entered parents' responses into PEDS Online, offering an interview if forms were incomplete or if evidence of limited literacy was present (e.g., no words written on the PEDS Response Form and skipped questions on other measures). Nurses also often clarified parents' comments and answers to items and pasted final results into EHRs or printed them out for paper charts.
Measures were sometimes (more frequently in clinics) completed in examination rooms, in which case nurses or other staff accompanied families into examination rooms, helped them finish partially completed forms, gathered responses by interview or on paper, entered results into PEDS Online, and also gathered vital health information. In these cases, computers in the examination room were often available (or were brought in by staff), EHRs were in use, and staff often left the EHR age-specific encounter form open alongside the PEDS Online Web site showing results. In clinics without EHRs, staff entering examination rooms, most often nurses, ensured that paper and pencil forms were complete, and then left the examination room with screening test forms to enter information and to print out results to attach to paper charts.
Hands-on Screening. Some providers preferred to directly administer PEDS:DM items to children. Although this approach was rare, teaching hospital clinics and practices working with medical and nursing students reported using this approach to ensure that trainees mastered skills in behavior management and learned critical milestones. Although not represented in this study, by report, many early intervention intake services, developmental-behavioral clinics, and subspecialty clinics also used hands-on administration for training purposes, to gain more detailed insight into children's challenges, and to meet program/measurement requirements.
Gated Screening. A few practices used a gated screening process wherein PEDS Online was used as the front-line series of screens. Children performing poorly were then seen by a nurse practitioner or developmental specialist for additional assessment. Measures included the Brigance Screens (Brigance, 2014) and the PEDS:DM Assessment Level (Glascoe & Robertshaw, 2007). A few sites used the Denver-II, although this measure has limited accuracy and should be avoided, especially when clinics train young professionals and should set an evidence-based example (Glascoe et al., 1992).