Introduction
The PhenX (consensus measures for Phenotypes and eXposures) Toolkit is a publicly available catalog of measurement protocols for use in research with human participants that are recommended by experts in their field. PhenX has been funded since 2007 by the National Human Genome Research Institute (NHGRI), with additional funding from other National Institutes of Health (NIH) Institutes and Centers. The goal of the PhenX Toolkit is to provide investigators with a common currency of protocols for biomedical research that they can use with confidence, especially in areas outside of their primary area of expertise. Research that addresses pediatric health often must integrate multiple scientific disciplines, including (but not limited to) genetics, epigenetics, developmental neuroscience, psychology, psychiatry, epidemiology, pediatrics, and public health (Tarazi et al., 2016). The PhenX project strives to include measurement protocols suitable for all life stages, with many protocols in the Toolkit suitable for pediatric populations.
In 2018, the PhenX project recognized that critical areas in pediatric development had not been addressed within the Toolkit (Gitterman et al., 2018; National Institutes of Health, 2015). Accordingly, an interdisciplinary working group (WG) was assembled to identify developmentally informed, broadly validated measurement protocols that address the unique needs of pediatric populations and were not already represented in the Toolkit. This Pediatric Development Working Group (PDWG), using the well-established PhenX consensus process (Maiese et al., 2013), including vetting recommended protocols with the scientific community, identified protocols for inclusion in the PhenX Toolkit. The 18 measurement protocols were released in the PhenX Toolkit in the Pediatric Development domain in December 2019, along with the associated tools and information needed to incorporate these protocols into study designs. This resource is intended to facilitate collaboration and data sharing among pediatric investigators and improve the consistency of data collection, data quality, and reproducibility. The goal of this report is to introduce the scientific community to the new standard protocols in the PhenX Toolkit for Pediatric Development.
Methods
PhenX Project Overview
In 2007, the National Human Genome Research Institute (NHGRI) recognized that identifying and promoting a common currency of standard measures would facilitate data sharing and cross-study analyses and potentially increase the scientific impact of individual studies (National Institutes of Health, 2007). The goal for the resultant PhenX Toolkit is to address a broad range of research topics or domains, with a relatively small number of recommended measurement protocols (~15) per domain, to facilitate investigators’ selection of protocols for their study designs. In general, the protocols in the Toolkit are well established, recommended by experts in the scientific community, and low burden in terms of cost, time, and logistic complexity for both investigators and participants. The Toolkit includes protocols in the languages in which they are available from the source; for example, when the source provides the Toolkit with the protocol in Spanish and Chinese, it is made available to users. All Toolkit protocols are identified by WGs of experts in the relevant research domain.
The PhenX Steering Committee (SC) provides ongoing, overarching guidance to the PhenX project, prioritizes the research domains for the Toolkit, and establishes the initial scope of each domain. PhenX WGs operate within the guidance provided by the SC, including established criteria to inform the selection of protocols to be included in the PhenX Toolkit. For a detailed discussion of SC guidance and the PhenX consensus process, see Maiese et al (Maiese et al., 2013).
PhenX Pediatric Development Working Group
In 2018, the PhenX SC prioritized the addition of a PhenX Pediatric Growth and Development domain to address the need for a set of well-established research protocols for pediatric research. Upon initial review of existing PhenX Toolkit measurement protocols, the SC determined that the Toolkit had robust content on pediatric growth (e.g., anthropometric protocols for measuring the height, weight, and other growth characteristics of a child). Thus, the SC decided on a more focused domain for Pediatric Development. The initial scope for this domain included the areas of development, exposures, social environment, and child lifestyle. The scope, in turn, guided the identification and selection of six experts to serve on the PDWG, which was led by a chair and supported by a PhenX SC Liaison. The PDWG comprised researchers from five academic institutions and the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD), and members had expertise in perinatology, pediatrics, developmental and clinical psychology, public health, and epidemiology.
The PDWG followed the standard process for selecting measurement protocols, which includes an introductory call, a scope review call, a protocol consensus workshop, and a final recommendations meeting. Additional interim meetings were scheduled as needed. Because the PhenX Toolkit covers a wide range of research domains, there were already numerous protocols available for administration to pediatric populations. These included protocols for commonly collected sociodemographic data (e.g., ethnicity and race, annual family income, household roster); diet and nutrition; speech, hearing, and language; and child-specific assessments of temperament, intelligence, autism, school attendance and performance, and environmental exposures. The PDWG was tasked with building on the existing Toolkit content by focusing on the selection of protocols that are unique or critical to pediatric development research needs and that complement existing PhenX Toolkit content.
The PDWG discussed the age ranges to cover with the new protocols. The guidance was to include a continuum from birth through adolescence, as relevant for a given construct; prenatal exposures were also considered applicable for some constructs. The PDWG decided not to pursue the initially proposed scope elements of physical environmental exposures in childhood (e.g., exposure to pollutants, toxins, chemicals, medications) to focus more tightly on pediatric development. Each WG member assumed responsibility for scope elements for which they had expertise. During the Protocol Consensus Workshop, each member presented their findings from a literature review and made recommendations about which protocols to include, given the guidance from the SC. Their consideration included identifying existing protocols already in the Toolkit that sufficiently addressed their assigned scope element to avoid adding competing protocols to the Toolkit.
Following PhenX guidance, the PDWG prioritized consideration of existing, well-established measures that are in the public domain. Measurement protocols were assessed in terms of the burden and requirements for both study participants and investigators. Burden of use was assessed in terms of time, equipment, training, and data collection requirements. The PDWG considered measures outside of the public domain if those measures were well-established and best addressed the construct of interest or if suitable measures did not exist in the public domain. Because the PhenX Toolkit is a US effort, the primary focus was on measures for use in domestic research, but the PDWG considered international measurement standards as appropriate.
The PDWG initially selected 20 protocols for distribution to and feedback from the scientific community, including registered users of the PhenX Toolkit and other project participants, such as National Institutes of Health (NIH) liaisons, prior and current members of other PhenX WGs, and PhenX SC members. Respondents were asked to provide comments about the appropriateness, usability, and limitations of the proposed protocols and the value of including each in the PhenX Toolkit. There was general agreement about the inclusion of these recommended protocols, with lack of public access noted as a limitation for those protocols that were not the public domain. In the final meeting to review feedback from outreach, the PDWG considered and agreed with this limitation but concluded that the four proprietary protocols were the most widely used and that there were not alternative protocols publicly available that met the criteria of being well-established or providing sufficient coverage of the scope element.
The PDWG selected 18 measurement protocols (described in Table 1) to include in the new Pediatric Development domain of the PhenX Toolkit. As with all PhenX Toolkit protocols, a bioinformatics pipeline processed information about these 18 measurement protocols for loading into the Toolkit, and protocol-specific data dictionaries and data collection worksheets were developed (Cox et al., 2021). The PDWG agreed that an additional 2 protocols were important resources but were more appropriate as Supplemental Information reference materials. The first Supplemental Information protocol, Child’s Living Environment, was from the Avon Longitudinal Study of Parents and Children (ALSPAC). This protocol includes extensive use of British English words (e.g., English heating systems, currency), potentially limiting its usefulness in US-based research, which is the main target of the NIH-funded PhenX Toolkit. However, some researchers may find the protocol valuable as a model for collecting data regarding a child’s living environment. Another ALSPAC-derived protocol, Child’s Lifetime Household Composition, was included as one of the 18 Pediatric Development measurement protocols, because it included very limited use of British English. The other Supplemental Information protocol, the Youth Alcohol Screener, does not provide sufficient details to ensure consistent data collection and thus did not meet criteria for inclusion in the Toolkit; however, it may be of interest to researchers requiring a brief screening measure.
Results
Measurement Protocols in the PhenX Toolkit
Table 1 presents the 18 Pediatric Development measurement protocols selected by the PDWG. These protocols cover a range of areas of child and caregiver functioning and psychosocial exposures relevant to pediatric development research. Specifically, the Pediatric Development domain includes validated measurement protocols of neonatal abstinence syndrome and opioid withdrawal syndrome; child care/preschool quality; academic performance; quality of peer relationships; bullying experiences; child mental health; psychological resilience; household stability, quality, and functioning; and caregiver mental health and parenting style. The domain includes both self-report and other-report measurement protocols, where relevant, to provide flexibility in study design and comprehensive assessments of children’s functioning and their environments.
The protocols are largely questionnaires or interviews that can be flexibly administered with a low burden in a variety of settings with minimal training; only one protocol, the Neonatal Abstinence Syndrome (NAS) and Neonatal Opioid Withdrawal Syndrome (NOWS), requires independent clinical observation with specialized training. The PDWG considered existing Toolkit content carefully when recommending these measurement protocols for inclusion in the Pediatric Development domain. Researchers are encouraged to use the life-stage filter within the search function of the Toolkit to identify the large number of measures that could be relevant to their investigations of child health and development, including many measures outside of the Pediatric Development domain. The measurement protocols added by the PDWG build on the existing content to add depth to the Toolkit for pediatric research.
Discussion
The PhenX Toolkit is a web-based resource (http://www.phenxtoolkit.org/) that researchers can use to inform their choice of protocols or to broaden their studies by adding protocols outside the scope of their original study design or their range of expertise. The Toolkit is not prescriptive, so investigators may select study-specific protocols from the PhenX Toolkit that best meet their needs within the context of available resources, study design, and research questions. Use of common measures allows for easier comparability of findings across studies. Because the majority of protocols in the PhenX Pediatric Development Domain have established validity with low burden and low training needs, they can be used by investigators and clinicians for whom pediatric well-being is a primary research focus as well as by researchers with expertise in other areas who would like to adopt pediatric measures into their studies. Incorporating well-established and validated protocols during the initial stages of study design will ensure that data are collected according to accepted standards (Kahn et al., 2014). The development of a pediatric domain within the PhenX Toolkit aligns with NIH’s recognition of the critical importance of pediatric research and consequent policies requiring the inclusion of children in clinical research (Gitterman et al., 2018; Lauer, 2018; National Institutes of Health, 2015). Notably, PhenX, an NIH-supported endeavor, has been cited in 520 NIH funding opportunity announcements to date; thus, investigators can be confident that proposed use of PhenX protocols will be well received by NIH review panels.
The adoption of common, validated protocols into pediatric development research is timely in light of recent efforts by funding agencies, including the NIH, to support “big science” that combines data across existing datasets and collects data from multiple cohorts across settings to increase sample sizes and, consequently, the power to address pressing, complex developmental questions for improving child health (e.g., Environmental influences on Child Health Outcomes, or ECHO) (Gillman & Blaisdell, 2018). The use of common measurement protocols minimizes the challenges inherent in harmonizing data collected across studies that use different instruments and more easily allows validation of study findings through replication of results across studies that used the same protocols for data collection.
Greater implementation of common research measures also would address several scientific concerns that have emerged in recent years. For example, failed replication of published findings raises questions about the validity of the findings and may consequently lead to declines in public support of research funding and reluctance to implement health recommendations that emerge from such studies (Chung, 2017; Maxwell et al., 2015). Potential solutions to this problem include conducting multiple replication studies and studies with large sample sizes, which would require coordinated efforts among multiple investigators and informed selection of well-validated measures of the constructs of interest (Maxwell et al., 2015).
Harmonization across studies can be hindered by differing definitions of outcomes and heterogeneity of data collection methods across studies, which can limit direct between-study comparisons, meta-analysis of data, and the combining of datasets across studies to create large sample sizes (Kahn et al., 2014; Malinowski et al., 2017). Moreover, the adoption of standard measures would support more effective and efficient use of limited resources by allowing the combination of separate datasets into larger composite datasets that can be used to address novel research questions that may not have been considered at the time of data collection (Kahn et al., 2014). These efforts would have the potential to increase the scientific impact of individual studies.
Use of standard measures will have particular importance in the study of rarer conditions in small- to moderate-sized populations, allowing for statistical power that is typically not achievable via individual studies (Kahn et al., 2014). Notably, in recent years, there has been unprecedented growth in the collection of longitudinal patient-level pediatric data as a result of the adoption of a variety of electronic systems in clinical settings (Kahn et al., 2014). The PhenX Toolkit provides a resource for clinicians to identify easily a set of established research measures, including measures outside of their area of expertise, that then can be incorporated into electronic health systems, enabling the expansion of the size, scope, and reach of pediatric research by including clinical settings that historically may have been excluded from the scientific base (Kahn et al., 2014).
Limitations
Although freely available protocols were preferred for inclusion in the Toolkit, the PDWG selected some protocols that are categorized in the Toolkit as “limited availability,” meaning that users must contact the source to obtain access (e.g., register for permission to use, pay fees for usage). Such measures were chosen when the PDWG determined that a suitable, freely available protocol did not exist. Additionally, in its review of the literature, the PDWG acknowledged important areas for which measures could not be identified that met PhenX Toolkit requirements (e.g., well-established, low burden) or have not yet been developed. As described below, these are areas for further research and exploration. Also included below is a description of challenges the PDWG encountered around the language used in some of the selected measures for inclusion in the Toolkit.
Child Sleep
The PDWG recognized the critical importance of sleep to child health and well-being (Armstrong et al., 2014; Baddam et al., 2018; Cook et al., 2020; Li et al., 2017; Martin et al., 2019; Smaldone et al., 2009). However, the group was unable to identify a broadly validated instrument that it could recommend for use in pediatric populations. The PDWG sought input from the research community during outreach but received no protocol suggestions.
Anxiety in Pregnancy
The PDWG noted the large body of scientific literature demonstrating the significant impact of maternal mental health in pregnancy on a range of pediatric health outcomes (Becker et al., 2016; Korja et al., 2017; Madigan et al., 2018; Van den Bergh et al., 2020). Pregnancy and the postpartum period present specific demands that may not allow for standard mental health assessments to be used validly in the peripartum period but rather require tailored instruments. For example, diagnostic criteria for depression include several symptoms that may be present in pregnancy and not be indicative of psychological distress (e.g., changes in appetite, weight, and sleep). At the time the PDWG was convened, the PhenX Toolkit already included a validated measurement protocol of maternal depressive symptoms during the peripartum period (Edinburgh Postnatal Depression Scale [Gibson et al., 2009]) that takes into account these considerations. However, the PDWG could not identify an appropriate instrument for assessing anxiety in pregnancy.
Although several clinical interviews and questionnaires exist for evaluating general anxiety in the PhenX Toolkit (e.g., anxiety subscale of the Symptom Check List 90 [Derogatis et al., 1973]; Structured Clinical Interview for DSM-5® Disorders-Clinician Version [First et al., 2015]), these protocols do not distinguish clinical anxiety symptoms from common experiences resulting from pregnancy and in caring for a newborn infant (e.g., somatic symptoms). They therefore may have decreased validity during the peripartum period. Furthermore, these measures do not address pregnancy-related anxiety (i.e., anxiety specific to worries about the health of the fetus/infant and concerns about the labor and delivery process). The extant literature suggests that pregnancy-related anxiety is distinct from general anxiety and depressive symptoms and has specific impact on maternal and child outcomes (Brunton et al., 2015, 2018; Fallon et al., 2016). Although the PDWG identified several pregnancy-related anxiety measures in the literature, none of them met PhenX Toolkit criteria for inclusion because they lacked sound theoretical or psychometric properties or sufficient scope and depth for adequate assessment. Given that the pregnancy and postpartum periods are known risk periods for the onset or exacerbation of maternal mental health difficulties and that maternal mental health during these periods can have major, long-term impact on child health (Cameron et al., 2016; Falah-Hassani et al., 2017; Shorey et al., 2018), the PDWG encourages researchers to develop standardized, validated instruments that specifically address the unique considerations of assessing maternal mental health comprehensively during these periods.
Parenting and Caregiving Quality
A large body of research indicates the importance of parenting/caregiving quality, particularly in early life, on child health outcomes, and direct observation is considered the “gold standard” for assessing caregiving quality (Bosquet Enlow et al., 2014; Koehn & Kerns, 2018; Pallini et al., 2018; Spruit et al., 2020; Suglia et al., 2009). However, assessing caregiving quality by direct observation is quite burdensome given the need to have highly trained coders score observations that are collected in a standardized format. Therefore, the PDWG elected to recommend the Alabama Parenting Questionnaires, well-established instruments that are easy to use and involve much less burden on the investigator and the participant. In addition, the recommended Home Observation for Measurement of the Environment protocol, also well established, may be used to capture additional information about aspects of the caregiving environment. Studies that involve caregiving quality as a central construct may elect to use an established observation measure but consider using the PDWG-recommended questionnaires as adjunct measures.
Similarly, standardized observational data collection protocols exist for assessing the quality of a child’s daycare, child care, or preschool environment. However, the PDWG determined that such measures were highly burdensome because they require trained coders to conduct hours of observation in a child’s care setting. Further, the PDWG concluded that such observational measures would provide data at a level of detail unlikely to be needed by pediatric studies outside of those centrally focused on daycare or preschool quality. Rather, the PDWG chose a protocol that addresses parents’ or caregivers’ perceptions of their child’s care, including questions related to the reliability, cost, and quality of care, because these characteristics were determined to be particularly relevant for child health outcomes and measurable via caregiver report (Charrois et al., 2017; Geoffroy et al., 2006; Love et al., 2003; Vandell et al., 2010).
Evolving Landscape and Language
The PDWG confronted challenges when considering protocols with questions based on outdated assumptions (e.g., heterosexual nuclear family structure, obsolete technology available in the home). Ultimately, the PDWG elected to include these measures in the Pediatric Development domain because the protocols were the best available to assess the constructs considered essential for inclusion. However, because existing protocols are not modified for inclusion in the PhenX Toolkit, the PDWG recognizes that these measures may need to be modified by users for their specific study populations and encourages investigators to undertake efforts to update these measures, including conducting studies to document the validity of revised measures.
Additionally, although the PDWG chose measurement protocols that have been validated in various populations, the group recognizes the need for further research on how the protocols may function differently in different sociodemographic groups. Of the 18 pediatric protocols added to the Toolkit, three are available from the source in languages other than English; two of these are not open source and therefore cannot be displayed in the Toolkit but can be obtained by contacting the publisher. Before implementing any PhenX Toolkit measurement protocol in a study, investigators should consider the suitability of the measurement protocol for their particular study population and research questions.
Biomarkers
The PDWG recognized the importance of biomarkers in predicting and tracking pediatric outcomes and considered a range of wearable devices and exposure measures. However, the group was unable to identify any biomarker measures that had achieved sufficient technological advancement and met PhenX requirements of being clearly defined, well established, low burden, and broadly validated to be recommended at this time for wide implementation. These protocols are expected to evolve such that they can be considered by a future PhenX WG focused on biomarker assessments.
Conclusions
Pediatric research aims to prevent and treat disease and to optimize children’s physical, mental, and developmental health outcomes. Preventing or resolving poor health among youth not only reduces suffering and psychosocial burdens in childhood but also promotes improved health and economic productivity into adulthood (Gitterman et al., 2018; Goldfeld et al., 2019). Moreover, an increasing body of research strongly indicates that several physical and mental health outcomes that emerge or persist in adulthood have origins in early life, further emphasizing the importance of pediatric research in maximizing human health, productivity, and well-being across the lifespan (Almquist, 2012; Appleton et al., 2012; Beach et al., 2021; Braveman & Barclay, 2009; Gitterman et al., 2018; Johnson, 2018; Johnson et al., 2012; Kravitz-Wirtz, 2016; Kuhlman et al., 2018; Messer et al., 2015; Roustit et al., 2011; Shonkoff et al., 2009; Sonu et al., 2019; Umberson et al., 2014).
The PhenX PDWG endeavored to provide the pediatric research and clinical community with a set of measurement protocols to address critical areas for child health and well-being and complement pediatric protocols already included in the PhenX Toolkit. The PDWG added 18 protocols to measure a range of indicators of child functioning and developmental contexts or psychosocial exposures. Incorporating standard protocols in research studies of children will foster collaboration and data sharing, provide consistency in data collection, and enable the harmonization of datasets across studies and cohorts to permit replication of study findings and sufficient power to address complex developmental questions.
The PhenX Toolkit also serves as a powerful, efficient resource to aid investigators searching for appropriate measures outside of their area of expertise to include in their studies. This resource provides data collection support, including available data collection worksheets to simplify creating data collection forms and data dictionaries to facilitate data management and integration into Research Electronic Data Capture (REDCap).
Using the PhenX Toolkit may help investigators respond to NIH’s growing emphasis on and requirements for data management and sharing practices consistent with FAIR (Findable, Accessible, Interoperable, and Reusable) data principles (National Institutes of Health, 2020; Wilkinson et al., 2016). The PhenX Toolkit was designed to address these needs, to hasten research advances, and ultimately, to maximize the potential impact of efforts to improve child health.
Acknowledgments
Thank you to the PhenX Toolkit team for their contributions in operationalizing and maintaining the PhenX Toolkit website. Thank you also to the PhenX Steering Committee for feedback and support throughout the process of developing this domain. Research reported in this publication was provided by a Genomic Research Grant (U41HG007050) from National Human Genome Research Institute (NHGRI), with current or prior funding support from the National Institute on Drug Abuse (NIDA); Office of Behavioral and Social Sciences Research (OBSSR); National Institute of Mental Health (NIMH); National Heart, Lung, and Blood Institute (NHLBI); National Institute on Minority Health and Health Disparities (NIMHD); and Tobacco Regulatory Science Program (TRSP).
Conflict of Interest
The authors have no conflicts of interest or financial relationships relevant to this article to disclose.
Disclaimer
This information or content and conclusions are those of the authors and should not be construed as the official position or policy of, nor should any endorsements be inferred by, the National Institutes of Health (NIH), the US Department of Health and Human Services (HHS), or the US Government.
RTI Press Associate Editor: Michael Willoughby