The Open Psychology Journal




ISSN: 1874-3501 ― Volume 12, 2019
RESEARCH ARTICLE

Formal Psychological Assessment for Autism Spectrum Disorder Diagnosis: A New Methodology to Build An Adaptive Testing System



Maria Chiara Pino1, *, Andrea Spoto2, Melania Mariano1, Umberto Granziol2, Sara Peretti1, Francesco Masedu1, Marco Valenti1, 3, Monica Mazza1, Giulio Vidotto2
1 Department of Applied Clinical Sciences and Biotechnology, University of L’Aquila, L’Aquila, Italy
2 Department of General Psychology, University of Padua, L’Aquila, Italy
3 Reference Regional Centre for Autism, Abruzzo Region Health System, L’Aquila, Italy

Abstract

Background:

Early identification of Autism Spectrum Disorder (ASD) is of paramount importance in establishing effective interventions. Currently, the accepted gold standard for ASD diagnosis is the Autism Diagnostic Observation Schedule, 2nd Edition (ADOS-2).

Methods:

In our research study, we applied a novel approach to the ADOS-2 known as the Formal Psychological Assessment (FPA). The innovation of this methodology comes from the construction of a matrix that allows (1) the identification of the existing relations among items in terms of clinical symptoms and (2) an adaptive assessment of an individual.

Objective:

The aim of this study was to use a practical application of the FPA on the ADOS-2 and show its potential for psychological assessment of individuals with ASD. Particularly, this research raises some important considerations regarding the evaluation of this population and the authors propose future work to improve the quality of clinical assessment.

Results:

Our results suggest potential for the application of this new methodology to the ADOS-2 to support clinicians using diagnostic procedures.

Conclusion:

The use of FPA allows analysis of each item of ADOS’s modules based on the presence/absence of clinical elements. This system allows the identification of the critical areas for each individual and reduces the number of items required for the test.

Keywords: Autism Spectrum Disorder (ASD), Autism Diagnostic Observation Schedule, Second Edition (ADOS-2), Diagnosis, Formal Psychological Assessment (FPA), Adaptive testing system, Methodology.


Article Information


Identifiers and Pagination:

Year: 2018
Volume: 11
First Page: 112
Last Page: 122
Publisher Id: TOPSYJ-11-112
DOI: 10.2174/1874350101811010112

Article History:

Received Date: 13/03/2018
Revision Received Date: 27/05/2018
Acceptance Date: 13/06/2018
Electronic publication date: 29/06/2018
Collection year: 2018

Article Metrics:

CrossRef Citations:
0

Total Statistics:

Full-Text HTML Views: 1455
Abstract HTML Views: 936
PDF Downloads: 555
ePub Downloads: 520
Total Views/Downloads: 3466

Unique Statistics:

Full-Text HTML Views: 739
Abstract HTML Views: 386
PDF Downloads: 230
ePub Downloads: 174
Total Views/Downloads: 1529
Geographical View

© 2018 Pino et al.

open-access license: This is an open access article distributed under the terms of the Creative Commons Attribution 4.0 International Public License (CC-BY 4.0), a copy of which is available at: (https://creativecommons.org/licenses/by/4.0/legalcode). This license permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.


* Address correspondence to this author at the Department of Applied Clinical Sciences and Biotechnology, University of L’Aquila, 67100 L’Aquila, Italy; Tel: 00390862434658; E-mail: mariachiara.piano@univao.it




1. INTRODUCTION

The prevalence of Autism Spectrum Disorder (ASD) has been steadily increasing. It is worth identifying individuals with ASD because of the societal costs due to the occurrence of this disorder and the importance of early diagnosis for rehabilitative benefits. In fact, the increase of ASD diagnosis over the last few years has become a much-debated topic [1Lai MC, Lombardo MV, Baron-Cohen S. Autism. Lancet 2014; 383(9920): 896-910.[http://dx.doi.org/10.1016/S0140-6736(13)61539-1] [PMID: 24074734] ]. The quality of clinical evaluation has an important role in diagnosis, as well as for future treatment [2De Berardis D, Fornaro M, Orsolini L, et al. Effect of agomelatine treatment on C-reactive protein levels in patients with major depressive disorder: An exploratory study in “real-world,” everyday clinical practice. CNS Spectr 2017; 22(4): 342-7.[http://dx.doi.org/10.1017/S1092852916000572] [PMID: 27702411] -4Veltro F, Mazza M, Vendittelli N, Alberti M, Casacchia M, Roncone R. A comparison of the effectiveness of problem solving training and of cognitive-emotional rehabilitation on neurocognition, social cognition and social functioning in people with schizophrenia. Clin Pract Epidemiol Ment Health 2011; 7: 123-32.[http://dx.doi.org/10.2174/1745017901107010123] [PMID: 21792373] ]. ASD can be distinguished by a pattern of multiple symptoms and is typically identified around 2 years of age (or later in the case of regressive forms of ASD). The symptoms of ASD include deficits in social communication and social interaction as well as the presence of repetitive and stereotypical behaviours [5American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders: DSM-V, 5 the dn 2013.].

The clinical diagnosis of ASD can be discerned by combining information from observing the child with parental information. Indeed, a correct evaluation aims to clarify the diagnostic process, which includes both meeting with the child and the family to thoroughly review the developmental history, as well as the current symptom profile. The evaluation includes the administration of the gold standard of diagnostic tools: The Autism Diagnostic Observation Schedule (ADOS-2) [6Lord C, Rutter M, DiLavore PC, Risi S, Gotham K, Bishop SL. Autism diagnostic observation schedule. J Psyedu Assess 2nd edition (ADOS-2) manual (Part I): Modules 1–4. 2012; 88-92.2nd edition (ADOS-2) manual (Part I): Modules 1–4.]. Individuals with autism are subjected to various assessments throughout their life to monitor their symptomatology following rehabilitation treatments. For this reason, it would be useful to have a test that allows faster evaluation of the symptomatology while also deepening the critical areas. Time reduction while preserving accuracy is an important and ambitious objective worth achieving [7Donadello I, Spoto A, Sambo F, Badaloni S, Granziol U, Vidotto G. ATS-PD: An Adaptive Testing System for Psychological Disorders. Educ Psychol Meas 2017; 77(5): 792-815.[http://dx.doi.org/10.1177/0013164416652188] [PMID: 29795931] ]. In this paper we propose the application of the Formal Psychological Assessment (FPA) [8Bottesi G, Spoto A, Freeston MH, Sanavio E, Vidotto G. Beyond the score: Clinical evaluation through formal psychological assessment. J Pers Assess 2015; 97(3): 252-60.[http://dx.doi.org/10.1080/00223891.2014.958846] [PMID: 25257993] , 9Spoto A, Bottesi G, Sanavio E, Vidotto G. Theoretical foundations and clinical implications of formal psychological assessment. Psychother Psychosom 2013; 82(3): 197-9.[http://dx.doi.org/10.1159/000345317] [PMID: 23549091] ] to the ADOS-2, the clinical test defined as the gold standard for ASD diagnosis, to reduce the time required for the test and to allow an adaptive assessment of an individual with ASD. Its introduction in psychological assessment will allow a deeper understanding of the evaluation process for an adaptive assessment procedure [7Donadello I, Spoto A, Sambo F, Badaloni S, Granziol U, Vidotto G. ATS-PD: An Adaptive Testing System for Psychological Disorders. Educ Psychol Meas 2017; 77(5): 792-815.[http://dx.doi.org/10.1177/0013164416652188] [PMID: 29795931] ] and a clear connection between the item asked and the clinical elements collected.

In addition, we aim at validating the clinical structures obtained for both the T module and module 1 of the tool. This process involves estimating the parameters and testing the fit of the model to the available data. This operation allows for the calibration of an adaptive version of the ADOS-2 able to provide detailed information about the symptoms displayed by each patient according to the FPA methodology.

1.1. The Formal Psychological Assessment

When assessing an individual, a clinician is more interested in the symptoms the individual presents rather than in the score he/she obtains on an instrument. The FPA produces an attempt to provide the clinician with exhaustive information about a patient’s specific set of clinical issues, to identify the situation in a both quantitatively and qualitatively rigorous way. What allows the FPA to carry out this critical task are its mathematical and theoretical foundations. In fact, the FPA is a conjoint application into the clinical framework of two theories from mathematical psychology: The Knowledge Space Theory [10Doignon JP, Falmagne JC. Knowledge Spaces 1999., 11Falmagne JC, Doignon JP. Learning Spaces 2011.[http://dx.doi.org/10.1007/978-3-642-01039-2] ], and formal concept analysis [12Ganter B, Wille R. Formal concept analysis: Mathematical Foundations 1999.[http://dx.doi.org/10.1007/978-3-642-59830-2] ]. A very trivial intuition behind the FPA’s conceptualization is the possibility of describing each item of an instrument in terms of the clinical-diagnostic issues it explores [8Bottesi G, Spoto A, Freeston MH, Sanavio E, Vidotto G. Beyond the score: Clinical evaluation through formal psychological assessment. J Pers Assess 2015; 97(3): 252-60.[http://dx.doi.org/10.1080/00223891.2014.958846] [PMID: 25257993] , 13Spoto A, Stefanutti L, Vidotto G. Knowledge space theory, formal concept analysis, and computerized psychological assessment. Behav Res Methods 2010; 42(1): 342-50.[http://dx.doi.org/10.3758/BRM.42.1.342] [PMID: 20160314] ]. In this way, it is possible to identify which clinical issues are investigated by any item of an assessment tool for some specific disorder. In the FPA, each item is named a clinical object, while each clinical-diagnostic issue (e.g., DSM diagnostic criteria) is named a clinical attribute. Starting from these basic definitions it is possible to build a matrix where each row is an object and each column is an attribute. Each cell ij of such a matrix, named clinical context, contains either a 1, whenever the item i investigates the attribute j, or a 0 otherwise. The clinical context displays and represents the object-attribute assignment. Notice that, once the set of attributes is specified, any item can be analysed with respect to the presence/absence of each attribute; this means that the same matrix could even contain items from different tools. Furthermore, this representation allows the clinician to discriminate between response patterns having the same numerical score. This aspect is particularly relevant because a patient could reach a given score by satisfying different sets of symptoms: each item is therefore important not for the score it provides, but for the attributes it investigates. This possibility is completely lost in traditional tools, and it represents one of the most relevant improvements provided by the FPA that differentiates patients not based on their score, but on the clinical symptoms they present.

Starting from the clinical context, through some formal passages [13Spoto A, Stefanutti L, Vidotto G. Knowledge space theory, formal concept analysis, and computerized psychological assessment. Behav Res Methods 2010; 42(1): 342-50.[http://dx.doi.org/10.3758/BRM.42.1.342] [PMID: 20160314] ] a clinical structure can be built. This structure uses the information included in the context to derive the prerequisite relations among the items of the context. This passage is crucial in the construction of an adaptive algorithm for psychological assessment [7Donadello I, Spoto A, Sambo F, Badaloni S, Granziol U, Vidotto G. ATS-PD: An Adaptive Testing System for Psychological Disorders. Educ Psychol Meas 2017; 77(5): 792-815.[http://dx.doi.org/10.1177/0013164416652188] [PMID: 29795931] ]. In fact, the structure defines the admissible response patterns (also named the clinical states) that are defined by the item-attribute assignment of the context. For instance, if an item i1 investigates attributes {a1, a2} and a second item i2 investigates attributes {a1, a2, a3}, any pattern including item i2 but not i1 will be excluded from the structure given the item-attribute assignment.

So far, the deterministic skeleton of the FPA has been described. In the deterministic case no error is assumed during the assessment phase; in other words, the answer to a specific item is assumed not to be affected by any kind of error, and therefore the collected information is treated as certain. It is clear how such an assumption does not adequately describe reality for two main reasons. On the one hand, two main errors could occur in the assessment phase; for instance, in the case of the ADOS-2, the clinician could report a certain behaviour even if it did not occur, or could miss a behaviour that did occur. These two kinds of errors are respectively a “false positive” and a “false negative” [8Bottesi G, Spoto A, Freeston MH, Sanavio E, Vidotto G. Beyond the score: Clinical evaluation through formal psychological assessment. J Pers Assess 2015; 97(3): 252-60.[http://dx.doi.org/10.1080/00223891.2014.958846] [PMID: 25257993] , 11Falmagne JC, Doignon JP. Learning Spaces 2011.[http://dx.doi.org/10.1007/978-3-642-01039-2] ]. On the other hand, the clinical states (i.e., the different clinical conditions in which a patient could be) could present different prevalence rates in the population. For these reasons, the introduction of a probabilistic model is needed. In the FPA, a model that accounts for both error rates and probability of the states is the Basic Local Independence Model (BLIM) [10Doignon JP, Falmagne JC. Knowledge Spaces 1999.]. Within the BLIM, all the responses to the items are locally independent given the state of the individual; moreover, a probability value is associated with each state. Formally, the clinical structure becomes a probabilistic clinical structure (Q, , p) where (Q,) is the clinical structure and p is a probability distribution of the states of . Given (Q,, p), each response pattern has a probability value: This last one is defined by means of a response function which assigns each response pattern its conditional probability given a state K (for all states K in [13Spoto A, Stefanutti L, Vidotto G. Knowledge space theory, formal concept analysis, and computerized psychological assessment. Behav Res Methods 2010; 42(1): 342-50.[http://dx.doi.org/10.3758/BRM.42.1.342] [PMID: 20160314] , 14Falmagne JC, Doignon JP. A class of stochastic procedures for the assessment of knowledge. Br J Math Stat Psychol 1988; 41: 1-23.[http://dx.doi.org/10.1111/j.2044-8317.1988.tb00884.x] ]:

The conditional probability p (R, K) is determined by the two error parameters β and η, respectively the false negative and false positive errors related to each item, as expressed by the following equation:

The model parameters are the two error rates ηq and βq for each item, and a probability value πK for each state. In this way, it is possible to estimate the probability of observing the real clinical state of a patient.

2. METHODS

2.1. Context Construction and Validation

The first step in applying the FPA is the construction of the clinical context that depicts the item/attribute assignment. This operation first requires the specification of the attributes that must be included in the context. Recalling that the clinical attributes can be symptoms, diagnostic criteria or theoretical considerations related to the disorder at hand, this operation has been conducted by referring to the DSM-5 (APA, 2013) and other information from the literature [1Lai MC, Lombardo MV, Baron-Cohen S. Autism. Lancet 2014; 383(9920): 896-910.[http://dx.doi.org/10.1016/S0140-6736(13)61539-1] [PMID: 24074734] , 6Lord C, Rutter M, DiLavore PC, Risi S, Gotham K, Bishop SL. Autism diagnostic observation schedule. J Psyedu Assess 2nd edition (ADOS-2) manual (Part I): Modules 1–4. 2012; 88-92.2nd edition (ADOS-2) manual (Part I): Modules 1–4.]. Table 1 lists the main attributes considered when assessing ASD symptoms.

Table 1
List of attributes of the clinical context for the ASD diagnosis.


The second operation conducted to build the context was the selection of the items. In the current application, all items of the ADOS-2 were included in the context. To have a clearer picture of the instrument, the five main parts of the tool were analysed separately. Therefore, the items of the ADOS-2 were subdivided into five different contexts (one for each part). Finally, six experts (four psychologists and two child psychiatrists) in the field of ASD and in the administration of the ADOS-2 independently performed the item/attribute assignment. Disagreements among raters were solved through direct discussion about the specific criticisms.

Whenever a clinical context is built, five important and informative configurations may occur due to the item-attribute assignment. First, some empty rows may occur (i.e., rows of the matrix that contain only 0s), meaning that the specific item does not investigate any of the selected attributes. Second, some empty columns may occur, indicating that none of the items of the tool investigate that specific attribute. This is crucial because it indicates that the assessment tool at hand does not investigate some of the selected (and thus necessary) attributes. Third, equal columns may occur (i.e., columns having 1s in the same positions) indicating that it is impossible to distinguish between the presence/absence of either of the equivalent attributes in the assessment. Fourth, equivalent rows may occur, which indicate equivalent items (i.e., items that convey, from a clinical perspective, the same information). These items are redundant and thus, in terms of efficiency, could be collapsed. Fifth, some rows are present that are included in other rows (i.e., the 1s in one row are all present in the other row that also has some other 1s). This last case is important because the included items represent so-called prerequisites of the included ones. In other words, in this case there is one item that investigates some attributes and another item that investigates all the attributes of the first item, plus some other attributes. Therefore, from a clinical perspective, an affirmative answer to this last item implies an affirmative answer to the first one as well.

The last step of the application of the FPA is the statistical testing of the robustness of the model in describing the collected data. To this aim, the BLIM parameters have been estimated starting from a dataset of patients evaluated by means of the ADOS-2. Parameter estimation has been iteratively carried out based on the Expectation-Maximization (EM) algorithm [9Spoto A, Bottesi G, Sanavio E, Vidotto G. Theoretical foundations and clinical implications of formal psychological assessment. Psychother Psychosom 2013; 82(3): 197-9.[http://dx.doi.org/10.1159/000345317] [PMID: 23549091] , 13Spoto A, Stefanutti L, Vidotto G. Knowledge space theory, formal concept analysis, and computerized psychological assessment. Behav Res Methods 2010; 42(1): 342-50.[http://dx.doi.org/10.3758/BRM.42.1.342] [PMID: 20160314] , 15Dempster AP, Laird NM, Rubin DB. Maximum likelihood estimation from incomplete data via the EM algorithm (with discussion). J R Stat Soc Series B Stat Methodol 1977; 39(1): 1-38.]. Through this procedure, we were able to obtain an estimate of the two error parameters for each item (false negative and false positive) and the probability of each clinical state in the structure. The procedure computes a fit statistic based on Pearson’s Chi-square; this statistic, due to the sparseness of this kind of data matrix, is not reliable. Therefore, the obtained value has been tested by means of a parametric bootstrap (with 5000 replications) to check for the reliability of the obtained value. Of course, the estimate of the error rates for each item can be viewed as fit indices. It has been observed [9Spoto A, Bottesi G, Sanavio E, Vidotto G. Theoretical foundations and clinical implications of formal psychological assessment. Psychother Psychosom 2013; 82(3): 197-9.[http://dx.doi.org/10.1159/000345317] [PMID: 23549091] ] that, in general, such values must be as low as possible. More precisely, the following condition must hold true for every item: ηq < 1- βq. In other words, a positive answer to an item is more likely to occur if the patient has its attributes, rather than as the result of a false positive on the item. In the next section, the results obtained from the construction and analysis of the contexts are displayed.

The BLIM was defined and tested on the collected data.

3. RESULTS

3.1. Clinical Contexts

3.1.1. T Module

For this module, 18 attributes were identified as relevant for the assessment of ASD, namely attributes A2, A7, A3, A5, A8, A12, A13, A16, A4, A6, A11, A15, A1, A9, A10, A14, A17 and A18 in Table 1. For the T module, there are no empty rows. This means that there are no useless items with respect to the selected attributes. Moreover, even if no empty columns were found, then all the selected attributes are investigated by at least one item of this module. In this module, equal columns are not present, but there are equivalent rows. In fact, two items (free play and free play-ball) of the T module evaluate the same set of attributes. These items are repetitive and thus, in terms of efficiency, one of them could be deleted to reduce the duration of the test and stress on the child (Table 2). The analysis showed that each item evaluates at least one selected attribute. This information shows that the T module assesses the main abilities to make ASD diagnosis in the age range of 12–30 months. However, two items (free play and free play-ball) perfectly evaluate the same attributes. Thus, these items should be considered repetitive.

Table 2
The Toddler module containing the 14 items and the 18 attributes. The items named free play and free play-Ball have been combined into a single item because they are equivalent.


3.1.2. Module 1

For this module, the following 17 attributes were selected: A13, A2, A5, A7, A9, A12, A3, A11, A16, A4, A6, A8, A15, A1, A10, A14 and A17 (Table 1). For module 1, there are no empty rows, so each item investigated at least one selected attribute. In this module, equal columns are not present, but there are equivalent rows. This result means that two items (symbolic and functional imitation and birthday party) of module 1 evaluate the exact same set of attributes (e.g., A13, A5 and A16).

For this module we found two equivalent rows for module 1 (symbolic and functional imitation and birthday party). These items evaluate the same attributes and can be considered repetitive.

Table 3
The module 1 containing the 9 items and the 16 attributes. The items named functional and symbolic imitation and birthday party have been combined into a single item because they are equivalent. The A9 has been not inserted in the table because not examined by any items of this module.


3.1.3. Module 2

For this module, the following 24 attributes were selected: A2, A3, A6, A7, A13, A8, A4, A5, A11, A12, A16, A22, A15 A1, A10, A19, A20, A21, A23, A24, A9, A14, A17 and A18 (see Table 1). Nevertheless, only 20 out of 24 attributes were investigated because, in this module, the A9, A14, A17 and A18 are not examined (see Table 4). In this module, there are no equivalent rows, but there are equal columns. This result means that two attributes (A19 and A20) of module 2 are identical and are assessed in only one item (construction task).

Table 4
The module 2 containing the 14 items and the 20 attributes. The A9, A14, A17 and A18 have been not inserted in the table because not examined by any items of this module.


3.1.4. Module 3

For this module, the following 30 attributes were selected: A22, A3, A8, A27, A29, A6, A21, A23, A30, A4, A13, A14, A25 A26, A18, A7, A19, A20, A28, A16, A24, A1, A2, A5, A9, A10, A11, A12, A15 and A17 (Table 1). However, only 21 out of 30 attributes were investigated because, in this module, A1, A2, A5, A9, A10, A11, A12, A15 and A17 are not examined. Particularly, A9 (use of gaze to understand the action of the examiner) and A15 (unusual body movements and unusual sensory interests) are important attributes in ASD diagnosis that were evaluated in this module. Equivalent rows and equal columns are present in this module (Table 5). Regarding equivalent rows, this result means that two items (social relationship difficulties and friendships, relationships and marriage) of module 3 perfectly evaluate the same attributes (A29 and A30). As to the equal columns, this result means that three attributes (A14, A25 and A26) are indicated in the same item (cartoons). In addition, three attributes (A7, A19 and A20) are indicated in the same item (construction task). Finally, both A16 and A24 are indicated in the same item (joint interactive play).

Table 5
The module 3 containing the 13 items and the 21 attributes. The items named Social difficulties and annoyance and friends, relationship and marriage have been combined a single item because they are equivalent. The A1, A2, A5, A9, A10, A11, A12, A15 and A17 have been not inserted in the table because not examined by any items of this module.


3.1.5. Module 4

For this module, the following 30 attributes were selected: A22, A29, A30, A27, A3, A8, A6, A23, A13, A18, A7, A19, A20, A21, A14, A25, A26, A28, A1, A2, A4, A5, A9, A10, A11, A12, A15, A16, A17 and A24 (see Table 1). Nevertheless, the following 12 attributes were not considered in this module: A1, A2, A4, A5, A9, A10, A11, A12, A15, A16, A17 and A24. Some of the attributes not considered in module 4 that are important for making an ASD diagnosis in this age range include the following: A9 (use of the gaze to understand the action of examiner), A15 (unusual body movements and unusual sensory interest) and A24 (gestures associated with verbal language). Equivalent rows and equal columns are present in this module (Table 6). Regarding equivalent rows, this result means that two items (current work or school; friends, relationship and marriage) in module 4 perfectly evaluate the same attributes (A29 and A30). In addition, another two items (daily living; plans and hopes) examined only one attribute (A30). For the equal columns, three attributes (A7, A19 and A20) are examined by the same item (construction task). In addition, three attributes (A14, A25 and A26) are examined by the same item (cartoons).

Table 6
The module 4 containing the 13 items and the 18 attributes. The items named daily living and plans and hopes have been combined a single item because they are equivalent. The A1, A2, A4, A5, A9, A10, A11, A12, A15, A16, A17 and A24 have been not inserted in the table because not examined by the items of this module.


3.2. Clinical Structures

The clinical structure is the formal and graphical representation of the relations among the items of the context and, furthermore, it can display the relations between sets of items and sets of attributes. It is usually represented by a graph in which every node is a collection of items (i.e., the clinical state) and the corresponding set of attributes (i.e., the set of criteria satisfied by a patient having that clinical state). The structure is built according to both the relations among items and the attributes depicted in the clinical context and the prerequisite relations depicted by the same context. Some of the crucial configurations regarding the rows and columns of the context have precise counterparts in the structure. For instance, an empty row (i.e., an item that does not investigate any of the selected attributes) will result in the absence of that item in the structure. The same happens to empty columns. When two rows are equivalent, the corresponding items will be included systematically in the same states. Finally, the prerequisite relation occurring whenever the attributes investigated by an item i are a subset of those investigated by j in terms of states in the structure means that there will not be any state including j that does not also include i.

3.2.1. T Module

There are 14 items in the T module. The obtained structure has 1156 states. Remembering that if no relations among items were present, the total theoretical response patterns would be 214 = 16384 (i.e. the cardinality of the power set of the domain). The obtained structure contains only 7% of the potential patterns, indicating how an adaptive version of the instrument could be much more efficient in evaluating patients. The application of this analysis demonstrates the existence of prerequisite relations between several items of the T module. In particular, item 1 is a prerequisite of items 7 and 8; item 2 is a prerequisite of item 13; items 3 and 6 are prerequisites of item 7; and item 12 is a prerequisite of item 13 (Table 2).

This implies that if the child does not pass item 1, and item 1 is a prerequisite of other items, it is not necessary to administer those items.

3.2.2. Module 1

There are 9 items in module 1. The obtained structure has 224 states, 43% of the potential patterns. The application of this analysis demonstrates the existence of a prerequisite relation between two items of module 1. In particular, item 1 is a prerequisite of item 5 (Table 3). This implies that if the child does not pass item 1, it is not necessary to administer item 5.

3.2.3. Module 2

There are 14 items in module 2. The obtained structure has 1384 states, 8% of the potential patterns. The application of this analysis demonstrates the existence of prerequisite relations between several items of the T module. In particular, item 1 is a prerequisite of item 2; in addition, item 4 is a prerequisite of items 9 and 10; and item 9 is a prerequisite of item 10 (Table 4). This analysis demonstrates that it is not necessary to administer item 2 (free play) if the child does not pass item 1 (bubbles-play), because item 1 is a prerequisite of item 2. Besides this, item 10 no longer needs to be administered if the child does not pass items 4 and 9.

3.2.4. Module 3

There are 13 items in module 3. The obtained structure has 1136 states, which is 13% of the potential patterns. The application of this analysis demonstrates the existence of prerequisite relations between several items of module 3. In particular, item 1 is a prerequisite of items 2, 3 and 4. In addition, items 4, 5 and 6 are prerequisites of item 3; and item 8 is a prerequisite of items 3 and 9 (Table 5). This suggests that the order of appearance of items in the module should be reviewed.

3.2.5. Module 4

There are 13 items in module 4. The obtained structure has 592 states, 7% of the potential patterns, indicating how an adaptive process could be much more efficient in evaluating patients. The application of this analysis demonstrates the existence of prerequisite relations between several items of module 4. In particular, items 1, 2, 3 and 4 are prerequisites of item 5; item 3 is a prerequisite of item 12; item 7 is a prerequisite of items 8 and 10; finally, item 10 is a prerequisite of item 8 (Table 6). This result means that the clinician can select the items based on the item the child passed previously. In this way, it is possible to reduce the time it takes to administer the test.

3.3. Clinical Structures Validation: Application to T Module and Module 1

The clinical structure of both the T module and module 1 were tested using the BLIM. The fit of the model parameters was estimated by means of the EM procedure [9Spoto A, Bottesi G, Sanavio E, Vidotto G. Theoretical foundations and clinical implications of formal psychological assessment. Psychother Psychosom 2013; 82(3): 197-9.[http://dx.doi.org/10.1159/000345317] [PMID: 23549091] , 15Dempster AP, Laird NM, Rubin DB. Maximum likelihood estimation from incomplete data via the EM algorithm (with discussion). J R Stat Soc Series B Stat Methodol 1977; 39(1): 1-38.]. To test the reliability of the observed Chi-square, a parametric bootstrap with 5000 replications was conducted for each of the two modules [13Spoto A, Stefanutti L, Vidotto G. Knowledge space theory, formal concept analysis, and computerized psychological assessment. Behav Res Methods 2010; 42(1): 342-50.[http://dx.doi.org/10.3758/BRM.42.1.342] [PMID: 20160314] ]. Concerning the T module, the results showed that the model adequately fit the data (x2(260,952) = 757.04; p > .05; number of states = 1,156). The reliability of this result was confirmed by the parametric bootstrap (bootstrap p > .05). Moreover, the estimates of the error rates for the T module, reported in Table 7, satisfy the condition of acceptability (i.e., βq + ηq < 1) for every item.

Table 7
Estimated parameters η and β for each item of T Module.


On the other hand, for module 1, the model fit the data quite well (x2(65, 280) = 17, 585; p > .05; number of states = 224) even though the reliability of the observed results is not supported by the bootstrap (bootstrap p > .05). Nonetheless, as can be seen in Table 8, even for this module the error rate estimates are acceptable for every item.

In general, it can be concluded that the two models accurately explain the observed data. The estimates can then be implemented into an algorithm constituting an adaptive tool for the assessment of the diagnostic issues for ASD. Such a tool is derived from the ADOS-2 by removing its redundancies and focusing on the specific issues useful for diagnosis.

Table 8
Estimated parameters η and β for each item of Module 1.


4. GENERAL DISCUSSION

The aim of the present study was to analyse the single modules of ADOS-2 using the FPA methodology. Diagnosis is a fundamental process that determines which disorder or condition explains the symptoms of a person. Clinical assessment can be described as an intelligent procedure the clinician carries out with the aim of collecting information about a patient to formulate the diagnosis and propose therapeutic treatment. The investigation proceeds through a sequence of hypothesis formulations and validations [7Donadello I, Spoto A, Sambo F, Badaloni S, Granziol U, Vidotto G. ATS-PD: An Adaptive Testing System for Psychological Disorders. Educ Psychol Meas 2017; 77(5): 792-815.[http://dx.doi.org/10.1177/0013164416652188] [PMID: 29795931] ]. This process is more difficult in neurodevelopmental disorders where it is not possible to ask patients to describe their symptoms (such as in the T module or module 1) or to measure the general capacity of the patient’s social and relational abilities (for example in modules 2, 3 and 4). In addition, the information provided by caregivers through diagnostic interviews is not always related to the hypothesis of a clinician. For this reason, it is important to maximize and optimize the collected information, reduce the errors, focus on the critical areas of an individual (e.g., deficit in social interaction), and reduce the number of items and time of evaluation, while trying to preserve its accuracy. We hypothesized that the reduction of the number of items considered redundant by the FPA would correlate with the improvement of the quality and quantity of information collected [7Donadello I, Spoto A, Sambo F, Badaloni S, Granziol U, Vidotto G. ATS-PD: An Adaptive Testing System for Psychological Disorders. Educ Psychol Meas 2017; 77(5): 792-815.[http://dx.doi.org/10.1177/0013164416652188] [PMID: 29795931] ]. Notice that such reduction has no effect on the amount of information collected and, in general, it does not necessarily mean the removal of the item from the tool. In fact, the adaptive assessment procedure could store the redundant items into a buffer and use them if needed. For example, the answers may not be coherent and, therefore, it would be necessary to administer the same item twice. In this case, having the redundant items available in the buffer would allow the clinician to investigate the same attributes twice, without repeating the same item twice. This is a strong advantage of such an approach. This methodology could be applied to all clinical tests used in the autism field (such as the Autism Diagnostic Interview – Revised, Vineland), to reduce the weight of items without diagnostic or repetitive significance based on evidence from an FPA analysis and the validation of its clinical structure. Our goal is to improve the diagnostic process for ASD by making it adaptive. This adaptive system can perform logically correct inferences based on all the information collected during the testing process [7Donadello I, Spoto A, Sambo F, Badaloni S, Granziol U, Vidotto G. ATS-PD: An Adaptive Testing System for Psychological Disorders. Educ Psychol Meas 2017; 77(5): 792-815.[http://dx.doi.org/10.1177/0013164416652188] [PMID: 29795931] ]. The system is adaptive in the sense that the question posed by the system at a given moment depends on the previously collected answers of the patient or behaviours observed. The adaptivity of this system lets it dynamically choose the best sequence of items or stimulus situations to be posed to maximize the informational content of each answer [7Donadello I, Spoto A, Sambo F, Badaloni S, Granziol U, Vidotto G. ATS-PD: An Adaptive Testing System for Psychological Disorders. Educ Psychol Meas 2017; 77(5): 792-815.[http://dx.doi.org/10.1177/0013164416652188] [PMID: 29795931] , 16Granziol U, Spoto A, Vidotto G. The assessment of nonverbal behavior in schizophrenia through the Formal Psychological Assessment. Int J Methods Psychiatr Res 2017.[http://dx.doi.org/10.1002/mpr.1595] [PMID: 29098743] ]. In this way, the system can often avoid posing the entire set of questions by inferring the answers to logically connected questions, thus saving time. A previous study found that the application of an adaptive version of a questionnaire allowed people to reach the end of the assessment without using 25-50% of the items [7Donadello I, Spoto A, Sambo F, Badaloni S, Granziol U, Vidotto G. ATS-PD: An Adaptive Testing System for Psychological Disorders. Educ Psychol Meas 2017; 77(5): 792-815.[http://dx.doi.org/10.1177/0013164416652188] [PMID: 29795931] ]. Once an adaptive algorithm is defined that can take into account the ADOS-2 assessment modality, a similar time savings is expected.

CONCLUSION

The aim of this study was to use a practical application of the FPA on the ADOS-2 and show its potential for psychological assessment for individuals with ASD. An important limit of the ADOS-2, like all tests based on observation, is that diagnostic characteristics seem to change among clinicians and the results for the same individual can be different if the evaluation is subsequently repeated. Unfortunately, the FPA method cannot enhance the reliability of the ADOS-2, but the main aim is to create an adaptive testing system for ASD diagnosis. We suppose that the reduction of the number of items considered repetitive by the FPA could to be correlated with an improvement of the quality and quantity of information collected. This issue represents a future goal of our research on autism. Nonetheless, some work is being carried out to provide new methodological tools for an accurate assessment of the inter-rater agreement for tasks like those involved in the administration of the ADOS-2 [17Spoto A, Nucci M, Pastore M, Altoé G. Bayesian modeling of inter-rater agreement: The role of rater’s theoretical beliefs and threshold. Eur Math Psy Group Meeting 2016.]. In addition, another limitation of this study is that the items of the ADOS-2 are then assigned to the attributes by six experts (four psychologists and two child psychiatrists) in the field of ASD and in the administration of the ADOS-2, thus these assignments are subjective.

Nevertheless, we believe our study is important and addresses a critical issue in the literature on autism. This study represents the first step toward creating an adaptive testing system for ASD based on an existing clinical test that would improve the diagnosis of ASD and assist clinicians in their diagnosis formulation. For example, if a clinician assigns the score 2 to an item that evaluates “used gestures to indicate requests”, the system, using the updating rule, increases the likelihood of the states containing the same attributes; on the contrary, if the clinician assigns the score 0 to this item, the system decreases the other attributes [7Donadello I, Spoto A, Sambo F, Badaloni S, Granziol U, Vidotto G. ATS-PD: An Adaptive Testing System for Psychological Disorders. Educ Psychol Meas 2017; 77(5): 792-815.[http://dx.doi.org/10.1177/0013164416652188] [PMID: 29795931] , 16Granziol U, Spoto A, Vidotto G. The assessment of nonverbal behavior in schizophrenia through the Formal Psychological Assessment. Int J Methods Psychiatr Res 2017.[http://dx.doi.org/10.1002/mpr.1595] [PMID: 29098743] ].

In line with Donadello and colleagues [7Donadello I, Spoto A, Sambo F, Badaloni S, Granziol U, Vidotto G. ATS-PD: An Adaptive Testing System for Psychological Disorders. Educ Psychol Meas 2017; 77(5): 792-815.[http://dx.doi.org/10.1177/0013164416652188] [PMID: 29795931] ], our ambitious objective is to develop a software product that can assist clinicians with the assessment of ASD.

FUNDING

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ETHICS APPROVAL AND CONSENT TO PARTICIPATE

Not applicable.

HUMAN AND ANIMAL RIGHTS

No animals/humans were used for studies that are the basis of this review.

CONSENT FOR PUBLICATION

Not applicable.

DECLARATION OF CONFLICTING INTERESTS

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

ACKNOWLEDGEMENTS

Declared none.

REFERENCES

[1] Lai MC, Lombardo MV, Baron-Cohen S. Autism. Lancet 2014; 383(9920): 896-910.[http://dx.doi.org/10.1016/S0140-6736(13)61539-1] [PMID: 24074734]
[2] De Berardis D, Fornaro M, Orsolini L, et al. Effect of agomelatine treatment on C-reactive protein levels in patients with major depressive disorder: An exploratory study in “real-world,” everyday clinical practice. CNS Spectr 2017; 22(4): 342-7.[http://dx.doi.org/10.1017/S1092852916000572] [PMID: 27702411]
[3] Serra F, Spoto A, Ghisi M, Vidotto G. Formal psychological assessment in evaluating depression: A new methodology to build exhaustive and irredundant adaptive questionnaires. PLoS One 2015; 10(4): e0122131.[http://dx.doi.org/10.1371/journal.pone.0122131] [PMID: 25875359]
[4] Veltro F, Mazza M, Vendittelli N, Alberti M, Casacchia M, Roncone R. A comparison of the effectiveness of problem solving training and of cognitive-emotional rehabilitation on neurocognition, social cognition and social functioning in people with schizophrenia. Clin Pract Epidemiol Ment Health 2011; 7: 123-32.[http://dx.doi.org/10.2174/1745017901107010123] [PMID: 21792373]
[5] American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders: DSM-V, 5 the dn 2013.
[6] Lord C, Rutter M, DiLavore PC, Risi S, Gotham K, Bishop SL. Autism diagnostic observation schedule. J Psyedu Assess 2nd edition (ADOS-2) manual (Part I): Modules 1–4. 2012; 88-92.2nd edition (ADOS-2) manual (Part I): Modules 1–4.
[7] Donadello I, Spoto A, Sambo F, Badaloni S, Granziol U, Vidotto G. ATS-PD: An Adaptive Testing System for Psychological Disorders. Educ Psychol Meas 2017; 77(5): 792-815.[http://dx.doi.org/10.1177/0013164416652188] [PMID: 29795931]
[8] Bottesi G, Spoto A, Freeston MH, Sanavio E, Vidotto G. Beyond the score: Clinical evaluation through formal psychological assessment. J Pers Assess 2015; 97(3): 252-60.[http://dx.doi.org/10.1080/00223891.2014.958846] [PMID: 25257993]
[9] Spoto A, Bottesi G, Sanavio E, Vidotto G. Theoretical foundations and clinical implications of formal psychological assessment. Psychother Psychosom 2013; 82(3): 197-9.[http://dx.doi.org/10.1159/000345317] [PMID: 23549091]
[10] Doignon JP, Falmagne JC. Knowledge Spaces 1999.
[11] Falmagne JC, Doignon JP. Learning Spaces 2011.[http://dx.doi.org/10.1007/978-3-642-01039-2]
[12] Ganter B, Wille R. Formal concept analysis: Mathematical Foundations 1999.[http://dx.doi.org/10.1007/978-3-642-59830-2]
[13] Spoto A, Stefanutti L, Vidotto G. Knowledge space theory, formal concept analysis, and computerized psychological assessment. Behav Res Methods 2010; 42(1): 342-50.[http://dx.doi.org/10.3758/BRM.42.1.342] [PMID: 20160314]
[14] Falmagne JC, Doignon JP. A class of stochastic procedures for the assessment of knowledge. Br J Math Stat Psychol 1988; 41: 1-23.[http://dx.doi.org/10.1111/j.2044-8317.1988.tb00884.x]
[15] Dempster AP, Laird NM, Rubin DB. Maximum likelihood estimation from incomplete data via the EM algorithm (with discussion). J R Stat Soc Series B Stat Methodol 1977; 39(1): 1-38.
[16] Granziol U, Spoto A, Vidotto G. The assessment of nonverbal behavior in schizophrenia through the Formal Psychological Assessment. Int J Methods Psychiatr Res 2017.[http://dx.doi.org/10.1002/mpr.1595] [PMID: 29098743]
[17] Spoto A, Nucci M, Pastore M, Altoé G. Bayesian modeling of inter-rater agreement: The role of rater’s theoretical beliefs and threshold. Eur Math Psy Group Meeting 2016.

Endorsements



"Open access will revolutionize 21st century knowledge work and accelerate the diffusion of ideas and evidence that support just in time learning and the evolution of thinking in a number of disciplines."


Daniel Pesut
(Indiana University School of Nursing, USA)

"It is important that students and researchers from all over the world can have easy access to relevant, high-standard and timely scientific information. This is exactly what Open Access Journals provide and this is the reason why I support this endeavor."


Jacques Descotes
(Centre Antipoison-Centre de Pharmacovigilance, France)

"Publishing research articles is the key for future scientific progress. Open Access publishing is therefore of utmost importance for wider dissemination of information, and will help serving the best interest of the scientific community."


Patrice Talaga
(UCB S.A., Belgium)

"Open access journals are a novel concept in the medical literature. They offer accessible information to a wide variety of individuals, including physicians, medical students, clinical investigators, and the general public. They are an outstanding source of medical and scientific information."


Jeffrey M. Weinberg
(St. Luke's-Roosevelt Hospital Center, USA)

"Open access journals are extremely useful for graduate students, investigators and all other interested persons to read important scientific articles and subscribe scientific journals. Indeed, the research articles span a wide range of area and of high quality. This is specially a must for researchers belonging to institutions with limited library facility and funding to subscribe scientific journals."


Debomoy K. Lahiri
(Indiana University School of Medicine, USA)

"Open access journals represent a major break-through in publishing. They provide easy access to the latest research on a wide variety of issues. Relevant and timely articles are made available in a fraction of the time taken by more conventional publishers. Articles are of uniformly high quality and written by the world's leading authorities."


Robert Looney
(Naval Postgraduate School, USA)

"Open access journals have transformed the way scientific data is published and disseminated: particularly, whilst ensuring a high quality standard and transparency in the editorial process, they have increased the access to the scientific literature by those researchers that have limited library support or that are working on small budgets."


Richard Reithinger
(Westat, USA)

"Not only do open access journals greatly improve the access to high quality information for scientists in the developing world, it also provides extra exposure for our papers."


J. Ferwerda
(University of Oxford, UK)

"Open Access 'Chemistry' Journals allow the dissemination of knowledge at your finger tips without paying for the scientific content."


Sean L. Kitson
(Almac Sciences, Northern Ireland)

"In principle, all scientific journals should have open access, as should be science itself. Open access journals are very helpful for students, researchers and the general public including people from institutions which do not have library or cannot afford to subscribe scientific journals. The articles are high standard and cover a wide area."


Hubert Wolterbeek
(Delft University of Technology, The Netherlands)

"The widest possible diffusion of information is critical for the advancement of science. In this perspective, open access journals are instrumental in fostering researches and achievements."


Alessandro Laviano
(Sapienza - University of Rome, Italy)

"Open access journals are very useful for all scientists as they can have quick information in the different fields of science."


Philippe Hernigou
(Paris University, France)

"There are many scientists who can not afford the rather expensive subscriptions to scientific journals. Open access journals offer a good alternative for free access to good quality scientific information."


Fidel Toldrá
(Instituto de Agroquimica y Tecnologia de Alimentos, Spain)

"Open access journals have become a fundamental tool for students, researchers, patients and the general public. Many people from institutions which do not have library or cannot afford to subscribe scientific journals benefit of them on a daily basis. The articles are among the best and cover most scientific areas."


M. Bendandi
(University Clinic of Navarre, Spain)

"These journals provide researchers with a platform for rapid, open access scientific communication. The articles are of high quality and broad scope."


Peter Chiba
(University of Vienna, Austria)

"Open access journals are probably one of the most important contributions to promote and diffuse science worldwide."


Jaime Sampaio
(University of Trás-os-Montes e Alto Douro, Portugal)

"Open access journals make up a new and rather revolutionary way to scientific publication. This option opens several quite interesting possibilities to disseminate openly and freely new knowledge and even to facilitate interpersonal communication among scientists."


Eduardo A. Castro
(INIFTA, Argentina)

"Open access journals are freely available online throughout the world, for you to read, download, copy, distribute, and use. The articles published in the open access journals are high quality and cover a wide range of fields."


Kenji Hashimoto
(Chiba University, Japan)

"Open Access journals offer an innovative and efficient way of publication for academics and professionals in a wide range of disciplines. The papers published are of high quality after rigorous peer review and they are Indexed in: major international databases. I read Open Access journals to keep abreast of the recent development in my field of study."


Daniel Shek
(Chinese University of Hong Kong, Hong Kong)

"It is a modern trend for publishers to establish open access journals. Researchers, faculty members, and students will be greatly benefited by the new journals of Bentham Science Publishers Ltd. in this category."


Jih Ru Hwu
(National Central University, Taiwan)


Browse Contents



Webmaster Contact: info@benthamopen.net
Copyright © 2019 Bentham Open