The Open Psychology Journal




ISSN: 1874-3501 ― Volume 13, 2020
RESEARCH ARTICLE

Development and Validation of Instrument to Measure Thinking Patterns: A Structural Equation Modeling Analysis



Ali Maksum1, *, Fifukha D. Khory2
1 School of Postgraduate, State University of Surabaya, Surabaya, Indonesia
2 Department of Physical Education, State University of Surabaya, Surabaya, Indonesia

Abstract

Background:

As an instrument that measures thinking processes, the cognitive reflection test still has a number of problems, especially in terms of its validity and reliability.

Aims:

This research aimed to develop instruments to identify patterns of thinking that meet psychometric requirements.

Methods and Results:

Participants in the research were 727 students from the State University of Surabaya, including 322 (44%) men and 405 (56%) women with a mean age of 19.17 years. The first examination using exploratory factor analysis showed that the scale of thinking patterns, which we later called Intuitive-Reflective Scale (IRS), had a conceptual relations structure consisting of 5 factors with a loading factor of .40 - .80. The five factors explained 52.57% of the total variance and had Cronbach’s Alpha reliability of .71. The second examination using confirmatory factor analysis based on structural equation modeling proved that the IRS had factors structure that was consistent with the results of the first examination and was a significant predictor of academic performance.

Conclusion:

Hypothesized factor structure fits with empirical data based on the comparative fit index of .96 and root mean square error of approximation of .07.

Keywords: Intuitive-reflective scale, Scale development, Structural equation modelling, Thinking patterns, Psychometric requirements, Cognitive reflection, Reliability.


Article Information


Identifiers and Pagination:

Year: 2020
Volume: 13
First Page: 58
Last Page: 66
Publisher Id: TOPSYJ-13-58
DOI: 10.2174/1874350102013010058

Article History:

Received Date: 02/02/2020
Revision Received Date: 08/04/2020
Acceptance Date: 13/04/2020
Electronic publication date: 23/05/2020
Collection year: 2020

© 2020 Maksum and Khory.

open-access license: This is an open access article distributed under the terms of the Creative Commons Attribution 4.0 International Public License (CC-BY 4.0), a copy of which is available at: https://creativecommons.org/licenses/by/4.0/legalcode. This license permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.


* Address correspondence to this author at the School of Postgraduate, State University of Surabaya, Indonesia; Tel: 081332539555; E-mail: alimaksum@unesa.ac.id





1. INTRODUCTION

As an instrument for measuring thought processes, the Cognitive Reflection Test (CRT) has been widely used in a number of studies [1Haigh M. Has the standard cognitive reflection test become a victim of its own success? Adv Cogn Psychol 2016; 12(3): 145-9.
[http://dx.doi.org/10.5709/acp-0193-5] [PMID: 28115997]
-3Stupple EJN, Pitchford M, Ball LJ, Hunt TE, Steel R. Slower is not always better: Response-time evidence clarifies the limited role of miserly information processing in the Cognitive Reflection Test. PLoS One 2017; 12(11)e0186404
[http://dx.doi.org/10.1371/journal.pone.0186404] [PMID: 29099840]
]. Unfortunately, as an instrument, CRT exhibited a number of problems. First, the CRT consisting of 3 items more closely resembles intelligence tests that measure thinking ability, and not individual thought patterns. Second, the CRT does not adequately reflect the attributes of a comprehensive thought process because all CRT items are numerically based and measure the same aspects. Good instruments reflect the characteristics of the attributes tobe measured [4Jacoby WG. Data theory and dimensional analysis 1991.
[http://dx.doi.org/10.4135/9781412983860]
-6Nunnally IH. JC & Berstein, Psychometric theory 3rd ed. 1994.]. Third, the data generated from the CRT

is in the form of right or wrong answers, which are nominal or ordinal. Such data quality has limitations to be used in statistical analysis, because it is not applicable to an additional calculation operation or ratio scale data [7Hair WC. JH, Anderson, RE, Tatham, RL, & Black, Multivariate data analysis, Fifth 1998., 8Kerlinger FN. Foundations of behavioral research 3rd ed. 1986.], as a result, inferential statistical analysis that is more advanced cannot be applied. Fourth, the CRT results proved to be influenced by respondents' familiarity with the questions [9Stieger S, Reips UD. A limitation of the Cognitive Reflection Test: familiarity. PeerJ 2016; 4e2395
[http://dx.doi.org/10.7717/peerj.2395] [PMID: 27651989]
-11Cheng J, Janssen C. The relationship between an alternative form of cognitive reflection test and intertemporal choice Stud Psychol (Bratisl) 2019.
[http://dx.doi.org/10.21909/sp.2019.02.774]
]. That is, if the respondent has worked on the problem and already knows the logic of the problem, the CRT is no longer valid and effective for measuring the thought process. Fifth, the results of validity tests against CRTs are less encouraging, only valid for measuring reflective thinking and not intuitive [12Pennycook G, Cheyne JA, Koehler DJ, Fugelsang JA. Is the cognitive reflection test a measure of both reflection and intuition? Behav Res Methods 2016; 48(1): 341-8.
[http://dx.doi.org/10.3758/s13428-015-0576-1] [PMID: 25740762]
] and their validity is low when repeated testing is carried out [1Haigh M. Has the standard cognitive reflection test become a victim of its own success? Adv Cogn Psychol 2016; 12(3): 145-9.
[http://dx.doi.org/10.5709/acp-0193-5] [PMID: 28115997]
].

Many attempts have been made to overcome these weaknesses, for example, by Cheng and Janssen [11Cheng J, Janssen C. The relationship between an alternative form of cognitive reflection test and intertemporal choice Stud Psychol (Bratisl) 2019.
[http://dx.doi.org/10.21909/sp.2019.02.774]
] and Thomson [10Thomson KS. Investigating an alternate form of the cognitive reflection test. Judgement Decis Mak 2016; 11(1): 99-113.], who tried to compile a new item that they call CRT-2. However, it seems that these efforts cannot significantly eliminate the limitations of the methods. They will face problems that are more or less the same as seen in the first CRT. Therefore, the development of new instruments to measure thinking patterns becomes an urgent requirement. The instrument was developed based on dual-processing theory, namely system 1 and system 2 [13Kahneman D. Thinking fast and slow 2011., 14Evans JSBT. Dual-processing accounts of reasoning, judgment, and social cognition. Annu Rev Psychol 2008; 59: 255-78.
[http://dx.doi.org/10.1146/annurev.psych.59.103006.093629] [PMID: 18154502]
], which were then linked and explored with neuroscience [15Rosenzweig P. Left brain, right stuff 2015.-18Doidge N. Brain that changes itself 2007.]. According to the dual-processing theory, it is assumed that system 1 is related to the limbic system, while system 2 is related to the neocortex.

Neuroscientists also discovered the fact that the behavior displayed by humans has a neuro basis in brain structure [16Peters S. The chimp paradox: The mind management 2012., 19Kurzweil R. How to create a mind: The secret of human thought revealed 2013.]. Peters explained that the frontal area of the brain functions to make decisions based on rational reasons, including weighing, analyzing, and considering an action in depth from the good-bad and profit-loss aspects. People who think logically and deeply use a lot of frontal areas in their thought processes. In contrast to the frontal area, the limbic area functions to respond instinctively to emotional stimuli that tend to be rushed, impulsive, and not infrequently destructive. People, who respond to problems emotionally without thinking long, prejudice without data, and draw conclusions speculatively, use a lot of limbic areas in the thought process.

In psychology, the problem is actually not new, but has been studied since the 1970-1980s. Later this theme became phenomenal after Kahneman conducted an in-depth study of cognitive biases in decision making, especially relative to the economic behaviour of individuals [13Kahneman D. Thinking fast and slow 2011.]. Kahneman argues that to make a decision, humans generally use experience bias, and do not use their ratios optimally, and tend to discourage short-term interests. Consequently, many actions, as a result of the decision making process, are counterproductive and less effective in achieving goals. Kahneman uses the terms thinking fast and slow, which are the concepts derived from system 1 and system 2 [14Evans JSBT. Dual-processing accounts of reasoning, judgment, and social cognition. Annu Rev Psychol 2008; 59: 255-78.
[http://dx.doi.org/10.1146/annurev.psych.59.103006.093629] [PMID: 18154502]
-20Epstein S, Pacini R, Denes-Raj V, Heier H. Individual differences in intuitive-experiential and analytical-rational thinking styles. J Pers Soc Psychol 1996; 71(2): 390-405.
[http://dx.doi.org/10.1037/0022-3514.71.2.390] [PMID: 8765488]
]. Thinking fast is characterized by the decision-making process that is emotional, intuitive, automatic, lacks in-depth consideration, and is difficult to control over instinctive desires. Meanwhile, thinking slow is characterized by rational decision making, requires in-depth consideration, is relatively flexible, and adaptive to rules.

If an individual is accustomed to using intuitive thinking patterns in making decisions, then in the long run, his or her mind cannot function optimally and critical thinking habits will not be developed. Finally, individuals disregard the reference to understanding problems clearly, including solving problems in life. This situation is not only unfavorable but also counterproductive to progress. In the reality of everyday life, indications of the use of intuitive thinking patterns still occur in many societies. For example, many people are exposed to fraud under the guise of investment plans , which obviously makes no sense. The abundance of information that is often called “big data” is apparently not directly proportional to the intelligence of individuals in acting and making decisions [21Nichols T. The death of expertise: The campaign against established knowledge and why it matters 2017., 22Schwab K. The fourth industrial revolution. Genewa: World Economic Forum. 2017.2017.]. What emerges is precisely the prevailing irrationality marked by the loss of rational reasoning and the lack of awareness of the importance of the verification process before coming to conclusions. People become rushed, spontaneous, instant, and are impulsive without thinking about the truth and dignity that underly it.

For education, the thinking patterns are closely related to the process and learning outcomes [23Wulandari NH, Widayati KA, Suryobroto B. Cognitive style and creative quality in fluence on academic achievement of university students in indonesia. Hayati J Biosci 2016; 23(3): 121-4.
[http://dx.doi.org/10.1016/j.hjb.2016.09.001]
-25Usán Supervía P, Salavera Bordás C. Academic performance, emotional intelligence and academic engagement in adolescents. Electron J Res Educ Psychol 2019.]. Learning is not just meant as an effort to get an academic score, but rather as an effort to build a civilization that educates life [26Maksum A. The dilemma of character education in disruptive social power: An analysis from a neuropsychological perspective. Integrasi PPK dan Gerakan Literasi Nasional dalam Pembelajaran Berbasis HOTS 2019., 27Maksum A. Curricullum and learning in higher education: Towards an empowering education. Seminar nasional hasil penelitian pendidikan dan pembelajaran 2015.]. Information as a source of learning can come from classroom activities such as teaching materials, explanations from lecturers, and communication between students through a discussion process, and can also spread via the public cyberspace such as the internet and social media. The research on thinking patterns becomes urgent in the current post-truth era, when people tend to accept truth based on their emotions and beliefs rather than based on the proven facts [21Nichols T. The death of expertise: The campaign against established knowledge and why it matters 2017., 22Schwab K. The fourth industrial revolution. Genewa: World Economic Forum. 2017.2017.]. The truth seems not to be determined by data and facts, but by individual perceptions and beliefs. If the facts are in line with their beliefs, they are convinced of the truth without their verification [28McIntyre LC. Post truth 2018.
[http://dx.doi.org/10.7551/mitpress/11483.001.0001]
]. This condition is amplified by VUCA (Volatility, Uncertainty, Complexity, Ambiguity).

Thus, the thinking patterns become a crucial problem in building civilization and are a fundamental problem in education. The question is how to identify thinking patterns that meet psychometric eligibility [6Nunnally IH. JC & Berstein, Psychometric theory 3rd ed. 1994., 29Cohen ME. RJ & Swerdlik, Psychological testing and assessment: An introduction to test and measurement 6th ed. 2005.]. So far, two common instruments are used by researchers, namely the Cognitive Reflection Test – CRT [9Stieger S, Reips UD. A limitation of the Cognitive Reflection Test: familiarity. PeerJ 2016; 4e2395
[http://dx.doi.org/10.7717/peerj.2395] [PMID: 27651989]
, 30Frederick S. Cognitive reflection and decision making. J Econ Perspect 2005.
[http://dx.doi.org/10.1257/089533005775196732]
] and self-reports such as Thinking Styles Inventory – TSI [31Sternberg RJ. Thinking and Problem Solving 2013., 32Grigorenko EL, Sternberg RJ. Thinking Styles In: Saklofske M, DH, Zeidner, Eds. International Handbook of Personality and Intelligence 1995; 205-29.
[http://dx.doi.org/10.1007/978-1-4757-5571-8_11]
]. The first instrument is more like an intelligence test that measures an individual's cognitive abilities, rather than revealing thinking patterns preferred by the individual or habits of thinking [1Haigh M. Has the standard cognitive reflection test become a victim of its own success? Adv Cogn Psychol 2016; 12(3): 145-9.
[http://dx.doi.org/10.5709/acp-0193-5] [PMID: 28115997]
]. While the second instrument may identify a person's thinking preferences, but after being tested on a broader scale, many of the items are underrated. Out of a total of 104 items, only 32 have psychometric eligibility [33Black AC, McCoach DB. Validity study of the thinking styles inventory. J Educ Gift 2008.
[http://dx.doi.org/10.4219/jeg-2008-849]
]. It seems that the theoretical basis used, namely mental self-government, which considers the brain mechanism as a government organization less relevant. More or less the same situation happened in the Rational Experiential Inventory - REI [20Epstein S, Pacini R, Denes-Raj V, Heier H. Individual differences in intuitive-experiential and analytical-rational thinking styles. J Pers Soc Psychol 1996; 71(2): 390-405.
[http://dx.doi.org/10.1037/0022-3514.71.2.390] [PMID: 8765488]
], the development of which is itself based on Cognitive Experiential Self-Theory (CEST). Thus, the development of thinking pattern instruments is still open to concern.

Identifying thinking patterns is an important part of the process of improving thinking performance in education. Unfortunately, until now, there is no instrument that can be used to identify patterns of thinking adequately. Therefore, developing instruments that include psychometric requirements is a necessity task. This research aims to develop a thinking pattern instrument that meets psychometric requirements, namely validity and reliability. If the purpose of this research could be achieved, as a result, a new model that adequately describes the thinking patterns in students could be implemented by the researcher and lectures to improve student education. As educators, we will not be able to improve students' thinking if we are unable to find the relevant thinking patterns. Error thinking, in turn, will have an impact on errors in behavior. Thus, the results of this research are very useful for making an intelligent civilization in a constructive and progressive manner.

2. STUDY I

2.1. Methods

At this stage, the research aims to compile the items developed based on dual-processing theory and test them with exploratory factor analysis in order to find the conceptual structure and loading factors of each item. Participants at this stage were 310 students from the Faculty of Sports Science, State University of Surabaya, consisting of 196 men and 114 women, with a mean age of 19.13 ±1.02 years.

In developing the instrument, there are five steps that need to be done [29Cohen ME. RJ & Swerdlik, Psychological testing and assessment: An introduction to test and measurement 6th ed. 2005.], namely (1) conceptualization, (2) compilation of items and scales, (3) testing, (4) analysis of items, which includes testing of validity and reliability, and (5) final revision and format. The instrument was developed based on the process of cognition commonly referred to as system 1 (i.e. limbic-related system) and system 2 (i.e. neocortex-related system). The pattern of thought process for system 1 is characterized by unconsciousness, automatic, fast, perceptive, associative, pragmatic, parallel, sterotype, the results of which are more optimal; while the pattern of thought process for system 2 is characterized by awareness, control, seriousness of efforts, slow, analytical, procedural, logical, sequential, egalitarian, and the final result is more optimal. These characteristics for both the system were described in detail by Evans [14Evans JSBT. Dual-processing accounts of reasoning, judgment, and social cognition. Annu Rev Psychol 2008; 59: 255-78.
[http://dx.doi.org/10.1146/annurev.psych.59.103006.093629] [PMID: 18154502]
] and Epstein et al. [20Epstein S, Pacini R, Denes-Raj V, Heier H. Individual differences in intuitive-experiential and analytical-rational thinking styles. J Pers Soc Psychol 1996; 71(2): 390-405.
[http://dx.doi.org/10.1037/0022-3514.71.2.390] [PMID: 8765488]
]. In the brain mechanism, system 1 is influenced by the limbic system, the area of ​​the brain responsible for emotions, whereas system 2 is driven by the neo-cortex, the area of ​​the brain responsible for rational thoughts. Thus, according to these concepts and indicators (indicated characteristics) forty statements of relevant items were developed and used in this study. The set of items was designed in the form of self-report using a Likert scale, from 1 (strongly disagree) to 6 (strongly agree).

After the items were arranged including the procedures for their implementation, the researcher conducted a field trial. In this study, we identified problems emerged due to the implementation of collected data, and we also tested the validity of the construct using an exploratory factor analysis [6Nunnally IH. JC & Berstein, Psychometric theory 3rd ed. 1994., 7Hair WC. JH, Anderson, RE, Tatham, RL, & Black, Multivariate data analysis, Fifth 1998.]. In conducting exploratory factor analysis, there are two stages: factor extraction and factor rotation. At the factor extraction stage, the results are often not interpretable, therefore, a second stage of factor rotation is needed. In this research, factor extraction was carried out by the principal component analysis, while factor rotation was carried out by the oblique component. The method ensures that there is a correlation between the different factors. In other words, the internal factors are dependent.

2.2. Results

The factor analysis has shown that five factors of the IRS explain 52.57% of the total variance and their item loading factors are within 0.40 - 0.80. The loading factor is one important indicator to find out to what extent an item contributes to the total variance. According to Nunnally and Berstein [6Nunnally IH. JC & Berstein, Psychometric theory 3rd ed. 1994.], the loading factor is considered sufficient if ≥ .30. Additionally, the IRS factors are determined by eigenvalue > 1. A number of items that have a loading below .30 and an eigenvalue <1 were automatically eliminated. Thus, the selected items had a high enough feasibility, especially in terms of construct validity (Table 1).

Table 1
Factor loadings and structure of the IRS


By using loading factor > .30 and eigenvalue > 1, we have 20 items categorized into 5 factors. To clarify, Fig. (1) illustrates the scree plot to determine the number of latent factors. Factor 1 (effort) explains 14.97% of the total variance, consisting of 7 items with has a loading factor of 0.40 - 0.77. Factor 2 (evaluation) explains 10.98% of the total variance, consisting of 4 items with loading factors 0.61 - 0.79. Factor 3 (tempo) explains 9.23% of the total variance, consisting of 3 items with loading factors 0.61 - 0.75. Factor 4 (response) explains 8.86% of the total variance, consisting of 3 items with loading factors 0.46 - 0.71. Factor 5 (result) explains 8.51% of the total variance, consisting of 3 items with loading factors 0.38 - 0.80.The naming of the five factors is related to the substance of the items in each factor by referring to dual-processing theory [13Kahneman D. Thinking fast and slow 2011., 14Evans JSBT. Dual-processing accounts of reasoning, judgment, and social cognition. Annu Rev Psychol 2008; 59: 255-78.
[http://dx.doi.org/10.1146/annurev.psych.59.103006.093629] [PMID: 18154502]
]. In theory, the differences in thinking patterns are determined by individual efforts, willingness to evaluate, and the speed of responding to stimuli. At this stage, the IRS reliability was also estimated using Cronbach’s Alpha. The test was carried out simultaneously for 20 items of IRS. The test results show that the criterion of the IRS reliability is that it must be equal to 0.71. Referring to the reliability criteria [7Hair WC. JH, Anderson, RE, Tatham, RL, & Black, Multivariate data analysis, Fifth 1998., 34Farooq R. Role of structural equation modeling in scale development. J Adv Manag Res 2016; 13(1): 75-91.
[http://dx.doi.org/10.1108/JAMR-05-2015-0037]
], the IRS is proven to be reliable.

Fig. (1)
Scree plot of the latent factors.


3. STUDY II

3.1. Methods

At this second stage, the research aims to test the consistency of the IRS conceptual structure by using confirmatory factor analysis based on Structural Equation Modeling (SEM) with IBM Amos 23. In addition, the effect of thinking patterns (IRS) was also tested on academic performance variables. Participants at this stage of the research were students of the State University of Surabaya who came from the departments of psychology, economics, and science. Respondents were 344 students, including 64 men and 280 women with a mean age of 19.36 ±1.01 years.

Before filling out the questionnaire, the respondents were informed about the objectives and procedures of the IRS test in order to make data collection effective. The IRS consists of 20 bipolar items with 10 positive and 10 negative statements. The IRS response uses a Likert scale, ranging from 1 (disagree) to 6 (agree). If a score is close to 6, it means the respondents tend to have reflective thinking patterns and if a score is close to 1, it means the respondents tend to have intuitive thinking patterns. IRS filling is done by the respondents in ± 15 minutes. All collected data were verified to ensure their feasibility. Recoding was done on negative statements. The data were analyzed using confirmatory factor analysis based on SEM [7Hair WC. JH, Anderson, RE, Tatham, RL, & Black, Multivariate data analysis, Fifth 1998., 35Byrne BM. Structural equation modeling with lisrel, prelis, and simplis: basic concepts, applications, and programming 1998.].

It is assumed that a test as psychometric instrument has adequate validity and reliability; if it is a replicable test, its results are relatively consistent [4Jacoby WG. Data theory and dimensional analysis 1991.
[http://dx.doi.org/10.4135/9781412983860]
-6Nunnally IH. JC & Berstein, Psychometric theory 3rd ed. 1994.]. Therefore, after the confirmatory factor analysis, the relationship of thinking patterns with other relevant variables was determined. These variables of academic performance are indicators of curiosity [36Cacioppo JT, Petty RE. The Need for Cognition. J Pers Soc Psychol 1982; 42(1): 116-31.
[http://dx.doi.org/10.1037/0022-3514.42.1.116]
] and grade point average (IPK, Indeks Prestasi Komulatif). IPK is calculated as the ratio of the score obtained in every subject matter weighted with the total number of class credit hours she/he took. The IPK has a scale ranging from 0 to 4. It is assessed at the end of each semester.

3.2. Results

The results of confirmatory factor analysis have shown that the conceptual structure of IRS is confirmed by the collected data. IRS consists of 5 factors, which include effort, evaluation, tempo, response, and result. All items correlate significantly with their respective factors (Fig. 2). In the effort factor, regression coefficients range between 0.41 - 0.71 with p<.01; factor evaluation has a regression coefficient of 0.52 - 0.69 with p<.01; the tempo factor has a regression coefficient of 0.47 - 0.72 with p<.01; response factor has a regression coefficient of 0.40 - 0.69 with p<.01; and the result factor has a regression coefficient of 0.27 - 0.73 with p<.01. Meanwhile, the interrelations between factors are also significant, except the interrelations between effort-tempo and effort-response factors.

The analysis of the relationship of thinking patterns with academic performance is shown in Fig. (3). The results of SEM analysis prove that the thinking patterns measured by the IRS have a significant correlation with students’ academic performance, that is indicated by a regression coefficient of 0.82 with p<.01. Thus, the thinking pattern is an effective predictor of academic performance estimated via IPK and curiosity. In order to examine whether the model of structural relations between variables fit with the empirical data, the CFI (comparative fit index) and RMSEA (root mean square error of approximation) were used as well [7Hair WC. JH, Anderson, RE, Tatham, RL, & Black, Multivariate data analysis, Fifth 1998., 37Byrne BM. Structural equation modeling with AMOS: Basic concepts, applications, and programming 2010.]. Our results have shown that there is no discrepancy between the empirical data and the hypothesized mode of structural relation of variables (CFI = 0.96 and RMSEA = 0.07).

Observing the effect size between variables, both direct and indirect effects explain a fairly strong relationship (Table 2). The strongest direct effect of thinking pattern and the effort variable was detected (regression coefficient= 0.84, SE= 0.09,p<.01). Also, the significant direct effect of thinking pattern on the result variable (regression coefficient =0.51, SE= 0.27, p<.01) and the evaluation variable (regression coefficient =0.30, SE=0.66, p<0.01) was determined. The data explains that a person's thinking patterns are largely influenced by efforts to search and find solutions, evaluations of information and events, and the quality of the results obtained. Serious efforts and evaluation indicate that a person has reflective thinking, on the contrary, if both variables are weak, then a person has an intuitive thinking pattern [4Jacoby WG. Data theory and dimensional analysis 1991.
[http://dx.doi.org/10.4135/9781412983860]
, 14Evans JSBT. Dual-processing accounts of reasoning, judgment, and social cognition. Annu Rev Psychol 2008; 59: 255-78.
[http://dx.doi.org/10.1146/annurev.psych.59.103006.093629] [PMID: 18154502]
]. Likewise is the result variable, the reflective thinking tends to produce meaningful and satisfying decisions in the long run. Meanwhile, the intuitive thinking tends to produce decisions that are less meaningful and only satisfying in the short term. In addition, in

Fig. (2)
Five-factor model of IRS.


Fig. (3)
Causal structure of thinking patterns and academic performance.


Table 2
Effect size in adjusted structural equation model0.


Table 2, there are data of significant indirect effect of thinking patterns on curious variables and GPA. The regression coefficients were equal to 0.51 for curious variable and 0.21 for GPA (p<0.01). The data explains that the individuals with the reflective thinking pattern tend to have a high desire to gain knowledge and will achieve high academic performance as well.

It is interesting to know whether there is a correlation between IRS and CRT analyses. We have discussed above that the CRT analysis is commonly used to measure thinking patterns, but it has several weaknesses. This research has determined the relationship between the IRS and CRT. The correlation test results showed that the IRS correlated with the CRT coefficient 0.088 with p= 0.001. That is, IRS has validity equivalent to CRT, both methods can be used to measure one's thinking patterns. However, when the predictive power for these two instruments was determined using regression analysis, we found that the IRS could also predict a student’s success in academic performance (Table 3), while the CRT cannot predict.

Table 3
Regression analysis of IRS and CRT on GPA*.


4. DISCUSSION

Through an in-depth examination process, starting from exploratory factor analysis, confirmatory factor analysis, and SEM, this research succeeded in developing a scale of thinking patterns that meets psychometric criteria. The IRS has 20 items (see appendix) that are bipolar-related, consisting of 5 factors, namely: effort, evaluation, tempo, response, and result. These factors reflect the attributes of thinking as demanded by a good instrument [4Jacoby WG. Data theory and dimensional analysis 1991.
[http://dx.doi.org/10.4135/9781412983860]
-6Nunnally IH. JC & Berstein, Psychometric theory 3rd ed. 1994.]. We identified a number of characteristics such as efforts in searching and finding solutions, speed of responding to stimulus, and evaluation of stimulus, and we have also found new aspects in the thinking process that have been overlooked so far, namely regarding “results” as a dependent variable of thinking patterns. The intuitive thinking patterns often produce dissatisfaction, loss, and even the feeling of regret; while reflective thinking has a greater chance of producing satisfaction [17Chopra RE. D & Tanzi, Super brain: Unleashing the explosive power of your mind to maximize health, happiness, and spiritual well-being 2012.].

IRS has a loading factor of 0.40 - 0.80 with an eigenvalue greater than 1 and explains 52.57% variance, and has a reliability of 0.71. IRS as a psychometric indicator of thinking process is equivalent to REI which consists of 10 items. REI has a loading factor of 0.59 - 0.79 with an eigenvalue greater than 1 and explains 48.2% of variance, with reliability of 0.72 and 0.73 [20Epstein S, Pacini R, Denes-Raj V, Heier H. Individual differences in intuitive-experiential and analytical-rational thinking styles. J Pers Soc Psychol 1996; 71(2): 390-405.
[http://dx.doi.org/10.1037/0022-3514.71.2.390] [PMID: 8765488]
]. The development of IRS is not only based on statistical analysis that is commonly used in the preparation of scala such as inter-item correlations, regression analysis, and Cronbach's Alpha reliability [34Farooq R. Role of structural equation modeling in scale development. J Adv Manag Res 2016; 13(1): 75-91.
[http://dx.doi.org/10.1108/JAMR-05-2015-0037]
], but it includes a more sophisticated approach, namely using SEM. By using SEM, the conceptual relations model between variables can be seen simultaneously [7Hair WC. JH, Anderson, RE, Tatham, RL, & Black, Multivariate data analysis, Fifth 1998., 37Byrne BM. Structural equation modeling with AMOS: Basic concepts, applications, and programming 2010.]. The test results show a little difference between the conceptual model and research observed data with CFI= 0.96 and RMSEA= 0.07.

IRS is an alternative instrument to measure a person's cognitive pattern. It is known that to measure cognition, CRT is a commonly used instrument [1Haigh M. Has the standard cognitive reflection test become a victim of its own success? Adv Cogn Psychol 2016; 12(3): 145-9.
[http://dx.doi.org/10.5709/acp-0193-5] [PMID: 28115997]
, 9Stieger S, Reips UD. A limitation of the Cognitive Reflection Test: familiarity. PeerJ 2016; 4e2395
[http://dx.doi.org/10.7717/peerj.2395] [PMID: 27651989]
, 30Frederick S. Cognitive reflection and decision making. J Econ Perspect 2005.
[http://dx.doi.org/10.1257/089533005775196732]
], although it has many weaknesses [12Pennycook G, Cheyne JA, Koehler DJ, Fugelsang JA. Is the cognitive reflection test a measure of both reflection and intuition? Behav Res Methods 2016; 48(1): 341-8.
[http://dx.doi.org/10.3758/s13428-015-0576-1] [PMID: 25740762]
]. A number of researchers have identified CRT’s weaknesses, such as it is more like intelligence tests, in the form of numerical calculations, and there is an aspect of familiarity that affects the validity of the results. Thus, the results of our investigation are in agreement with some previous studies [1Haigh M. Has the standard cognitive reflection test become a victim of its own success? Adv Cogn Psychol 2016; 12(3): 145-9.
[http://dx.doi.org/10.5709/acp-0193-5] [PMID: 28115997]
, 12Pennycook G, Cheyne JA, Koehler DJ, Fugelsang JA. Is the cognitive reflection test a measure of both reflection and intuition? Behav Res Methods 2016; 48(1): 341-8.
[http://dx.doi.org/10.3758/s13428-015-0576-1] [PMID: 25740762]
]. Our results have proven that IRS is an effective predictor of academic performance. The relationship between thinking patterns and academic performance is a necessity [24Zhang LF. Thinking styles: Their relationship with modes of thinking and academic performance. Educ Psychol 2002; 22: 331-48.
[http://dx.doi.org/10.1080/01443410220138557]
, 38Bahr N. Thinking critically about critical thinking in higher education. Int J Scholarsh Teach Learn 2010.
[http://dx.doi.org/10.20429/ijsotl.2010.040209]
] and that is supported by our finding as well. Thereby, the thinking pattern should directly relate to the learning achievements in students.

What are the implications of this research? Identification of thinking patterns by using good instruments becomes urgent in the education and training of individuals, regarding decision-making. It has been reported that humans do not have an encouraging track record in making decisions and tend to be biased in thinking [39Dobelli R. The art of thinking clearly 2013.]. Many initial decisions become inappropriate after in-depth evaluation of the facts and truths. For instance, it is described that 44% of advocates do not recommend their children to choose the same careers, 40% of executives leave their jobs in the first 18 months, and more than half of the teachers leave their jobs in the first 4 years [40C. H. & D. Heath. Decisive: How to make better choices in life and work 2013.]. In our recent investigation, 13% of students made the wrong choice of the education program. The highest rate occurred in the faculty of social sciences by 26%, followed by the faculty of language and arts by 18%. Regarding the choice of profession, 63% of the students said they would choose a non-educational profession and only 37% chose the teaching profession [41Maksum A, Khory FD. Structural relations model of thinking pattern, curiosity, and academic performance.Research Report 2019.]. Many facts reported in the literature affirm that individual decision making tends to be inaccurate and often leads to regret. The dominated role of limbic system in the process of the intuitive thinking is incontestable.

We designed IRS analysis and used it to describe the thinking pattern in students population. This is to be considered that the problem of thinking is not only the monopoly of the education community [38Bahr N. Thinking critically about critical thinking in higher education. Int J Scholarsh Teach Learn 2010.
[http://dx.doi.org/10.20429/ijsotl.2010.040209]
, 42Moreira C. F., Ferreira, M.J., Cardoso, A., Gomes, A.S. & Collazos, “Learning ecosystem for higher education disruption: A new approach proposal based on blended active learning techniques Proceeding of Edulearn 18 Conference 2018.], but it exists in almost in all areas of life including politics, economics, and government bureaucracy. Thus, in the future, other researchers could implement similar methods and procedures for identification of “typical thinking patterns” in other population groups and compare them. It is possible that there could be a difference in structure on thinking patterns in various professionals .

CONCLUSION AND IMPLICATIONS

This research has answered a number of problems that are of concern to researchers related to the use of CRT as an instrument to measure thinking models. This research has succeeded in developing a scale of thought patterns which we later call IRS. The instrument has a conceptual relationship structure consisting of 5 factors with a loading factor of 0.40 - 0.80. The five factors explain 52.57% of the total variance. From the reliability testing, IRS has a Cronbach’s Alpha of 0.71. IRS is correlated and is a significant predictor of students’ academic performance. In terms of the conceptual relationship model, the IRS has been tested using SEM and proven to be fit with empirical data. In connection with the results of this research, it is recommended that the IRS can be used as an alternative to measure individual thinking tendencies. As an instrument, the IRS needs to be further tested by involving other relevant variables, such as socioeconomic level, education level, parenting, leadership patterns, and successful performance.

ETHICS APPROVAL AND CONSENT TO PARTICIPATE

Not Applicable.

HUMAN AND ANIMAL RIGHTS

Not Applicable.

CONSENT FOR PUBLICATION

All patients participated on a voluntary basis and gave their informed consent.

AVAILABILITY OF DATA AND MATERIALS

Data is available but cannot be shared.

FUNDING

None.

CONFLICT OF INTEREST

The authors declare no conflict of interest, financial or otherwise.

ACKNOWLEDGEMENTS

Declared none.

APPENDIX

Items of IRS.


REFERENCES

[1] Haigh M. Has the standard cognitive reflection test become a victim of its own success? Adv Cogn Psychol 2016; 12(3): 145-9.
[http://dx.doi.org/10.5709/acp-0193-5] [PMID: 28115997]
[2] Travers E, Rolison JJ, Feeney A. The time course of conflict on the Cognitive Reflection Test 2016; Vol. 150
[3] Stupple EJN, Pitchford M, Ball LJ, Hunt TE, Steel R. Slower is not always better: Response-time evidence clarifies the limited role of miserly information processing in the Cognitive Reflection Test. PLoS One 2017; 12(11)e0186404
[http://dx.doi.org/10.1371/journal.pone.0186404] [PMID: 29099840]
[4] Jacoby WG. Data theory and dimensional analysis 1991.
[http://dx.doi.org/10.4135/9781412983860]
[5] Pedhazur LP. EJ & Schmelkin, Measurement, design, and analysis: An integrated approach 1991.
[6] Nunnally IH. JC & Berstein, Psychometric theory 3rd ed. 1994.
[7] Hair WC. JH, Anderson, RE, Tatham, RL, & Black, Multivariate data analysis, Fifth 1998.
[8] Kerlinger FN. Foundations of behavioral research 3rd ed. 1986.
[9] Stieger S, Reips UD. A limitation of the Cognitive Reflection Test: familiarity. PeerJ 2016; 4e2395
[http://dx.doi.org/10.7717/peerj.2395] [PMID: 27651989]
[10] Thomson KS. Investigating an alternate form of the cognitive reflection test. Judgement Decis Mak 2016; 11(1): 99-113.
[11] Cheng J, Janssen C. The relationship between an alternative form of cognitive reflection test and intertemporal choice Stud Psychol (Bratisl) 2019.
[http://dx.doi.org/10.21909/sp.2019.02.774]
[12] Pennycook G, Cheyne JA, Koehler DJ, Fugelsang JA. Is the cognitive reflection test a measure of both reflection and intuition? Behav Res Methods 2016; 48(1): 341-8.
[http://dx.doi.org/10.3758/s13428-015-0576-1] [PMID: 25740762]
[13] Kahneman D. Thinking fast and slow 2011.
[14] Evans JSBT. Dual-processing accounts of reasoning, judgment, and social cognition. Annu Rev Psychol 2008; 59: 255-78.
[http://dx.doi.org/10.1146/annurev.psych.59.103006.093629] [PMID: 18154502]
[15] Rosenzweig P. Left brain, right stuff 2015.
[16] Peters S. The chimp paradox: The mind management 2012.
[17] Chopra RE. D & Tanzi, Super brain: Unleashing the explosive power of your mind to maximize health, happiness, and spiritual well-being 2012.
[18] Doidge N. Brain that changes itself 2007.
[19] Kurzweil R. How to create a mind: The secret of human thought revealed 2013.
[20] Epstein S, Pacini R, Denes-Raj V, Heier H. Individual differences in intuitive-experiential and analytical-rational thinking styles. J Pers Soc Psychol 1996; 71(2): 390-405.
[http://dx.doi.org/10.1037/0022-3514.71.2.390] [PMID: 8765488]
[21] Nichols T. The death of expertise: The campaign against established knowledge and why it matters 2017.
[22] Schwab K. The fourth industrial revolution. Genewa: World Economic Forum. 2017.2017.
[23] Wulandari NH, Widayati KA, Suryobroto B. Cognitive style and creative quality in fluence on academic achievement of university students in indonesia. Hayati J Biosci 2016; 23(3): 121-4.
[http://dx.doi.org/10.1016/j.hjb.2016.09.001]
[24] Zhang LF. Thinking styles: Their relationship with modes of thinking and academic performance. Educ Psychol 2002; 22: 331-48.
[http://dx.doi.org/10.1080/01443410220138557]
[25] Usán Supervía P, Salavera Bordás C. Academic performance, emotional intelligence and academic engagement in adolescents. Electron J Res Educ Psychol 2019.
[26] Maksum A. The dilemma of character education in disruptive social power: An analysis from a neuropsychological perspective. Integrasi PPK dan Gerakan Literasi Nasional dalam Pembelajaran Berbasis HOTS 2019.
[27] Maksum A. Curricullum and learning in higher education: Towards an empowering education. Seminar nasional hasil penelitian pendidikan dan pembelajaran 2015.
[28] McIntyre LC. Post truth 2018.
[http://dx.doi.org/10.7551/mitpress/11483.001.0001]
[29] Cohen ME. RJ & Swerdlik, Psychological testing and assessment: An introduction to test and measurement 6th ed. 2005.
[30] Frederick S. Cognitive reflection and decision making. J Econ Perspect 2005.
[http://dx.doi.org/10.1257/089533005775196732]
[31] Sternberg RJ. Thinking and Problem Solving 2013.
[32] Grigorenko EL, Sternberg RJ. Thinking Styles In: Saklofske M, DH, Zeidner, Eds. International Handbook of Personality and Intelligence 1995; 205-29.
[http://dx.doi.org/10.1007/978-1-4757-5571-8_11]
[33] Black AC, McCoach DB. Validity study of the thinking styles inventory. J Educ Gift 2008.
[http://dx.doi.org/10.4219/jeg-2008-849]
[34] Farooq R. Role of structural equation modeling in scale development. J Adv Manag Res 2016; 13(1): 75-91.
[http://dx.doi.org/10.1108/JAMR-05-2015-0037]
[35] Byrne BM. Structural equation modeling with lisrel, prelis, and simplis: basic concepts, applications, and programming 1998.
[36] Cacioppo JT, Petty RE. The Need for Cognition. J Pers Soc Psychol 1982; 42(1): 116-31.
[http://dx.doi.org/10.1037/0022-3514.42.1.116]
[37] Byrne BM. Structural equation modeling with AMOS: Basic concepts, applications, and programming 2010.
[38] Bahr N. Thinking critically about critical thinking in higher education. Int J Scholarsh Teach Learn 2010.
[http://dx.doi.org/10.20429/ijsotl.2010.040209]
[39] Dobelli R. The art of thinking clearly 2013.
[40] C. H. & D. Heath. Decisive: How to make better choices in life and work 2013.
[41] Maksum A, Khory FD. Structural relations model of thinking pattern, curiosity, and academic performance.Research Report 2019.
[42] Moreira C. F., Ferreira, M.J., Cardoso, A., Gomes, A.S. & Collazos, “Learning ecosystem for higher education disruption: A new approach proposal based on blended active learning techniques Proceeding of Edulearn 18 Conference 2018.
Track Your Manuscript:


Endorsements



"Open access will revolutionize 21st century knowledge work and accelerate the diffusion of ideas and evidence that support just in time learning and the evolution of thinking in a number of disciplines."


Daniel Pesut
(Indiana University School of Nursing, USA)

"It is important that students and researchers from all over the world can have easy access to relevant, high-standard and timely scientific information. This is exactly what Open Access Journals provide and this is the reason why I support this endeavor."


Jacques Descotes
(Centre Antipoison-Centre de Pharmacovigilance, France)

"Publishing research articles is the key for future scientific progress. Open Access publishing is therefore of utmost importance for wider dissemination of information, and will help serving the best interest of the scientific community."


Patrice Talaga
(UCB S.A., Belgium)

"Open access journals are a novel concept in the medical literature. They offer accessible information to a wide variety of individuals, including physicians, medical students, clinical investigators, and the general public. They are an outstanding source of medical and scientific information."


Jeffrey M. Weinberg
(St. Luke's-Roosevelt Hospital Center, USA)

"Open access journals are extremely useful for graduate students, investigators and all other interested persons to read important scientific articles and subscribe scientific journals. Indeed, the research articles span a wide range of area and of high quality. This is specially a must for researchers belonging to institutions with limited library facility and funding to subscribe scientific journals."


Debomoy K. Lahiri
(Indiana University School of Medicine, USA)

"Open access journals represent a major break-through in publishing. They provide easy access to the latest research on a wide variety of issues. Relevant and timely articles are made available in a fraction of the time taken by more conventional publishers. Articles are of uniformly high quality and written by the world's leading authorities."


Robert Looney
(Naval Postgraduate School, USA)

"Open access journals have transformed the way scientific data is published and disseminated: particularly, whilst ensuring a high quality standard and transparency in the editorial process, they have increased the access to the scientific literature by those researchers that have limited library support or that are working on small budgets."


Richard Reithinger
(Westat, USA)

"Not only do open access journals greatly improve the access to high quality information for scientists in the developing world, it also provides extra exposure for our papers."


J. Ferwerda
(University of Oxford, UK)

"Open Access 'Chemistry' Journals allow the dissemination of knowledge at your finger tips without paying for the scientific content."


Sean L. Kitson
(Almac Sciences, Northern Ireland)

"In principle, all scientific journals should have open access, as should be science itself. Open access journals are very helpful for students, researchers and the general public including people from institutions which do not have library or cannot afford to subscribe scientific journals. The articles are high standard and cover a wide area."


Hubert Wolterbeek
(Delft University of Technology, The Netherlands)

"The widest possible diffusion of information is critical for the advancement of science. In this perspective, open access journals are instrumental in fostering researches and achievements."


Alessandro Laviano
(Sapienza - University of Rome, Italy)

"Open access journals are very useful for all scientists as they can have quick information in the different fields of science."


Philippe Hernigou
(Paris University, France)

"There are many scientists who can not afford the rather expensive subscriptions to scientific journals. Open access journals offer a good alternative for free access to good quality scientific information."


Fidel Toldrá
(Instituto de Agroquimica y Tecnologia de Alimentos, Spain)

"Open access journals have become a fundamental tool for students, researchers, patients and the general public. Many people from institutions which do not have library or cannot afford to subscribe scientific journals benefit of them on a daily basis. The articles are among the best and cover most scientific areas."


M. Bendandi
(University Clinic of Navarre, Spain)

"These journals provide researchers with a platform for rapid, open access scientific communication. The articles are of high quality and broad scope."


Peter Chiba
(University of Vienna, Austria)

"Open access journals are probably one of the most important contributions to promote and diffuse science worldwide."


Jaime Sampaio
(University of Trás-os-Montes e Alto Douro, Portugal)

"Open access journals make up a new and rather revolutionary way to scientific publication. This option opens several quite interesting possibilities to disseminate openly and freely new knowledge and even to facilitate interpersonal communication among scientists."


Eduardo A. Castro
(INIFTA, Argentina)

"Open access journals are freely available online throughout the world, for you to read, download, copy, distribute, and use. The articles published in the open access journals are high quality and cover a wide range of fields."


Kenji Hashimoto
(Chiba University, Japan)

"Open Access journals offer an innovative and efficient way of publication for academics and professionals in a wide range of disciplines. The papers published are of high quality after rigorous peer review and they are Indexed in: major international databases. I read Open Access journals to keep abreast of the recent development in my field of study."


Daniel Shek
(Chinese University of Hong Kong, Hong Kong)

"It is a modern trend for publishers to establish open access journals. Researchers, faculty members, and students will be greatly benefited by the new journals of Bentham Science Publishers Ltd. in this category."


Jih Ru Hwu
(National Central University, Taiwan)


Browse Contents




Webmaster Contact: info@benthamopen.net
Copyright © 2020 Bentham Open