RESEARCH ARTICLE


Development and Validation of the Fatigue State Questionnaire: Preliminary Findings



Spencer Greenberg*, Pluta Aislinn, DeConti Kirsten
Department of Mathematics, New York University and Gimbel Technologies, New York, USA


Article Metrics

CrossRef Citations:
5
Total Statistics:

Full-Text HTML Views: 6588
Abstract HTML Views: 2812
PDF Downloads: 2967
ePub Downloads: 767
Total Views/Downloads: 13134
Unique Statistics:

Full-Text HTML Views: 3345
Abstract HTML Views: 1381
PDF Downloads: 2093
ePub Downloads: 502
Total Views/Downloads: 7321



Creative Commons License
© Greenberg et al.; Licensee Bentham Open

open-access license: This is an open access article licensed under the terms of the Creative Commons Attribution-Non-Commercial 4.0 International Public License (CC BY-NC 4.0) (https://creativecommons.org/licenses/by-nc/4.0/legalcode), which permits unrestricted, non-commercial use, distribution and reproduction in any medium, provided the work is properly cited.

* Address correspondence to this author at the Gimbel Technologies, 350 E 54th St, #1A, New York, NY 10022 USA; Email: spencer@gimbeltech.com.


Abstract

Study Objectives:

To develop and test an easy to administer, conceptually sound, self-report fatigue state questionnaire, the Fatigue State Questionnaire (FSQ).

Design:

A self-report study.

Setting:

Internet-based study.

Participants:

214 adults recruited via the Internet website, Mechanical Turk.

Interventions:

Not applicable

Measurements and Results:

The FSQ showed adequate internal consistency; Chronbach's alpha ranged from .73 to .82. Test-retest reliability after a ten-minute interval was also acceptable (r=.71). The FSQ had incremental validity over the (SSS) in predicting measures of participant health (r=-.25 vs. r=-.11, z=-2.30, p=<.05), sleep debt (r=.30 vs. r= .15, z=2.82, p<.01) and sleep changes (over or under sleeping by 90 minutes or more) on the night prior (r=.35 vs. r=.22, z=2.20, p<.05). FSQ scores were significantly higher in unhealthy participants compared to healthy participants and in participants with a sleep debt or a sleep change compared to participants with their ideal amount of sleep. FSQ scores were also significantly higher in participants taking the test during a circadian low with sleep debt or sleep changes than in participants taking the test during a circadian mid or high point with these sleep differences.

Conclusion:

The FSQ shows promise as a reliable, valid instrument for measuring the fatigue state. Future research should compare within-subject FSQ scores at multiple intervals across the circadian cycle to further assess validity.

Keywords: Circadian rhythm, Fatigue, Health, Sleepiness, Sleep deprivation.



INTRODUCTION

Fluctuations in daytime sleepiness and fatigue can range from normal to clinical with prevalence rates estimated between 30 and 35% in the general population [1-5]. Public safety officers, physicians, and psychologists recognize the need to manage moment-to-moment changes in daytime sleepiness and fatigue [2] in order to prevent accidents [1-6], to optimize health [7, 8], and to increase productivity on cognitive tasks [9, 10]. Though over 100 tools exist to measure fatigue and sleepiness [11], these are largely targeted toward diagnosis of clinical sleep disorders, involve costly or cumbersome medical equipment, or track sleepiness and fatigue across a wide spectrum of time rather than on a state basis [12]. Quick, reliable, and conceptually sound self-report tools are needed to track fluctuations in fatigue and sleepiness states [5-13].

Definitions of Daytime Sleepiness and Fatigue

Sleepiness and fatigue have been recognized as distinct concepts [1, 3, 5, 13, 14]. Daytime sleepiness has been defined by Johns [13 p2] as “inclined to sleep, having difficulty in keeping awake, drowsy, somnolent”. A similar definition of sleepiness has been proposed by Pigeon et al. [5] and includes “drowsiness, sleep propensity, and decreased alertness”. In lay terms, sleepiness is synonymous with drowsiness [13]. On the clinical ends of the sleepiness spectrum are the disorders known as excessive daytime sleepiness and insomnia [13]. Fatigue, on the other hand, has been defined as an “overwhelming sense of tiredness, lack of energy and a feeling of exhaustion, associated with impaired physical and/or cognitive functioning” [4 p70, 15]. Fatigue is linked with physical and/or mental exertion and the inability to continue performing a task [1], and is synonymous with the common term tiredness [13]. Pigeon et al. [5 pp62-67] operationalize fatigue as “weariness, weakness, and depleted energy” and also note its possible manifestation as mental and/or physical. Acute fatigue occurs daily with a rapid onset and dissipates after rest. It is considered normal and does not interfere with the enjoyment of daily activities. Chronic fatigue occurs in clinical populations, does not respond to alterations in activity level or sleep, and diminishes the quality of life [4].

Causes of Daytime Sleepiness and Fatigue

Complex causation models have been developed to explain daytime sleepiness [1, 13, 16] and daytime fatigue [17, 18]. Although the models use slightly different terminology, they are remarkably similar. The models identify four major causative factors: the circadian cycle [19], the homeostatic level or sleep debt [20, 21], and internal and external variables [12].

In the model for sleepiness described by Johns, [13 p4] the sleep drive and the wake drive interact with each other by mutual inhibition to determine the current level of sleepiness. The sleep drive refers to the homeostatic level as measured by the amount of adenosine throughout the brain (sleep debt in the Berg [1] model). The wake drive is divided into the primary and secondary wake drives. The primary wake drive refers to the circadian point as measured by the amount of melatonin in the suprachiasmatic nuclei. The secondary wake drive is composed of a number of exteroceptive variables, such as task type and noise level, and interoceptive variables (internal and external stimulation in the Berg [1] model). The interoceptive variables can be temporary (e.g., number of calories consumed in the last hour) or sustaining individual differences [23] like chronotype (i.e. morning vs night preference [10, 24], or age [25]). Daytime sleepiness occurs when the primary and secondary wake drives “combined, are not strong enough to counter and inhibit our sleep drive sufficiently so as to maintain alert wakefulness” [13 p4] (See also [22] for a mathematical explanation of the interaction between circadian and homeostatic processes).

In the model for fatigue described by Belenky [18], the four causative factors are termed: homeostatic sleep drive, circadian rhythm phase, workload, and individual differences. The homeostatic sleep drive is determined by the number of hours awake and prior sleep/wake history. Fatigue increases with time awake, but the sinusoidal circadian rhythm (time of day) can modulate this linear increase. Workload is a combination of task type and time on task. Fatigue increases as task demands and task time increase. These effects are amplified when time awake is high and circadian nadir is low. The homeostatic sleep drive, the circadian rhythm, and workload are modulated by individual differences such as chronotype and responses to caffeine and other countermeasures.

Relationship Between Daytime Sleepiness and Fatigue

Although the models of sleepiness and fatigue identify the same causative factors, it is unclear whether sleepiness and fatigue develop simultaneously or sequentially, and whether the two are independent or interacting consequences. Johns [13] suggest that fatigue can occur after prolonged wakefulness because of the mental and physical effort involved in maintaining alertness and an upright body position. In this case, fatigue occurs after sleepiness that has been masked or attenuated by environmental (e.g., light, noise) or other means (e.g., caffeine). Fatigue can also occur long prior to sleepiness after mentally or physically taxing experiences. In sleep disordered patients fatigue was found to occur independently of sleepiness on a regular basis [3]. This pattern is also found in other clinical populations [5]. However, in a study of over 5000 women, Theorell-Haglow et al. [26] found that 9.5% of the women routinely suffer from both daytime fatigue and sleepiness. Whether sleepiness and fatigue develop side by side or in sequence requires further study and the patterns identified will likely be different in normal and clinical populations.

Findings on the causal relationship between sleepiness and fatigue are also less than clear. Studies employing driving simulators have found that fatigue and the resulting performance decline can ultimately result in sleepiness [27]. The authors concluded that long periods of driving may lead to mental and physical fatigue when there are no landscape variations to spike mental alertness and no steering demands to activate physical processes. When attention shifts from external to internal schemas of driving, and this shift is uninterrupted for a period of time, sleepiness can result.

Other research suggests that sleepiness may cause greater mental fatigue. Subjects who have sleep/wake cycles that are not entrained to day and night may fatigue more easily on cognitive tasks than subjects whose sleep schedules are entrained [28, 29]. The relationship between sleep disruption (presumed sleepiness) and fatigue has been attributed to core body temperature. When body temperature is at its circadian low, more sleepiness is likely to be reported that when it is at its circadian high. Low core body temperatures are also associated with fatigue (e.g., reaction time, memory) on cognitive tasks. Perhaps the relationship between sleepiness and fatigue is modulated by at least one common biological process (i.e., core body temperature). In a study focusing only on fatigue, mentally fatigued subjects reached physical fatigue faster than mentally rested subjects even though their cardio-vascular and muscular expenditures were the same. Underlying changes in brain activity were proposed to account for the causal relation between mental and physical fatigue [30]. Fatigue and sleepiness may also share neuro-cognitive mechanisms such that fatigue is reached more quickly in sleepy subjects than in well-rested subjects.

Measures of Daytime Sleepiness and Fatigue

Compared to objective measures, there are relatively few subjective measures of daytime sleepiness and fatigue. Of those few, most are not relevant to sleepiness and fatigue states because they measure sleepiness and fatigue as traits over some period of time. The remaining measures of state sleepiness and fatigue have been criticized because of their outdated language, confusion of sleepiness and fatigue terms, and their psychometric limitations (Table 1).

Table 1. State measures of daytime sleepiness and fatigue.
Fatigue
Number of items Sample question or descriptor Updated Language? Distinguishes sleepiness and fatigue? Psychometric Usefulness?
VAS 18 Not at all active vs. very active YES NO Requires z-scores for between subject designs
Samn-
Perelli
1 Moderately tired, let down YES YES Limited to one question
FSQ 4 How mentally tired are you? YES YES Good
Sleepiness
SSS 1 Relaxed; awake; not at full alertness YES NO Limited to one question
KSS 1 Neither alert nor sleepy YES NO Limited to one question

Numerous objective tools for measuring sleepiness and fatigue have been developed and are widely used for research purposes [4, 11, 13]. Objective measures of sleepiness include measures of slow eye movements (EOG), electrical activity in the brain (EEG) or in the muscles and nerves (EMG), and sleep latency (MSLT). Objective measures of fatigue include measures of heart rate and heart rate variability, reaction time on tasks, and performance on vigilance tasks [1]. As noted by Berg [1], the objective measures tend to be too costly, cumbersome and time consuming for work in the field where monitoring moment-to-moment changes in sleepiness and fatigue is most important.

Although the subjective measures of sleepiness and fatigue are easier to use, there are far fewer of them. In addition, the tests have been criticized on a number of grounds. Some of these tests, like the Epworth Sleepiness Scale [31], the Chalder Fatigue Scale [32], the Brief Fatigue Inventory [33], and the sleepiness subscale of the Profile of Mood States [34], are not suited to measure moment to moment changes in sleepiness because they are designed to measure subjects’ typical levels of sleepiness (i.e., trait) rather than subjects’ current level of sleepiness (i.e., state) [4-35]. For example, the ESS asks subjects to rate how likely (from 0 never, to 3 high chance) they are to “doze off” in eight different situations (e.g., driving, watching TV) [31]. Most of the subjective fatigue tests reviewed by Shen et al. [4] ask about fatigue during the past week (Multi-dimensional Fatigue Inventory-20) or in typical situations (Fatigue Impact Scale). Many of the scales developed for use with specific clinical populations similarly ask about fatigue levels in the recent past rather than in the moment [5].

Measures of Sleepiness State

The Stanford Sleepiness Scale (SSS) [36] and the Karolinska Sleepiness Scale (KSS) [37] were designed to measure sleepiness in the moment. The SSS is a single item test that measures state sleepiness [21, 36]. Subjects are asked to rate their current level of subjective sleepiness on a 7 point scale with complex descriptors. The descriptors have been criticized for being ambiguous [11] and inconsistent at each level [38]. Because of the numerous descriptors at each level, the test seems to measure more than a unidimensional concept of sleepiness. Factor analyses have identified two factors (activation and sleepiness) [38] or three factors (alertness/sleepiness, loss of control, and a cognitive factor) [4]. The findings of multiple factors and the ambiguous wording of descriptors suggest that subjective feelings of sleepiness and fatigue are confounded within this measure. Pigeon et al. [5] report a similar clouding of concepts on measures of sleepiness and fatigue used by physicians. The KSS measures state sleepiness using a single Likert item scale [37]. Both five and nine point scales have been used. The nine-point scale has briefer descriptions than the SSS, but uses terms associated with both fatigue (e.g. "alert") and sleepiness (e.g. "fighting sleep").

Measures of Fatigue State

Although it is called the Visual Analogue Sleepiness Scale, the VAS was designed to measure fatigue [39]. Subjects are asked to rate how they are feeling “RIGHT NOW” by marking along a 100 cm horizontal line between polar opposites on 18 items. Thirteen of the items are fatigue related (e.g., not at all worn out vs extremely worn out) and five of the items are energy related (e.g., not at all active vs extremely active [39]). Scores are tallied using the length of the line between the subject’s mark and the pole descriptor. Shen et al. [4] caution that the VAS may be well suited only for within-subject comparisons. Using the VAS for between-subject comparisons may require standardization of scores (z-scores) because of individual differences in subjects’ marking styles. This criticism also applies to the pictorial version of the VAS [40].

The Samn-Perelli Fatigue Checklist was originally designed as a simple measure of subjective state fatigue in pilots [17] and is now widely used in fatigue research. Subjects rate their current fatigue on a 7-point Likert scale: 1 Fully Alert, Wide Awake, Extremely Peppy; 2 Very Lively, Responsive, But Not at Peak; 3 Okay; Somewhat Fresh; 4 A Little Tired, Less Than Fresh; 5 Moderately Tired, Let Down; 6 Extremely Tired, Very Difficult to Concentrate; 7 Completely Exhausted, Unable to Function Effectively, Ready to Drop. The test is easy to administer and reliable, but the single item format may not capture the dual manifestations (mental/physical) of fatigue [41]. The Pearson-Byars Fatigue Feeling Checklist [42] uses similar language but has 13 items that are rated “better than”, “same as”, or “worse than”. Mendoza et al. [33] have criticized the Pearson-Byars for its idiomatic language because it is difficult to understand and to translate.

Johns [13] notes that the VAS, KSS, and SSS, don’t distinguish between sleepiness and fatigue. For example, although the SSS was designed to measure sleepiness, the items contain descriptors asking about both sleepiness (e.g., fighting sleep) and fatigue (e.g., feeling active and vital). The KSS is a cleaner measure of sleepiness in that all the descriptors on the single item scale relate to sleepiness. The same can be said of the Samn-Perilli Scale for fatigue. However, the instability of single item subjective measures has been noted [39]. The VAS was designed to measure fatigue and has two subscales, one labeled fatigue and the other labeled energy and vigor. The later consists of 5 items, some of which may reflect sleepiness rather than fatigue (e.g., I have absolutely no desire at all to close my eyes). Future scales should measure sleepiness and fatigue separately, use language that conforms closely to the conceptual definitions of the two concepts [5] and contain more than one item to insure strong test- retest reliability.

The purpose of this study is to develop a measure of fatigue that is as easy, or easier, to administer as the existing tests, includes more than one item to increase reliability, uses up to date language, and does not confound fatigue with sleepiness. The new measure, called the Fatigue State Questionnaire, consists of 4 items, and is predicted to be more reliable than a single item measure of fatigue or a single item measure of sleepiness (SSS). Only modest correlations between scores on the FSQ and the SSS are expected as daytime fatigue and sleepiness are related, but distinct states. Scores on the FSQ are also predicted to differ depending on the amount of sleep the night prior, circadian level, and health, three variables thought to be causes of fatigue.

METHODS

Design of the Questionnaire

The FSQ was developed from an initial list of 27 questions generated by the researchers (Appendix 1). The questions were tied closely to the conceptual definition of fatigue. For example, “weariness and weakness” were targeted with questions like “How slow and sluggish are you right now? The concept of energy level was explored with questions like “How awake do you feel right now?” The original set of questions also queried both the mental and physical aspects of the definition of fatigue (e.g. “How tired does your mind feel right now?” and “How tired does your body feel right now?”).

The original question list was narrowed using both qualitative and quantitative methods. In a pilot study, 20 subjects were asked to answer all 27 questions and to explain their answers in an open-ended format. Questions that elicited confused responses or responses unrelated to fatigue were deleted. Through additional pilot testing, items with low item-total correlations were removed. In addition, redundant questions with very high item-to-item correlations were removed. The four items that remained were used as the measure of fatigue or the FSQ (Appendix 2). The four questions on the FSQ were presented in a multiple-choice format with five options: Not at all; A little; Moderately; Very; and Extremely. Responses to questions 1, 2, and 4 were scored as 0 for Not at All, 1 for A Little, 2 for Moderately, 3 for Very and 4 for Extremely. Responses to question 3 were reversed scored with 0 for Extremely and 4 for Not at All. Total scores were calculated by adding the responses from the four questions. Possible scores ranged from 0 to 16.

The pilot procedure can be viewed in detail at www.guidedtrack.com/programs/121cp82/run.

Participants

Three hundred forty paid participants were recruited from an internet-based recruitment platform (Mechanical Turk) to take part in the study. Individuals were eligible to participate in the study if they were at least 18 years old. Participants were paid $0.30 US dollars for their time. Of the 340 participants recruited, 66 withdrew early from the study. An additional 15 were not included in follow-up analysis due to taking more than 25 minutes to complete the study, the predetermined maximum length deemed acceptable. Another 45 participants were not included in follow-up analysis for failing one or more of two simple multiple choice questions that were added to ensure participants were proficient in English and were reading the questions thoroughly. Outliers were not anticipated. Of the initial pool of participants, 214 were included in the analyses.

Materials

To assess fatigue, in addition to the FSQ, all subjects completed a single-item measure included as a comparison. The common synonym for fatigue, “tired”, was used to form this single fatigue state question: “How tired are you, right now?” Participants answered this question on a five-point Likert scale that ranged from 2 to -2: 2 “Very tired”, 1 “Somewhat tired, 0 “Neither tired nor energetic”, -1 “Somewhat energetic” and -2 “Very energetic.” This question was included to compare the robustness of the FSQ to a single item measure of fatigue (Single Fatigue Question).

To assess sleepiness, all participants completed the SSS. Options on the Likert scale ranged from 1 to 7: 1=Feeling active and vital, alert, wide awake; 2=Functioning at a high level, but not peak, able to concentrate; 3=Relaxed, awake, not at full alertness, responsive; 4=A little foggy, not at peak, let down; 5=-Fogginess, beginning to lose interest in remaining awake, slowed down; 6=Sleepiness, prefer to be lying down, fighting sleep, woozy; 7=Almost in reverie, sleep onset soon, lost struggle to remain awake. A measurement of sleepiness was included to assess the relationship between the sleepiness state and the fatigue state. It was predicted that sleepiness and fatigue would be moderately positively correlated.

A binary measurement of participants’ current health status was taken by asking, “Have you been feeling sick at all today?” Participants could select “No, I'm healthy today”, “Yes, I have a cold today”, “Yes, I have a temporary sickness (other than a cold)”, or “Yes, I have a chronic sickness that is affecting me today”. Participants were coded as either feeling healthy (those who selected the “No” option) or as feeling unhealthy (those who selected a “Yes” option). Sickness and other health problems tend to increase feelings of fatigue and tiredness [3, 33]. Therefore, it was predicted that FSQ scores would negatively correlate with health.

To assess the amount of sleep on the night prior to the survey, participants were asked two questions: “How many hours did you sleep last night?” and “On an ideal night, how many hours of sleep do you normally need in order to feel your best?” Research suggests that the ideal amount of sleep individuals require can vary widely [43]. In addition, both under sleeping and oversleeping can produce fatigue [28]. Therefore, the measure of adequate sleep was not the quantity of sleep participants had the night before, but whether or not they obtained within 90 minutes of their ideal level of sleep. In advance of data collection, it was decided to code participants who obtained no more than 90 minutes above their ideal, or no fewer than 90 minutes below their ideal, as having had adequate sleep. Participants with who were more than 90 minutes under or over their ideal amount of sleep were coded as having sleep changes. Participants were coded as having sleep debt when the quantity of sleep reported was more than 90 minutes less than their ideal. Both sleep changes and sleep debt were predicted to be positively correlated with FSQ scores.

Participants were also asked for the local time and for the amount of sleep they had two nights prior if they remembered. Local time was used to estimate circadian low points. The point in the circadian cycle was coded as low if the local time was between 12 AM and 5 AM or between 1 PM and 3 PM. The amount of sleep two nights prior was not analyzed in this study. These variables, as well as the prior measures listed, constitute an exhaustive list of all data collected.

Procedures

Participants completed the study from their personal computers. The first few questions were related to a sound check procedure to ensure their computer's volume was on and audible. After completing demographic questions, participants answered questions relating to fatigue (FSQ and Single Fatigue Question), sleepiness (SSS), the previous nights’ rest (3 questions) and health (1 question). The language proficiency/attention question was inserted between the FSQ and the Single Fatigue Question. Participants then had a ten-minute break. They were instructed as follows: “You may do whatever you like in these ten minutes, but you must not close the window that this screen is in. You may view other websites if they're in other tabs of your browser.” The amount of time remaining in the ten-minute interval was periodically displayed. Participants were instructed to return when they heard the chime or saw that the time was up. Participants then completed the FSQ, a different language proficiency/attention question, the Single Fatigue Question, and the SSS a second time. The complete procedure can be viewed at www.guidedtrack.com/programs/axqxghh/run. The GuidedTrack code for the program can be found at www.guidedtrack.com/programs/664/edit.

RESULTS

Participant Demographics

The demographic characteristics of the participants are listed in Table. 2. The participants were 214 adults ranging in age from 19 to 70 living in either the United States, India or other countries. The majority of the participants were between 22 and 32 years of age. There were about equal numbers of males and females and about 66% held college degrees.

Descriptive Statistics for the FSQ, SSS and The Single Fatigue Question

The mean score on the FSQ out of a possible 16 was 4.76 (SD=2.86) at time 1 and 3.99 (SD=2.97) at time 2 (Table 3). The frequency distribution of scores for time 1 shows a skewed distribution with the majority of the scores occurring toward the lower range of possible scores (Fig. 1). The distribution for time 2 is similar. The mean score on the SSS out of a possible 7 was 2.46 (SD=1.47) at time 1 and 2.25 (SD=1.29) at time 2. The frequency distributions of SSS scores for time 1 and time 2 were also skewed with the majority of the scores occurring toward the lower range of possible scores. The mean score on the Single Fatigue Question out of a possible 2 was -.09 (SD=1.19) at time 1 and -.33 (SD=1.17) at time 2. The frequency distributions of scores for time 1 and time 2 showed the majority of the scores occurring at zero, indicating that the majority of the participants were feeling “neither tired nor energetic”. The low scores on the FSQ, the SSS and the Single Fatigue Question are not surprising given that the participants volunteered to complete an online questionnaire, an activity that requires some mental and physical energy.

Table 2. Participant characteristics.
Number 214
Age, mean ± SD (range) 32.8 ± 10.0 (19-70)
Gender, n (%)
  Female 97 (45.3%)
  Male 117 (54.7%)
Education status, n (%)
  High school graduate or below 65 (30.4%)
  Trade/technical/vocational training 7 (3.3%)
  Associate degree 13 (6.0%)
  Bachelor's degree 91 (42.5%)
  Master's degree 31 (14.5%)
  Professional degree (JD, MD, or other) 5 (2.3%)
  Doctorate degree (PhD, PsyD or other) 2 (0.9%)
Country of residence, n (%)
  United States 99 (46.3%)
  India 101 (47.2%)
  Other 14 (6.5%)
Table 3. Mean scores on the FSQ, the single fatigue question, and the SSS at time 1 and time 2.
FSQ
(0-16)
Single Fatigue Question
(-2 to 2)
SSS
(1-7)
Time 1 Time 2 Time 1-Time2 Time 1 Time 2 Time 1-Time 2 Time 1 Time 2 Time 1– Time 2
Mean 4.76 3.99 .77* -.09 -.33 -.23 2.46 2.25 -.22
SD 2.86 2.97 1.19 1.17 1.47 1.29
95% CI 4.38-5.14 3.59-4.39 .25-.07 -.48-.17 2.27-2.66 2.08-2.42
Median 5 4 0 0 2 2
Range 0-12 0-15 -2 – 2 -2 – 2 1-7 1-7
*t=-2.74, p=.006, Cohen’s d=-.27
Fig. (1). Frequency Distribution of FSQ scores at Time 1.

FSQ Item Analysis and Internal Consistency

Internal consistency of the FSQ was adequate, with a Chronbach coefficient α of .73 (Table 4). The majority of the variance in internal consistency came from item 3, the question that targets the energy level of fatigue. Internal consistency without this item rose to .80. The energy and alertness component of fatigue may be distinct from the weak and weary component.

Table 4. Chronbach's alpha of FSQ items as a measurement of internal consistency.
Α
All items .73
All items excluding item 1 .63
All items excluding item 2 .63
All items excluding item 3 .80
All items excluding item 4 .61
Inter-correlations Α
Q1 vs. Q2, Q3, Q4 .60
Q2 vs. Q1, Q3, Q4 .60
Q3 vs. Q1, Q2, Q4 .31
Q4 vs. Q1, Q2, Q3 .63
Fig. (2). Scatter plot of time 1 and time 2 Estimates of FSQ scores.

Test-Retest Reliability

The scatter plot of time 1 versus time 2 FSQ scores is illustrated in Fig (2). The correlation was positive and significant (r=.71, p=.001, Table 5), with the fitted regression line accounting for approximately 50% of the variance.

Table 5. Test-retest, inter measure, and measure-predictor correlations.
Fatigue State Questionnaire Single Fatigue Question Stamford Sleepiness Scale
Test-Retest .71*** .60*** .75***
Fatigue State Questionnaire Time 1 .59***
Time 2 .72***
Time 1 .50***
Time 2 .68***
Single Fatigue Question Time 1 .43***
Time 2 .52***
Sleep Changes .35*** .28*** .22***
Sleep Debt .30*** .22*** .15*
Circadian Point .10 .01 -.08
Health -.25*** -.17** -.11
+All correlations marked with a “*” are significantly different from zero: *p<.05, **p<.01, ***p<.001.
++The FSQ is a significantly better predictor of Sleep Changes (r=.35), Health (r=-.25) and Sleep Debt (r=.30) than the SSS (r=.22, z=2.49, p<.01; r=-.11, z=2.60, p<.01; and r=.15, z=2.82, p<.01) [42]. +++The correlation between the FSQ and the Single Fatigue Question is significantly greater than the correlation between the SSS and the Single Fatigue Question at both time 1 (r=.59 vs. r=.43 z=2.86, p<.01) and time 2 (r=.72 vs. r=.52, z=4.13, p<.001).

The time 1 and time 2 correlations for the Single Fatigue Question and the SSS are also shown in Table 5. As expected the test-retest correlation is lower for the Single Fatigue Question (r=.60) than for the FSQ (r=.71). This difference is statistically significant (z=-2.39, p<.05). The test-retest correlation for the SSS is comparable to that of the FSQ (z=-.99, p=.32).

The mean difference between the time 1 and time 2 FSQ scores was -.77. A repeated measures t-test yielded a t-score of -2.74 (p<.01, Cohen’s d=-.27) indicating that there was a significant 16% drop in fatigue scores after the 10-minute interval between time 1 and time 2. Perhaps the activities that the participants chose during the interval before the second test served to lessen their perceived fatigue. There was also a small but significant drop in scores on the Single Fatigue Question. The mean difference between time 1 and time 2 scores was -.23 (t=-2.05, p<.05, Cohen’s d=-.20) indicating that participants became more energetic at time 2. Sleepiness scores as measured by the SSS also dropped (-.22), but not significantly (t=-1.61, p=.11, Cohen’s d=-.16). The changes in perceived fatigue on the FSQ and the Single Fatigue Question and the lack of perceived changes on the SSS can be interpreted in a several different ways. It’s possible that perceived fatigue changes more rapidly than perceived sleepiness. It’s also possible that the SSS is not sensitive to perceived short-term (10 minutes) changes in sleepiness. Alternately, there could be no perceived changes in fatigue or sleepiness and the FSQ is less reliable than the SSS.

Convergent Validity of the FSQ with the SSS and the Single Fatigue Question

The scatter plot of FSQ scores versus SSS scores is illustrated in Fig. (3). The correlation was positive and significantly different from zero (r=.50, p<.001), with the fitted regression line accounting for approximately 25% of the variance. As expected, at time 1 and time 2, the FSQ correlated significantly better with the Single Fatigue Question than with the SSS (time 1, r=.59 vs. r=.43 and time 2, r= .72 vs. r=.52; Table 5).

The FSQ and the Single Fatigue Question where both designed to measure fatigue whereas the SSS was designed to measure sleepiness. The larger correlations between the FSQ and the Single Fatigue Question than between the SSS and the Single Fatigue Question are consistent with the idea that the FSQ measures a state that is unique from sleepiness. The differences in the correlations would likely have been larger if the SSS were a purer measure of sleepiness (i.e., one that didn’t contain descriptors referring to both sleepiness and fatigue).

Fig. (3). Scatter plot of FSQ scores and SSS scores.

Predictive Validity of the FSQ with Health and Prior Night’s Sleep

Thirteen percent of the participants reported feeling unwell with either an acute or chronic illness while taking the online survey (Table 6). Almost 33% said that their sleep time the night before was either 90 minutes more or 90 minutes less than their ideal sleep time (i.e., sleep change). In the vast majority of these cases, the change in sleep time was 90 minutes less than their ideal and was coded as sleep debt (28% of all participants). Forty percent of the participants were taking the survey during a circadian low (between 12 AM and 5 AM or between 1 PM and 3 PM).

Table 6. Percentage of subjects with illness, sleep changes, sleep debt, and participating during a circadian low.
Unhealthy, n (%) 28 (13.1%)
Sleep change, n (%) 70 (32.7%)
Sleep debt, n (%) 60 (28.0%)
Circadian low, n (%) 86 (40.2%)

The FSQ had incremental validity over the SSS in the binary measure of health (i.e. “Have you been feeling sick at all today?”; r=-.25, p<.001 vs. r=-.11, p=.11, respectively; Table 5). The difference was statistically significant (z=-2.30, p<.05) [42]. In addition, the FSQ was a significantly better predictor of sleep changes (r=.35 vs. r=.22; z=-2.20, p<.05) and sleep debt (r=.30 vs. r=.15; z=2.82, p<.01) than the SSS.

Predictive validity was also calculated for the Single Fatigue Question as a way to assess how well the FSQ performs over a single-item measure of fatigue. The FSQ was more highly correlated with health, sleep changes, and sleep debt than the Single Fatigue Question but the differences did not reach significance.

Fatigue and Prior Night’s Sleep, Health and Circadian Point

Participants who experienced either an increase or a decrease in their ideal sleep time the night before the survey had higher fatigue scores than participants who did not experience these sleep differences (Table 7). Specifically, the mean FSQ score (M=6.20) of participants with sleep changes was significantly higher than the mean FSQ score (M=4.06) of participants with the ideal amount of sleep (t=5.48, p<.001, Cohen’s d=.80). The mean FSQ score (M=6.10) of participants with a sleep debt (90 or more LESS sleep than ideal) was also significantly higher than the mean FSQ score (M=4.23) of participants with ideal sleep or more than 90 minutes more than ideal sleep (t=4.52, p<.001, Cohen’s d=.68). Participants who reported feeling unwell while taking the survey had significantly higher FSQ scores (M=6.61) than participants who reported feeling well (M=4.48; t=3.78, p<.001, Cohen’s d=.70).

Table 7. Mean scores on the FSQ* by health, sleep, and circadian groups.
Means T-Score
H0=μ=μ
Probability
Unhealthy (n=28)
6.61
Healthy (n=186)
4.48
3.78 p=0.001***
Sleep Changes (n=70)
6.20
Ideal Sleep (n=144)
4.06
5.48 p=0.001***
Sleep Debt (n=61)
6.10
Ideal or More Sleep (n=153)
4.23
4.52 p=0.001***
Circadian Low (n=85)
5.13
Circadian Mid or High (n=129)
4.52
1.53 p=.127
Circadian Low and Sleep Changes (n=29)
7.24
Circadian Mid or High and
Sleep Changes (n=41)
5.46
2.42 p=.018**
Circadian Low and Sleep Debt (n=25)
7.28
Circadian Mid or High and
Sleep Debt (n=36)
5.28
2.48 p=.016**
*FSQ Mean Scores at Time 1
**Statistically Significant at the .05 level
***Statistically Significant at the .001 level

Participants who took the survey during a circadian low point did not differ in fatigue from participants who took the survey during other circadian points. The mean FSQ score (M=5.13) of participants taking the survey during a circadian low was not significantly different from the mean FSQ score (M=4.52) of participants taking the survey during other times of day (t=1.53, p=.127, Cohen’s d=.21). However, fatigue levels did differ among participants who had experienced either a sleep change or a sleep debt the night prior to the survey AND were participating during a circadian low. The mean FSQ score (M=7.24) of participants working during a circadian low with sleep changes the night prior was significantly higher than the mean FSQ score (M=5.46) of participants working during a circadian mid or high point with sleep changes (t=2.42, p=.018, Cohen’s d=.59). There was also a significant difference between the mean FSQ score (M=7.28) of participants working during a circadian low with sleep debt the night prior and the mean FSQ score (M=5.28) of participants working during a circadian mid or high point with sleep debt (t=2.48, p=.016, Cohen’s d=.65).

Sleepiness and Prior Night’s Sleep, Health and Circadian Point

Like fatigue scores, sleepiness scores differed with the prior night’s sleep time (Table 8). Participants with either a sleep change or a sleep debt had higher mean sleepiness scores than participants with ideal or more than ideal sleep the night before. The mean SSS score (M=2.93) of participants with a sleep change was significantly higher than the mean SSS score (M=2.24) of participants with the ideal amount of sleep (t=3.30, p=0.001, Cohen’s d=.36). The mean SSS score (M=2.80) of participants with a sleep debt (90 or more less sleep than ideal) was also significantly higher than the mean SSS score (M=2.33) of participants with ideal sleep or more than 90 minutes more than ideal sleep (t=2.16, p=.032, Cohen’s d=.33).

Table 8. Mean scores on the SSS* by health, sleep, and circadian groups.
Means T-Score
H0=μ=μ
Probability
Unhealthy (n=28)
2.89
Healthy (n=186)
2.40
1.66 p=.097
Sleep Changes (n=70)
2.93
Ideal Sleep (n=144)
2.24
3.30 p=0.001***
Sleep Debt (n=61)
2.80
Ideal or More Sleep (n=153)
2.33
2.16 p=.032**
Circadian Low (n=85)
2.61
Circadian Mid or High (n=129)
2.36
1.21 p=.230
Circadian Low and Sleep Changes (n=29)
3.35
Circadian Mid or High and Sleep Changes (n=41)
2.63
1.96 p=.054
Circadian Low and
Sleep Debt (n=25)
3.24
Circadian Mid or High and Sleep Debt (n=36)
2.50
1.89 p=.064
*SSS Mean Scores at Time 1 **Statistically Significant at the .05 level ***Statistically Significant at the .001 level

Also like fatigue scores, sleepiness scores did not differ with circadian point alone. The mean SSS score (M=2.61) of participants taking the survey at a circadian low point did not differ significantly from the mean SSS score (M=2.36) of participants taking the survey at other times of the day (t=1.21, p=.230, Cohen’s d=.17). Unlike fatigue, sleepiness scores did not differ with circadian point in participants with sleep debt or sleep changes. The mean SSS score (M=3.35) of participants working during a circadian low with sleep changes the night before was not significantly from the mean SSS score (M=2.63) of participants working during a circadian mid or high point with sleep changes (t=1.96, p=.054, Cohen’s d=.46). Similarly, the mean SSS score (M=3.24) of circadian low participants with sleep debt did not differ significantly from the mean SSS score (M=2.5) of circadian mid or high participants with sleep debt (t=1.89, p=.064, Cohen’s d=.47).

Also unlike fatigue scores, sleepiness scores did not differ with participants’ health. The mean SSS score (M=2.89) of participants who reported feeling unwell did not differ significantly from the mean SSS score (M=2.40) of participants who reported feeling well while taking the survey (t=1.66, p=.097, Cohen’s d=.34).

DISCUSSION

The FSQ is a brief, self-report measurement of state-level fatigue. It showed good reliability as indicated by test-retest correlation. FSQ scores at time 1 and time 2 were expected to be related but not identical. In fact, the FSQ was expressly designed to measure moment-to-moment changes in the subjective fatigue state. The slight decrease in FSQ scores between test and re-test could not be explained in this non-experimental study. Future studies that manipulate the participants’ activities during the test-retest interval would help to differentiate among the possible hypotheses as to why the fatigue scores on both the FSQ and the Single Fatigue Question decreased between time 1 and time 2.

The FSQ also showed good internal consistency and external validity. The external validity of the FSQ was supported by its correlations with predictive measures of fatigue including health and prior night’s sleep. The FSQ correlated significantly better than the SSS with a binary measure of health, as well as with sleep debt and sleep changes.

The validity of the FSQ as a measure of fatigue that is conceptually distinct from sleepiness was supported by the following findings. First, the FSQ correlated better with a single common sense question about fatigue than with the SSS. This finding, along with the finding that scores on the FSQ and the SSS share only 25% of their variance, indicates that the FSQ is measuring a state that is related to sleepiness but still distinct. The nature of the relationship between fatigue and sleepiness could be examined by asking whether subjects who initially present as very sleep get more fatigued as time passes than subjects who initially present as not sleepy. Also, one could examine whether subjects who initially present as very fatigued become sleepier as time passes than subjects who initially present as not fatigued. The current study did not allow this type of analysis because of the relatively low fatigue and sleepiness levels of the participants.

Also in support of FSQ validity was the finding that fatigue scores of unhealthy participants were higher than those of healthy participants. This difference between healthy and unhealthy participants was not observed for sleepiness. The relationship between fatigue and health is well documented. The FSQ’s ability to detect differences in health over the SSS suggests that the two tests measure states that have distinct features. For instance, fatigue may be more sensitive to measurement at the low end of the spectrum than sleepiness. Also notable is that both sleep debt and sleep changes were found to accentuate circadian effects on fatigue, but not on sleepiness. Although sleep debt has been identified as a causative factor of both fatigue and sleepiness, over sleeping and disentrainment of rest/activity cycles are more clearly linked to fatigue [28]. The relationship between fatigue, circadian point and sleep change is not surprising given that sleep change was operationalized in this study to include oversleeping and under sleeping by 90 minutes or more. The lack of relationship between sleepiness, circadian point and sleep debt in this study may be due to a floor effect on the sleepiness measure. Based on the current models of sleepiness, it is likely that relationship between these variables would be evident in a study with sleepier subjects. The relative sensitivity of the FSQ even with mildly fatigued subjects suggests it will be a useful tool for delineating differences in the currently similar causative models of fatigue and sleepiness.

The FSQ's correlation to sleep changes, sleep debt, and health, our measures of predictive validity, were higher than for the Single Fatigue Question, but the results were not statistically significant. This suggests that single item fatigue scales may be valid for some uses. However, the FSQ did have a significantly higher test-retest reliability than the Single Fatigue Question. Further research is needed to determine if the FSQ has more predictive power than a single question for other variables thought to be related to fatigue.

Given the encouraging preliminary findings on the FSQ’s reliability and validity, the measure may, with additional testing, prove to be superior to other measures of state-level fatigue. The 4-item FSQ is shorter than the 18-item VAS and the 10-item Samn-Perelli. Unlike the VAS, the response mode on the FSQ allows for between-subject comparisons without the use of z scores. The FSQ also focuses only on the fatigue state rather than on both the fatigue and sleepiness states as in the SSS and the KSS.

This study was limited by a small sample size, only two within-subject FSQ measurements, and a limited number of comparative and predictive measures. Additional research with larger samples should be conducted to assess how the FSQ performs against other subjective measures of fatigue, such as the VAS and the Samn-Perelli. The FSQ should also be compared with objective measures of fatigue such as the Multiple Sleep Latency Test [45] and the Psychomotor Vigilance Task [46]. These studies should take within-subject FSQ measurements across several hours to better assess the FSQ's potential as an easy method of tracking fluctuating levels of daytime fatigue across the circadian cycle. The tracking of FSQ scores against the circadian cycle should ideally be done with subjects drawn from the same circadian typology. Previous studies have found variation in the timing of high and low circadian points in countries with different weather patterns [47]. Within-subject comparisons of FSQ scores and SSS scores across a longer time frame may shed light on the relationship between fatigue and sleepiness states.

In addition to health, the FSQ should be related to other internal variables thought to be associated with fatigue, such as chronotype, age, and mood. The relationship between FSQ scores and external variables such as exercise, caffeine consumption, napping, meditation, and other interventions should also be explored. Finally, the FSQ should be tested for its suitability in clinical populations with sleep and other disorders

APPENDIX 1

INITIAL QUESTION POOL FOR THE FSQ

How tired do you feel, right now?

How lacking in energy are you right now?

How much do you need to rest right now?

How alive and vital do you feel right now?

How slow and sluggish are you right now?

How wide-awake do you feel right now?

How energetic do you feel right now?

How drowsy or sleepy do you feel right now?

How awake do you feel right now?

How interested would you be in sleeping right now?

How mentally alert do you feel right now?

How exhausted do you feel right now?

How tired does your body feel right now

How tired does your mind feel right now?

How well can you concentrate right now?

How close do you feel to your typical level of strength right now?

How calm and peaceful do you feel right now?

How relaxed do you feel right now?

How easy would it be for you to fall asleep right now?

How easy would it be for you to do something very physically active right now?

How easy would it be for you to concentrate on a puzzle right now?

How drained do you feel right now?

How wiped out do you feel right now?

How worn out do you feel right now?

How rested do you feel right now?

How mentally sharp do you feel right now?

How refreshed do you feel right now?

APPENDIX 2

FATIGUE STATE QUESTIONNAIRE (FSQ)

Instructions: Please answer the following questions honestly and accurately about how you're feeling right now, in this present moment.

1. How tired does your body feel right now?

  1. Not at all
  2. A little
  3. Moderately
  4. Very
  5. Extremely

2. How tired does your mind feel right now?

  1. Not at all
  2. A little
  3. Moderately
  4. Very
  5. Extremely

3. How awake do you feel right now?

  1. Not at all
  2. A little
  3. Moderately
  4. Very
  5. Extremely

4. How slow and sluggish are you right now?

  1. Not at all
  2. A little
  3. Moderately
  4. Very
  5. Extremely

Scoring: Questions 1, 2, and 4 are scored with the value given (e.g. a response of 4 “Extremely” is given a score of 4). Question 3 is reverse scored (e.g., a response of 4 “Extremely” is given a 0). The total score is generated by summing the scores on all 4 questions. The total score can range from 0 to 16.

DISCLOSURE STATEMENT

The study was supported by Gimbel Technololgies, LLC. Gimbel Technologies is a privately owned company whose mission is to create free on-line research tools for social scientists; to create free on-line applications for optimizing psychological health for the general population; and to contribute to the academic literature on psychological health.

The Fatigue State Questionnaire discussed in this paper is available online free of charge at www.guidedtrack.com/programs/axqxghh/run.

CONFLICT OF INTEREST

The authors confirm that this article content has no conflict of interest.

ACKNOWLEDGEMENTS

Declared None.

REFERENCES

[1] Berg J. An overview of sleepiness aspects reflected in a balance scale model. Open Sleep J 2009; 2: 33-42.
[2] Eisenstein M. Chronobiology: stepping out of time. Nature 2013; 497(7450): S10-2.
[3] Hossain JL, Ahmad P, Reinish LW, Kayumov L, Hossain NK, Shapiro CM. Subjective fatigue and subjective sleepiness: two independent consequences of sleep disorders? J Sleep Res 2005; 14(3): 245-53.
[4] Shen J, Barbera J, Shapiro CM. Distinguishing sleepiness and fatigue: focus on definition and measurement. Sleep Med Rev 2006; 10(1): 63-76.
[5] Pigeon WR, Sateia MJ, Ferguson RJ. Distinguishing between excessive daytime sleepiness and fatigue: toward improved detection and treatment. J Psychosom Res 2003; 54(1): 61-9.
[6] Millar M. Measuring fatigue. In: International Civil Aviation Organization/International Air Transport Association/International Federation of Airline Pilots’ Association. Bangkok: Asia-Pacific Fatigue Risk Management Seminar 2012.
[7] Dement WC. The study of human sleep: a historical perspective. Thorax 1998; 53(Suppl. 3): S2-7.
[8] Smolensky M, Lamberg L. The body clock guide to better health. New York: H. Holt 2000.
[9] Thorpy M. Sleepiness: Causes, consequences, and treatment. Cambridge: Cambridge University Press 2011.
[10] Schmidt C, Collette F, Cajochen C, Peigneux P. A time to think: circadian rhythms in human cognition. Cogn Neuropsychol 2007; 24(7): 755-89.
[11] Shahid A. STOP, THAT and one hundred other sleep scales. New York, NY: Springer 2012.
[12] De Valck E, Cluydts R. Sleepiness as a state-trait phenomenon, comprising both a sleep drive and a wake drive. Med Hypotheses 2003; 60(4): 509-12.
[13] Johns M. What is excessive daytime sleepiness? In: Sleepiness: Causes, Consequences, and Treatment. USA: Nova Science Publishers 2009; pp. 1-37.
[14] Rupp T. Concepts of fatigue, sleepiness, and alertness. In: Encyclopedia of Sleep. Netherlands: Elsevier 2013; pp. 24-6.
[15] Radovic S, Malmgren H. Fatigue and fatigability. In: Neural Correlates of Conciousness: Emperical and conceptural issues. Berman, Germany 1998.
[16] Akerstedt T. Using mathematical models to predict sleepiness. In: Gararono S, Ed. Sleepiness and Human Impact Assessment Springer-Verlag Italia. 2014; pp. 63-9.
[17] Samn SW, Perelli P. Estimating aircrew fatigue: A technique with application to airlift operations. Brooks AFN, TX: USAF School of Aerospace Medicine 1982. Report No. 82-21, ADA 125319
[18] International Civil Aviation Organization [internet]. Bangkok: ICAO; April 1. Defining, measuring, and predicting fatigue by Greg Belenky Available from: www.icao.int/safety/fatiguemanagement/FRMS2011/PRESENTATION DOCUMENTS/3B - Greg Belenky.pdf 2011. [cited 2015 Jan 8].
[19] Borbély AA. A two process model of sleep regulation. Hum Neurobiol 1982; 1(3): 195-204.
[20] Curcio G, Casagrande M, Bertini M. Sleepiness: evaluating and quantifying methods. Int J Psychophysiol 2001; 41(3): 251-63.
[21] Horne J. Sleepiness as a need for sleep: when is enough, enough? Neurosci Biobehav Rev 2010; 34(1): 108-18.
[22] Mairesse O, De Valck E, Quanten S, et al. Sleepiness phenomics: modeling individual differences in subjective sleepiness profiles. Int J Psychophysiol 2014; 93(1): 150-61.
[23] Mathis J, Hess CW. Sleepiness and vigilance tests. Swiss Med Wkly 2009; 139(15-16): 214-9.
[24] Taillard J, Philip P, Claustrat B, et al. Time course of neurobehavioral alertness during extended wakefulness in morning- and evening-type healthy sleepers. Chronobiol Int 2011; 28(6): 520-7.
[25] National Sleep Foundation. Sleep research & education [internet]. Arlington, VA: NSF; Available from: http://sleepfoundation.org/ 2014. [Jan 1 cited 2014 Dec 5].
[26] Theorell-Haglöw J, Lindberg E, Janson C. What are the important risk factors for daytime sleepiness and fatigue in women? Sleep 2006; 29(6): 751-7.
[27] Thiffault P, Bergeron J. Monotony of road environment and driver fatigue: a simulator study. Accid Anal Prev 2003; 35(3): 381-91.
[28] Ancoli-Israel S, Moore PJ, Jones V. The relationship between fatigue and sleep in cancer patients: a review. Eur J Cancer Care (Engl) 2001; 10(4): 245-55.
[29] Wright KP Jr, Hull JT, Czeisler CA. Relationship between alertness, performance, and body temperature in humans. Am J Physiol Regul Integr Comp Physiol 2002; 283(6): R1370-7.
[30] Marcora SM, Staiano W, Manning V. Mental fatigue impairs physical performance in humans. J Appl Physiol 2009; 106(3): 857-64.
[31] Johns MW. A new method for measuring daytime sleepiness: the Epworth sleepiness scale. Sleep 1991; 14(6): 540-5.
[32] Chalder T, Berelowitz G, Pawlikowska T, et al. Development of a fatigue scale. J Psychosom Res 1993; 37(2): 147-53.
[33] Mendoza TR, Wang XS, Cleeland CS, et al. The rapid assessment of fatigue severity in cancer patients: use of the Brief Fatigue Inventory. Cancer 1999; 85(5): 1186-96.
[34] Pilcher JJ, Pury CL, Muth ER. Assessing subjective daytime sleepiness: an internal state versus behavior approach. Behav Med 2003; 29(2): 60-7.
[35] Bakotic M, Radosevic-Vidacek B. State-trait arousal and daytime sleepiness after sleep restriction. Int J Psychophysiol 2013; 88(2): 164-70.
[36] Hoddes E, Zarcone V, Smythe H, Phillips R, Dement WC. Quantification of sleepiness: a new approach. Psychophysiology 1973; 10(4): 431-6.
[37] Kaida K, Takahashi M, Åkerstedt T, et al. Validation of the Karolinska sleepiness scale against performance and EEG variables. Clin Neurophysiol 2006; 117(7): 1574-81.
[38] MacLean AW, Fekken GC, Saskin P, Knowles JB. Psychometric evaluation of the stanford sleepiness scale. J Sleep Res 1992; 1(1): 35-9.
[39] Lee KA, Hicks G, Nino-Murcia G. Validity and reliability of a scale to assess fatigue. Psychiatry Res 1991; 36(3): 291-8.
[40] Maldonado CC, Bentley AJ, Mitchell D. A pictorial sleepiness scale based on cartoon faces. Sleep 2004; 27(3): 541-8.
[41] Loo R. A caveat on using single-item versus multiple-item scales. J Manage Psycho 2002; 17(1): 68-75.
[42] Pearson PG, Byars G. The development and validation of a checklist measuring subjective fatigue. 1956.
[43] Ferrara M, De Gennaro L. How much sleep do we need? Sleep Med Rev 2001; 5(2): 155-79.
[44] Dunn OJ, Clark VA. Correlation coefficients measured on the same individuals. J Am Stat Assoc 1969; 64: 366-77.
[45] Carskadon MA, Dement WC. The multiple sleep latency test: what does it measure? Sleep 1982; 5(2)(Suppl. 2): S67-72.
[46] Dinges DF, Powell JW. Microcomputer analysis of performance on a portable, simple visual RT task during sustained operations. Behav Res Methods Instrum Comput 1985; 17: 652-5.
[47] Randler C, Prokop P, Sahu S, Haldar P. Cross-cultural comparison of seven morningness and sleep-wake measures from Germany, India and Slovakia. Int J Psychol 2015; 50(4): 279-87.