LETTER
Quantitative Evaluation of Work Efficiency and Eye Strain for Remote Control Construction Equipment Using 2D/3D Displays
Yoshihiro Sato1, *, Shohei Yamaguchi1, Shota Funaki1, Atsutoshi Kurihara1, Yuki Kumagai1, Satoru Nakamura2, Takao Ueno2, Yue Bao1
Article Information
Identifiers and Pagination:
Year: 2020Volume: 14
First Page: 133
Last Page: 138
Publisher ID: TOBCTJ-14-133
DOI: 10.2174/1874836802014010133
Article History:
Received Date: 02/02/2020Revision Received Date: 06/03/2020
Acceptance Date: 05/05/2020
Electronic publication date: 29/06/2020
Collection year: 2020
open-access license: This is an open access article distributed under the terms of the Creative Commons Attribution 4.0 International Public License (CC-BY 4.0), a copy of which is available at: (https://creativecommons.org/licenses/by/4.0/legalcode). This license permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Abstract
Remotely operated construction machines are used in cases where the operators are in danger, such as on steep slopes or contaminated sites. However, remote operation differs from what operators perceive during hands-on machine operation. Various studies have focused on improving work efficiency by employing remote control operation to reduce operator workload. In these studies, questionnaires were generally employed to evaluate the operator workload. However, the results obtained from the questionnaires varied depending on the physical conditions and the mood of the person on that day. It was therefore concluded that an accurate evaluation cannot be performed based on this method. Hence, in this study, the eye strain of machine operators was measured using an Auto Refractor/Keratometer. In particular, the ciliary muscle activities were measured before and after operating three display systems used for remote control of construction equipment. A quantitative evaluation was then conducted based on the eye strain data. The 2D display system exhibited low work accuracy and efficiency and resulted in significant eye strain. Although the 3D display system that required glasses exhibited high accuracy and efficiency, it resulted in significant eye strain. The 3D display system that did not require glasses demonstrated high accuracy and lower eye strain. From the results presented above, it was confirmed that the autostereoscopic 3D system is suitable for operators.
1. INTRODUCTION
Remotely operated construction equipment has recently been used in locations where operators may be in danger, such as on steep slopes or contaminated sites [1]. The most prevalent method of remote control involves the use of a Two-Dimensional (2D) display. The operator performs a remote operation while checking an image on a display from a camera installed in the cockpit of the machine. However, remote operation differs from what an operator perceives during the hands-on operation of the machine. Hence, operation accuracy and work efficiency are reduced, and the operator's strain and work time are increased by focusing on the 2D display to ensure accuracy [2].
Various studies have been conducted to improve the work efficiency of remote control operation and decrease operator workload. To address this issue, a remote control method was proposed that incorporates Three-Dimensional (3D) displays; some require [3] and some do not require the user to wear active shutter glasses [4]. 3D displays provide a sense of depth. However, 3D displays with active shutter glasses require the machine operator to wear glasses, which can be cumbersome. The operator can experience the 3D effect by opening and closing the active shutter. However, significant ocular fatigue and discomfort have been reported, in addition to difficulties due to extended use. For 3D displays that do not require glasses, the ease of use is increased. However, in order to obtain an accurate view of the 3D display, the position and distance of the observer are required to be within pre-determined ranges. When the ranges are exceeded (crosstalk), there is a reversal of the sense of depth provided by the 3D display [5]. Crosstalk can occur during the operation of heavy equipment and impact work efficiency and eye fatigue.
In the evaluations of display systems, work efficiency and eye fatigue are critical factors, and eye fatigue has typically been evaluated by providing questionnaires to the operator [6]. Questionnaire results vary depending on the physical condition and mood of the operator at that time; answers vary depending on the discretion of the respondent. Therefore, the comparative evaluation of each display system cannot be accurately and objectively carried out. In addition, eye fatigue has been reported to result in significant symptoms in accordance with the amount of work, which is not easily alleviated by resting [7]. Hence, a system that effectively suppresses operator eye fatigue is required.
In this study, the behavior of the ciliary muscles of machine operators was measured before and after the operation of three display systems used for the remote operation of construction equipment, and the eye strains were quantified. The aim of this study was to evaluate each display system based on this data.
2. MATERIALS AND METHODS
2.1. Quantification Method for Eyestrain
Work using displays is referred to as Video Display Terminal (VDT) work. The causes of eyestrain due to VDT work can be divided into visual factors and external environmental factors. The visual factors are due to diseases of the eye; and the external environmental factors include focus-adjustment impediments and convergence insufficiencies, which are referred to as near reactions [8]. These near reactions are caused by repetitive gazing at a close object such as a display, in addition to excessive focus adjustment on objects at varying distances. Focus adjustment is performed by the ciliary muscle (ciliary body) within the eye [9]. However, it has been reported that when the eyes are fatigued, ciliary muscle activity is reduced, thus resulting in micro-convulsions when focusing on a close-positioned stationary object [10]. The fine spasms of the ciliary muscle slightly change the shape of the ocular lens. Therefore, micro-convulsions have an influence on focus, and the focusing operation of the eye can be measured using an Auto Refractor/Keratometer [11]. By measuring the degree of focus adjustment with respect to time and conducting a Fourier analysis of the variation in focus adjustment, changes in the activity of the ciliary muscle can be extracted as a frequency signal. Moreover, by extracting only the high-frequency components, fine cramps of the ciliary muscle can be determined [10]. The fine cramps of the ciliary muscle are referred to as accommodation cramps. Based on this measurement method, these cramps can be quantified, and the eyestrain can be quantitatively evaluated.
2.2. 2D/3D Display System
In recent years, a glasses-type 3D display, a naked eye–type 3D display, and Virtual Reality (VR) display system have been proposed as 3D display systems. This study evaluates a remote-control system that can be used at construction sites. When operating machines for construction work, surrounding safety cannot be ensured solely using images from the cockpit; it is often necessary to check the surroundings as well. Therefore, display systems that cannot capture the surroundings, such as VR, are excluded. Further, this study employed a 2D display system as well as 3D display systems with and without glasses. Details of each display system are presented below.
2.2.1. 2D Display System
A remote construction system manufactured by Nishio Rent All Co. Ltd. was used as the 2D display system [12]. This system displays images obtained from a 2D camera installed above the cockpit of a machine on a viewing screen tied to a pre-fabricated controller. The video footage from the camera is transmitted and displayed on the viewing screen within 10 ms. Fig. (1) presents an actual use.
 |
Fig. (1). 2D display system: Use case. |
2.2.2. 3D Display System (Glasses Required)
The 3D display system that requires glasses was the NVidia 3D Vision [13] with active shutter glasses [14]. The images were obtained from a stereo camera installed above the cockpit of a construction machine. This system shows the images on a display installed in a prefabricated house at a distance. The operator can wear the active shutter glasses and observe the displayed images in a stereoscopic view. In this system, left- and right-parallax images are alternately displayed frame-by-frame on a display that supports 120 fps (frames per second), and the left and right shutters of the glasses are alternately opened and closed in accordance with the images, thus providing 3D images. The video footage from the camera is transmitted and displayed within 10 ms. Fig. (2) shows an image of the stereo camera used, and Fig. (3) presents an image of a use case.
 |
Fig. (2). ZED (Stereo camera). |
2.2.3. 3D Display System (No Glasses Required)
A 3D display system with a lenticular lens was employed as the 3D display system with no glasses required [15]. This system displays images obtained from a stereo camera installed in the same manner as the glasses-type 3D display system with a lenticular lens installed in a prefabricated house at a distance. With the lenticular lens method, several long vertical lenses are arranged and positioned at the front of the display, and the pixels seen through each lens differ depending on the viewpoint; thus, different images are presented to the right and left eyes with respect to the interocular distance. Therefore, the resolution of the display is decreased by an amount that corresponds to the display of the parallax image. Moreover, there is an optimum observation distance based on the angle and interval of the lens and the distance between the lens and the display. The optimal observation distance of the display was 3 m. The video footage is transmitted from the camera within 10 ms, and the parallax image is generated and displayed. Fig. (4) shows an image of the display employed, and Fig. (5) presents a use case.
 |
Fig. (3). 3D display system (glasses required): Use case. |
 |
Fig. (4). 3D display system (no glasses required). |
 |
Fig. (5). 3D display system (no glasses required): Use case. |
3. EXPERIMENTAL METHOD
3.1. Remote Work Evaluation
The methods for evaluating eyestrain and work efficiency were determined with reference to the model task reported previously [16]. For the evaluation of the high-precision operation of the machine, a task was carried out in which drums with a framework several centimeters wider than the bucket of the machine were sequentially moved to four designated positions. When setting the drums at the designated positions, the bucket was removed from the drum and then set on the ground to the left or right side of the drum. Thus, the bucket was to be passed through the drum frame at each instance, which required a precise operation. In addition, four square mats with dimensions of 50 cm × 50 cm were placed in designated positions denoted as 1-4; and the configuration accuracy (work accuracy) was calculated. The drum was set on the first mat as the initial position, and then moved in the order of Mats 2 to 3, and then to 4. After placing the drum on Mat 4, the drum was placed on Mat 1. One turn is defined as the time when the bucket on the ground starts to move to the time when the drum is placed on Mat 1 and the bucket remains stable on the ground. Remote operation was carried out for 30 min, and the work efficiency was evaluated based on the maximum number of cycles that could be carried out in 30 min. The eye strain was measured before and after the task, and the difference was evaluated as the degree of strain. Fig. (6) shows the work site.
 |
Fig. (6). Work site. |
3.2. Evaluation Method for Eye Strain
The operator’s eyes are evaluated following approval for an eye examination. The Auto Refractor/Keratometer for the eye strain measurement was based on Nidek ARK-1 software. In addition, using the Nidek AA-2 software, the focal length could be changed in seven increments as follows: ∞, 2.0, 1.0, 0.67, 0.5, 0.4, and 0.33 m; and the measurement target for each focal length was the focusing action of the machine operator for 12 s. Only the high-frequency components of the convulsions measured for 12 s were extracted by Fourier analysis, and the frequency of occurrence of the accommodation cramps was calculated. When the frequency of the accommodation cramps was high, the eyestrain was considered high; and when the frequency of the accommodation cramps was low, there was no eyestrain. The frequency of occurrence is referred to as the High-Frequency Component (HFC) [17]. The eye fatigue was evaluated by calculating the difference in HFCs before and after work for each focal length with respect to the three display systems.
4. RESULTS AND DISCUSSION
In this study, two licensed machine operators cooperated as test subjects. Subject A does not operate construction work machines regularly. Subject B regularly operates construction equipment. Subject B has been operating construction equipment for 17 years. Subjects A and B are both men in their forties.
The results of work accuracy and efficiency are shown in Table 1 and 2. Table 1 presents the number of turns, average time, and placement error of Subject A in each display system within 30 min. Table 2 presents the results of the work accuracy and efficiency of Subject B. Fig. (7) shows the eye strain of Subject A for each display system, and Fig. (6) shows those of Subject B.
| - | 2D Display System | 3 Display System | 3D Display System |
|---|---|---|---|
| - | - | (Glasses Required) | (No Glasses Required) |
| Number of turns | 3 | 6 | 4 |
| Average lap | 11:06.76 s | 05:04.31 s | 07:41.33 s |
| Placement error | 12.30 cm | 8.45 cm | 5.87 cm |
| - | 2D Display System | 3D Display System | 3d Display System |
|---|---|---|---|
| - | - | (Glasses Required) | (No Glasses Required) |
| Number of turns | 5 | 4 | 3 |
| Average lap | 06:16.61 s | 08:57.71 s | 09:58.64 s |
| Placement error | 4.69 cm | 3.69 cm | 1.92 cm |
| - | - | - | - |
 |
Fig. (7). Comparison result of eye strain of Subject A. |
 |
Fig. (8). Comparison result of eye strain of Subject B. |
For Subject A (Table 1), the 2D display system exhibited the lowest efficiency, and the accuracy was lower. The 3D display system that requires glasses was found to be more efficient. However, the most accurate system was the 3D display system that does not require glasses. Subject A does not operate construction equipment on a daily basis, a long time has passed between his last operation of equipment and this experiment, and he has little experience in operating construction equipment. Based on this information, it appears that placement error was reduced as familiarity with the machines increased by conducting an experiment using three display systems. The number of turns and average lap improved as the operator got used to operating the equipment. However, the naked-eye 3D display has an observation distance of 3 m, making it difficult to observe details; it takes more time to grasp the details; and it takes more time than the glasses-type 3D method. Hence, it is suggested that work efficiency can be improved by developing an autostereoscopic 3D display with a higher resolution in the future. Based on Fig. (7), for the 3D display that requires glasses, the change in fatigue is minimal (between 0.67 m and 0.4 m), and the HFC amplitudes increased rapidly at other focal lengths. Due to operation within the range of 0.4-0.67 m, a near-sighted reaction occurred, and regulatory convulsions increased. The shutters opened and closed, given that the glasses system is based on an active shutter system. Frequent focus adjustments to the display and shutter due to the shutter opening and closing are one of the major causes of eye strain. The system in which there were no major changes in eye strain was the 3D display system that does not require glasses. In this experiment, given that a 3D display with an observation distance of 3 m was used, no differences in the angle of convergence were observed, and the eye fatigue was small.
Based on the results of Subject B (Table 2), the 3D display system that does not require glasses exhibited the lowest efficiency, and the 2D display system exhibited the lowest accuracy. The 3D display system that does not require glasses exhibited the highest accuracy. Subject B controls a construction machine as a job and has experience in remote control operation on a 2D display system. Therefore, with regard to the 2D display system, the average lap time is almost the same as that in which Subject A has become accustomed to the operation. Placement error is smaller than that of Subject A. However, unlike Subject A, the average lap time of the glasses-based 3D system and the naked-eye 3D display system increased. Since Subject B is experienced, there was no change in time due to getting used to equipment operation. Possible causes for the longer times could be an aspect of both 3D displays which allows fine adjustments before and after the placement with a sense of depth. Therefore, the work time increased compared with the 2D display system in which front and rear adjustments were not performed. Therefore, the placement error is lower for the 3D systems vs. the 2D system. In addition, with respect to the working efficiency of the 3D display system that does not require glasses, the resolution is significantly decreased by the displayed parallax images when compared with the 2D display and 3D display that requires glasses; thus, a significant amount of time was required to observe a detailed portion. This issue can be resolved by increasing the resolution of the 3D display that does not require glasses. As shown in Fig. (8), with the 3D system that requires glasses, with respect to Subject A, no changes in fatigue were observed within the range of 1–0.5 m. However, for other observation distances, the HFC amplitudes increased abruptly, which were influenced by the active shutter method. The 3D display system that does not require glasses resulted in overall lower eye fatigue than the 2D display system.
Based on the results above, the 2D display system exhibited lower work accuracy than the 3D display systems. In addition, the 3D display system that required glasses was confirmed to cause significant eye strain. The working efficiency of the 3D display systems decreased due to the decrease in resolution, based on their structure. The 3D system that does not require glasses was confirmed to exhibit the highest accuracy and the lowest eye strain.
CONCLUSION
For remotely operated construction equipment, various systems have been proposed to improve work efficiency and reduce operator workload. Previous studies employed a questionnaire to determine the difference in operator workload. In this study, the behavior of ciliary muscles was measured before and after operating three types of display systems for remotely controlled construction work; eye strains were quantified and compared. The aim of this study was to evaluate each display system, and the following results were obtained.
(1) The 2D display system showed low work accuracy and efficiency and resulted in significant eye strain.
(2) Although the 3D display system that requires glasses exhibited high accuracy and high efficiency, it resulted in significant eye strain.
(3) The 3D display system that does not require glasses demonstrated a high accuracy and lower eye strain.
From the results presented above, the autostereoscopic 3D system is a suitable system for operators. The objective of future work is the further improvement of the resolution of 3D display systems that do not require glasses, thereby eliminating the need for an optimum viewing distance.
CONSENT FOR PUBLICATION
Not applicable.
AVAILABILITY OF DATA AND MATERIALS
Not applicable.
FUNDING
This study was supported by Tokyu Construction Co., Ltd. Grant number: 18TC09.
CONFLICT OF INTEREST
The author declares no conflict of interest, financial or otherwise.
ACKNOWLEDGEMENTS
This study was jointly conducted by Tokyo City University and Tokyu Construction Co., Ltd. We would like to thank Tokyu Construction Co., Ltd. for the co-operation of the construction machine operator as an experimental subject.
