Abstract— In pursuance of new assistive technologies, Brain-Computer Interface (BCI) has captured the attention of many researchers throughout the last years. This thriving new method converts brain activity to proper output-commands, which can then be used to conceive new communication pathways and recreate movement control for patients with impairments. The translated neuroelectric signals can be performed by several systems, such as mobile robots that are able to execute the user’s wishes and thoughts. Consequently, BCI embodies an advance towards the independence of the disabled, carrying great importance on the improvement of our modern society. However, in addition to therapeutic applications, BCI can be applied in other fields like security, games and entertainment.
The neuroelectric responses, accountable for the exploit of semi-humanoid robots and others operating systems, are commonly obtained through an electroencephalogram (EEG), which as the advantage of being a non-invasive method, as it doesn’t imply the penetration of the patient’s scalp. Further use of the acquisition outcomes requires a digital processing, responsible for eliminating noise and artefacts from the signal under analyses.
An effective form of using BCI is in a shared approach with an artificial controller, that not only adapts to the surrounding conditions as well as to the users conducts, therefore it is in a continuous learning process. Additionally, the interaction between man and machine is safer and less exhaustive for the user.
Index Terms — assistive technologies, brain–computer interface (BCI), ECoG, electroencephalograph (EEG), event-related potentials (ERPs), humanoid robots, magnetoencephalography (MEG), motor imagery, P300, shared control robots, robotic wheelchair, visual evoked potentials (VEPs).
I. INTRODUCTION
For the past few decades there has been an interest and need for the development of novel interface techniques. One of these technologies is the brain-computer interface (BCI), which consists in creating a connection amid humans and computers without physical contact, that is between our brain activity and a computer output signal [1]. Therefore, it has the potential to control machines and devices to stably produce and carry out simple or complex actions with only the power of our minds, which was considered as science fiction in the middle of the past century.
Brain signal acquisition can be obtained through invasive or non-invasive techniques, such as subdural or depth surgically implanted electrodes (cortical stimulation) and electroencephalograms (EEGs) respectively. By assuring the patient physical integrity, EEG based BCI also constitutes a cost-effective and portable method, hence it is the most commonly used [2][3]. This neurotechnology can then be utilized to detect and translate EEG signals, that is neural responses created by the nervous cells of the brain, into instructions later performed by devices such as neuroprosthesis, robotic arms, humanoid walking robots, virtual robots and many other machineries as to exploit their movement and object manipulation [4]. However, due to the lower signal quality and poor spatial resolution in EEG that results from interferences and noise, BCI technology requires a signal processing between the stages of signal acquisition and interface employment [5].
One of the greatest difficulties to overcome in BCI is to achieve a high-speed and accurate output command, for the sake of producing a more satisfying and natural interaction between human and machine. Consequently, BCI can be based on visual evoked potential (VEPs) or on event related potentials (ERPs), being that the first method requires stimulation which only allows the user to convey his thoughts in a predetermined and limited time span, therefore it constitutes a synchronous transmission. On the other hand, ERP based BCI decodes and translates information whenever the patient decides to do so, thus it constitutes an asynchronous transmission of information that creates a more self-controlled system [6][7].
According to the World Health Organization, about fifteen percent of the world population suffers from a sort of impairment, including physical and/or mental disabilities [8]. Due to independence loss, these illnesses can result in severe limitations in the subject’s daily actions besides function loss, being that in more acute cases patients have a full motor disability that obliges them to a state of total paralyzes and reduced social interaction [9]. At the same time, there’s also the issue of the progressive aging of our society, which results in chronical diseases and impairments that in some cases demand complete surveillance (24 hours a day) by health care professionals [10]. Considering the negative effects of these illnesses and due to the urgency for new assistive technologies that improve the patient’s quality of life, this paper focuses on a specific application of BCI technology, specifically in BCI controlled robots for rehabilitation. Therefore, with subject training, EEG based BCI applications have the potential to restore injured and immobilized patients by conduction brain signals through the muscles, as well as to fortify the networks between different regions of the brain and the spinal cord [11].
II. WHAT IS A BCI?
A brain-computer interface (BCI) is a computer-based system that measures brain activity and converts it into a digital form that controls some external device, without the use of the spinal and peripheral motor system. Thus, the use of this term is limited to systems that can acquire and use signals produced by the central nervous system [12][13]. It’s a new output channel for the brain that restores communication for patients with severe motor disorders such as spinal cord injury, amyotrophic lateral sclerosis (ALS), brainstem stroke, among others, allowing them to act on the world using their thoughts instead of their muscles [14].
A BCI operation depends on the interactions of two microcontrollers: the user, whose intent produces specific brain signals, and the hardware and software integrated into the system, which converts those signals into an output that accomplishes the user’s desire. To be successful it must provide real-time feedback to the user and operate as a closed loop control system [15][16].
A. How Does it work?
There are three types of neurophysiologic signals that can be acquired and used to drive the BCI: electrophysiological, magnetic and the metabolic signals.
However, regardless the signal type, the BCI system consists of four elements: signal acquisition, feature extraction, feature translation, and device output, as can be observed in figure 1. To manage these elements there’s a system operating protocol [15][17].
B. Signal Acquisition
The first step is the acquisition of the brain signal by the electrodes, which can be in the frequency or time domain. There are four types of signals frequently used by BCI [12[16-19][21].
Visual evoked potentials (VEPs) are time domain signals and consists in evoked potentials induced by a visual stimulus. By its definition, between every two stimulations exists a resting period. On the other hand, when applied a high frequency stimulation, that is, a high repetition rate of the stimulus, they are entitled steady state VEPs (SSVEP).
P300 evoked potentials results from a request for the user to attend a task. It doesn’t need any type of training, but it’s affected by the distraction of the user and habituation to the stimulus, resulting in a decrease of signal’s amplitude. Its acquisition is done in the time domain.
Slow cortical potential (SCP) is generated by changes in the depolarization levels of several dendrites. Negative shifts represent the cortical activation while positive shifts represent its reduce activity. It belongs to the time domain signal and a long training procedure is demanded.
Sensorimotor rhythms are frequency domain signals and, depending on the frequency band, are designated either μ( 8-13 Hz) or β (13-30 Hz). They are generated spontaneously, and its signal amplitude can be modulated by the user, knowing that movement or its picturization reduces the amount of mu and beta rhythms activity whereas relaxation increases them. However, it requires training.
C. Feature extraction
The next step is the feature extraction, where the acquired signals are digitized and analyzed in order to distinguish the signal’s most important characteristics and represent them in a way that eases its interpretation by human or computer. In other words, this process isolates the features, such as time, frequency and phase, which encodes the user’s intent, using a filtering algorithm, and removes the corrupting information, as is the case of noise and artifacts. The noise regards external sources like the power-line 50 Hz frequency and radio frequencies, while artifacts are due to biological or internal sources, like, for example, eyes blink or the muscle noise.
To decode the desire of the user a direct measurement of the voltage difference between two electrodes, at a specific time, also known as a fundamental signal feature, is not enough. Unfortunately, a direct measure of the voltage difference between two electrodes in a specific time, better known as fundamental signal feature, is not enough to decode the user´s desired commands.
Usually, BCI uses features based on special, temporal and/or spectral analyses of brain signals. Besides, to be precise as possible the BCI extracts a feature vector, i.e., a set of features at a time. The attributes required by the set are [15][20]:
• Precise characterization of temporal, spatial and spectral characteristics as well as dynamics;
• Allows its manipulation by the user along with its association with other features;
• Stable correlation with the user’s intent;
• Consistent and trustworthy trackability.
There are several methods that can be used for feature extraction, being the choice of the method made according to the obtained signal. For time domain signals, the most used technique is the Independent Component Analysis (ICA), a computational method that divides mixed signals into its statistically independent components and has the disadvantage of having a highly complex algorithm. Regarding the frequency domain methods like Fast Fourier Transform (FFT) and Auto Regressive (AR) can be applied. The FFT has a good performance in linear random process and in stationary signals. As for the AR it is commonly used for non-stationary signals, obtaining a lower spectral loss as well as higher frequency resolution for the features extracted.
Some methods can extract both time and frequency domain feature like is the case of Wavelet Transform (WT), Wavelet Packet Decomposition (WPD) and Principal Component Analysis (PCA). The first is used for signal analysis with several window size and for extraction of features with B-spline parameters. Relatively to WPD this method is used for non-stationary signal analysis and low frequency wavelets decomposition. However, it requires higher computational time. [22]
D. Feature translation
The feature translation consists of converting the identified feature into device commands using an online algorithm, which must be dynamic and adaptable to the continuous changes in the user’s intent and/or performance [15][16][19].
E. Device commands
The external device is conducted by the commands created, resulting in outputs such as, for example, cursor movement on a computer screen, manipulation of a robotic arm or control of a wheelchair [15][16][19].
F. System protocol
Relatively to the system protocol, it defines the interactive operation of the BCI. It should discriminate the on/off mechanism, feedback parameters, such as response time, as well as the type and extent of user training. To be effective the protocol must grant flexibility to the BCI system and serve the needs of the individual user [15][21].
III. SIGNAL ACQUISITION
Coordination is essential in order to achieve BCI robots signal acquisition optimization, due to its importance between areas like electrode design, nature of information coding as well as other aspects of integrative and cognitive neuroscience [23].
BCI robots can be manipulated by many different methods of signal acquisition that can be divided into two groups: Invasive or non-invasive. The invasive methods are capable of acquiring a more accurate signal, since the electrodes are in direct contact with the neurons. However, the difficulty of this procedure can lead to various cases of post-surgical complications. On the other hand, non-invasive procedures have no complications due to its implementation procedures, nonetheless, the signal is measured above the scalp which can lead to different signals and noise. In Figure 2, it is showed the different types of processes of EEG signal acquisition [24].
A. Invasive methods
For the invasive methods, intracortical acquisition represents the most invasive technique because of its plantation under the cortex surface. It can measure the activity of a single neuron using a single or an array of electrodes. This approach requires a long-term stability, so electrodes must be placed very close to the signal source. On the other hand, Cortical Surface (ECoG) method is a less invasive option that can maintain the advantages of an invasive approach. Thus, ECoG recording is considered the half-term between invasiveness accuracy and the safety of non-invasiveness. It is based on the implantation of electrodes grids or strips over the cortex surface´[25].
B. Non-invasive methods
Concerning non-invasive techniques, there are several types, as shown in the Figure 2. The magnetoencephalography (MEG) measures magnetic fields produced by the neuronal activity. Signal reading is executed through superconducting quantum interference device. Also, the functional magnetic resonance imaging (fMIR), analyses the blood flow related to the neural activity in the brain. This technique makes the correlation between the used brain areas and the source of the patient’s disability. Likewise fMIR, the functional near-infrared spectroscopy (fNIRS) measures the blood dynamics, however, it uses near-infrared light range to acquire this measurement.
Lastly, the EEG is the most common type of signal acquisition, therefore it is going to be further approached throughout this paper. The EEG (electroencephalography) records the electric signal of the neuronal activity using electrodes attached in a cap-like device above the scalp. In this case, the measuring signal is based in fluctuations in neurotransmission activity, that also provides high temporal resolution. The fact that it is a portable and inexpensive device allows to create an accessible product for the commercial use. Consequently, experiments have been conducted with this method in order to solve the problems associated with EEG signal acquisition.
Table 1 summarizes the advantages and disadvantages of the different methods of signal acquisition to BCI [24].
Heading for a correct implantation of the electrodes, several normative lines are essential for researchers to follow. Thus, researchers believe that motor cortex is the best location because of its direct relevance to motor tasks and its relative accessibility. Other areas can be used, since by using fMRI and MEG it is possible to infer zones of better relevance for the study in question (subcortical motor areas or the thalamus) [26].
Concerning the signal analyses, the system must have the ability to record it. Several studies demonstrate the possibility of maintenance of the analysis for a long-term period, in mammal such as humans. This is essential for the viability of the method. These studies consider microwires and micromachined microelectrode arrays as an excellent way to make the acquisition [26].
C. EEG
Focusing in the EEG, the most common material for the electrodes are Ag/AgCl since they are low cost, have low contact impedance and relativity low impedance, normally in wet conditions. The electrodes are placed according to the standard, which means that the electrodes are located on the scalp at 10% and 20% of a measure distance from references sites. Those signals are normally recorded from the inferior frontal, central and parietal regions. For some specific analysis, they can be located at the occipital region [23].
After the signal acquisition, the signal must be processed to be analyzed. The processing of the signal can be divided in feature extracting and translation. The most common method is the filtering with a low-pass, high-pass and notch filter, that can be built to a certain cut-off frequency. Alternatively, ICA (independent component analysis) has been an increasingly technique that allows the removal these artefacts. Features extraction are obtained in the time domain and are typically frequency power spectra of EEG signals which can be estimated with Welch’s periodogram algorithm [27][28].
To translate this features from EEG signals output there are various classification methods such as K-nearest neighbor, linear discriminant analysis (LDA), nonlinear neural networks (NN), support vector machines (SVM) and statistical classifiers [23]. In the table 2, its summarize same features about the methods.
Table 2 – Characteristics of different classification methods
Method Description
LDA Linear classifier;
Simple to use;
Low computational complexity.
ANN Nonlinear classification;
Based on biological neural networks;
Less training set;
Simple, robust and easy computation
SVM Linear classification;
Easy to configure;
Suitable for several cases;
Small amount of training data is gained;
High computational complexity.
KNN Non-linear;
Easy implementation and debug;
Computationally intensive for large training sets;
Poor during large training set, sensitive for irrelevant and redundant feature.
To summarize, we can see in Figure 3 the various steps of signal acquisition of a BCI system, which provides feedback for the user during its process [9].
IV. BCI APPLICATIONS
BCI technology enhances communication capabilities while operating brain-actuated applications. It has been initially developed to be employed in biomedical study fields. Despite this preference it can be useful in other areas, such as: neuroergonomics and smart environment, neuromarketing and advertisement, games and entertainment, security and authentication as well as educational and self-regulation [24] [29].
Regarding the medical applications, BCI can be applied in the three medical field phases, them being prevention, detection and diagnosis and rehabilitation and restoration. Some researchers have developed different approaches on prevention by decreasing the level of alertness that outcomes from alcohol drinking and smoking. Moreover, studies on preventing traffic accidents have been enrolled and are based on analyzing drivers’ motion sickness [24].
In relation to detection and diagnosis, this technology enables the mental state monitoring function to detect abnormal brain structure and/or activity. Researchers have set their main priority to stablish tumors detection systems. Alternatively, systems to detect epilepsy seizures in its early phase are also being advanced since this neurological disorder is one of the most common nowadays [24].
In a more careful approach, rehabilitation and restoration are going to be the main subject, specifically the mobility rehabilitation which involves the physical rehabilitation of those whose mobility was prejudiced. This phase allows patients to regain their previous movement capabilities. However, if such abilities cannot be restored, it enables them to learn how to adapt to their acquired disabilities. In these cases, BCI can be seen as an alternative method to communicate [29]. Furthermore, when there are events as a stroke, the patient can lose the ability to move, yet through neuroplasticity the damaged motor functions can be re-stablished [24]. The rehabilitative interventions in the previous case are focused on active movement training, more specifically, passive mobilization and constraint-induced therapy [30].
On the other hand, there are neuroprosthetic devices that support patients that are uncapable of recovering the totality of their mobility. In order to help paralyzed patients completing regular activities, mobile robots have been created [30].
A. Performance Evaluation
The process of researching and developing brain-controlled mobile robots requires the evaluation and comparison of different performances. Even though it is pretended to evaluate the performance, no standardized method has been defined yet [31].
1) Subjects
When projecting a brain-controlled mobile robot several guidelines need to be taken into consideration, such as the condition of the patient, the specification of the test environments and tasks, and the evaluation metrics. Several studies have been pursued in order to determine the efficacy of the BCI system in healthy and disable patients. Those studies conclude that, wherever the motor disabilities are located, the healthy patient has a better control over the robot. Nevertheless, using ERD/ERS BCI, disabled people present a similar behaviour comparing with healthy patients [31].
2) Tasks and Environments
Regarding the test environments and tasks, the most common is the one that allows the patient to move the mobile robot from one position to another. The environments and tasks can be classified as simulation or realistic conditions. Between both, the simulated type can be cheaper and faster to realize as well as it can be used to extensively test different robots. Despite that, virtual mode could never be compared to the realistic one due to the several adversities existing in real life. To achieve the most complete and accurate experiment both classifications must be taken into the consideration [31].
3) Evaluation Metrics
To evaluate the brain-controlled mobile robot systems, the used metrics that can be divided in two categories. The first category is designated as the task metrics and is focused on specifying the functions to perform with the brain controlled robots. The most common task metric is task success which, as its’ name implies, indicates the accomplishment degree. The task completion time, number of collisions and BCI accuracy are other tasks that can be taken into consideration [31].
As for the second category, which is denominated as ergonomic metrics, instead of representing the user’s performance, it represents his/her state. It is essential to evaluate the workload imposed in the user, since he/she is realizing a mental effort during a certain concentration time. Additionally, there are two more ergonomic metrics, the first one being the ability to learn how to use the robot and the second the level of confidence that resulted from the participants experience [31].
Lastly, the cost metric should also be taken into account along with the task and ergonomic metrics [31].
B. Training to Use BCI
The process of implementing these methods of rehabilitation require training, which can be made in three different approaches: real, virtual and augmented. The real rehabilitation method compares decoded kinematic specifications with brain signals provided from healthy people. As for the virtual approach, it relies on exercising in a virtual reality and provides feedback to support learning skills since it monitories and controls avatar movement generated from the outgoing brain waves. Another approach is the augmented reality and it consists on amplifying a reality using an augmented mirror box system. Based on this technique, it was developed a Mirror Box Therapy (MTB) that through symmetrical movements incorporating healthy and injured limbs, acquires the generated brain signals [24][30].
Interacting and controlling the previous mentioned equipment, neuroprostheses and mobile robots require practice and can be more accessible when sharing autonomy frameworks. These allow to perceive the obstacles in the environment surrounding the robot and the position and velocity of the robot itself. The obtained information is then combined with the outputs of the BCI, giving a better perception of the response of the user’s intent [30].
C. Motor Imagery
Researchers have built a brain-controlled mobile robot based on motor imagery (MI) and able to perform three motion commands, such as, going forward and turning left and right. MI is defined as a dynamic state. It is also an alternative approach in all stages of recovering from a stroke to access the motor system and rehabilitation, since the representation of a specific motor behavior can be rehearsed without obtaining motor output [30][31][33].
Due to this approach and according to some studies, patients that use MI training achieve a better improvement on motor performance. Concerning the results of studies on MI, it is possible to conclude that the functional motor recovery is more effective. Combining MI with motor action observation results in a more effective BCI-based neurofeedback paradigm [30][34].
D. Shared Control robots
According to the operational modes of brain-controlled mobile robots, we can divide them into two categories, being one of them the “direct control” via BCI. This means that BCI translates a specific mental activity, that results on imagining movement of parts on the patient’s body, into a command to control robots, i.e., imagining realizing the movement activates the robot. On the other hand, the intelligence of the robots is included in the second category and provides more support to the user [29][31].
Additionally, because users are capable of controlling the robots as much as possible, this type of robots does not acquire more advanced intelligence [31].
In spite of patients not being able to have control over the robot for long periods of time, its control must be shared with an intelligent controller. Therefore, users are less feasible to become exhausted and the safety of driving these robots is enhanced [31].
Although these robots are more advantageous in certain aspects, they are also more expensive due to its complex computation and integrated sensors. As for the robots that belong to the first category of brain-controlled mobile robots, the computational complexity and the cost are low [29].
E. Humanoid Robots
Over the last decade, the development of humanoid robotics became possible due to the advances in electronics, mechanics and computer technology [36].
Nowadays, there are several BCI-based control systems for robots controlled by the most common method, a non-invasive BCI based on EEG analysis [29]. Such systems are: “Event Related Potential (ERP) for humanoid robot walking, steady-state visually evoked potential (SSVEP) for manipulating table-top objects, P300 evoked potential for robot navigation, controlling a virtual hand and manipulation of objects” [36].
On the subject of SSVEP, it is extremely common in researchers work and in the last five years it was combined with MI to provide humanoid robot control. In Figure 4 it is shown the NAO humanoid robot and several of its characteristics.
The humanoid robot NAO was created and has the following features:
• Programmable
• High: 57 cm
• Degrees of freedom: 25
• Electric motor actuators and several sensors, among others.
For this specific application, besides using MI to allow to turn left or/and right, the P300 method was also combined as step control [35].
Firstly, the EEG is implemented to collect the signal, which can be done through several devices. However, in situations of study, it was used a Bluetooth Interface [35].
Thus, sets of experiments are prepared so the humanoid robot can be tested and when the experimental phase is concretized, the results of the tests on obstacle avoidance are attained, proceeding to its analyses [35][36].
These studies and projects contribute to create a semiautonomous humanoid robot which main function will be to assist handicapped subjects with difficulties on communicating [35][36].
F. Robotic wheelchair
The brain-controlled robotic wheelchair was first developed by Tanaka and when the patient imagines left or right limb movements, the wheelchair turns according to what the user imagined [31][32].
Usually, the brain controlled wheelchairs are basic wheelchairs with an interface integrated that enables the user to control it and send that information to a laptop computer [37].
Likewise the mobile robot previously specified, in the referred robotic wheelchair it is used P300 BCI. In order to improve the wheelchair the P300 BCI was combined with a navigation system that is autonomous [31][37].
Moreover, using an architecture of shared control will aid in a higher level the possessor, since it provides a better perception of the environment surrounding the patient. In the next figure there is a schematic of how the brain controlled robotic wheelchair system functions [37].
The shared controller interprets the user’s input while the complementary sensors “study” the environment. Posteriorly, based on the patient input and on the occupancy grid, it is generated signals that allow the safe navigation [37].
Although it was mentioned a robotic wheelchair with shared controller, there are systems controlled uniquely by the user that are also as successful. In this cases, the driver controls the totality of wheelchair, following trajectories determined in real time [37].
Overall, the subjects that tested the robotic wheelchair are able to achieve a remarkable level of control, which enables to improve their life quality [31][37].
V. CHALLENGES AND PROPOSED SOLUTIONS
Even though there has been an increasingly development in robotics and control of robots via BCI, there are still various challenges and obstacles to overcome.
The main challenges are related to the signal in study and include noise, non-stationarity and non-linearity. The noise results on alterations of the electrodes placement and due to the environmental noise. As for the non-stationarity, it originates a constant alteration on the signals obtained in recorded sessions. In relation to the non-linearity, the chaotic neural behavior is detected in the acquisition signal [24]. Moreover, the EEG-based BCI possesses a limited frequency range and the brain signals acquired are relatively weak [15]. Hence, none of the brain controlled mobile robots have been tested out of the laboratory environment due to the instability of the EEG signal [31].
In order to diminish these interferences on the obtained signal, it can be used the Independent Component Analysis (ICA) which was mentioned previously [24].
Another challenge is associated with the training sets, which allow the user to gain experience to learn how to control the system and the neurophysiological signs. However, long training sessions are demanding for the subjects, since full attention is needed, and time consuming. Consequently, fatigue and therefore inconsistent operation by the users are common. To overcome the small training sets’ limitations and some variations associated to different trials it can be employed three machine learning techniques, such as k nearest neighbors (KNN), linear discriminant analysis (LDA) and support vector machines (SVM). These techniques were previously mentioned and the main aim is to accomplish higher performance [24].
Thereby, to overcome the current issues as well as to expand the capacity of BCI applications, continuous development is necessary in areas like accuracy, consistency, speed and practicality. [15]
VI. CONCLUSION
BCI main objective is to fabricate an alternative to the common interaction between humans and their enveloping environment. Despite its instability and need for digital signal processing, the most advantageous method of acquiring the neuroelectric activity is through EEG, since it can be applied either on disabled or healthy subjects. Furthermore, BCI is an ambitious technology that withal its advancements and remarkable results, still has several limitations such as real-time control and the execution of precise commands. However, BCI doesn’t have to be an ideal system since it features a better performance via a shared control, minimizing the user’s fatigue and enabling machine learning techniques. Therefore, it can be a versatile and adaptable cutting-edge technology