Electroencephalograph (EEG) has been widely used for BCI applications due to its non-invasiveness, ease of implementation, and cost-efficiency. The collected EEG signal is non-stationary and has task-related information buried in the frequency and temporal domains. In this book, we focused on developing time-frequency decomposition methods for improving the feature extraction module in BCI systems. The obtained features are then optimized by identifying subject-specific reactive band and employing evolutionary algorithm-based methods for optimizing the obtained features, improving the BCI systems' performance.
A signal model named as band-limited multiple Fourier linear combiner (BMFLC) is employed to model the non-stationarity in the EEG signal for feature extraction. The non-stationary amplitude oscillation is presented as adaptive weights in the model and estimated with various adaptive filters such as least-mean square (LMS), Kalman filter (KF), or Kalman Smoother (KS). The estimated coefficients serve as features for classification. Our results suggest that the BMFLC-LMS, BMFLC-KF, and BMFLC-KS are all sufficient in modeling EEG signal in the band with average estimation accuracies of 93%, 99%, and 98%, respectively. We modelled motion-induced EEG signal in frequency domain. We found that most subjects present a subject- specific reactive band during motion tasks. We then constructed features for a classifier that used only the frequency information in the subject-specific reactive band. As a result, the classification accuracy of the BCI system is improved compared to the system which uses the complete band information. Features obtained from multiple EEG channels need to be optimized to enhance the performance of the BCI systems. Essentially, two problems need to be resolved: 1) volume conduction; 2) dimensionality. The volume conduction can be mitigated if spatial filter is employed, and the dimension of the feature vector can be reduced if a feature selection procedure is adopted. An evolutionary algorithm (EA) based approach was developed to estimate spatial filter and reduce feature dimension simultaneously. We show that the BMFLC-KF combined with the evolutionary al- algorithm has the highest classification accuracy compared to other BMFLC-KF based approaches and is superior to the traditionally employed band-power methods.
Contents
List of Figures
List of Tables
1 Introduction
1.1 Background
1.1.1 EEG-based Brain-computer Interface
1.2 Motivations
1.3 Contribution of the Thesis
1.4 Organization of the Thesis
2 Literature Review
2.1 EEG for BCI
2.1.1 Origin of EEG
2.1.2 Rhythmic Pattern in EEG
2.1.3 Event-related De-synchronization and Synchronization
2.2 ERD/ERS-based BCI Systems
2.3 Feature Extraction of the ERD in EEG
2.3.1 Band Power based Methods
2.3.2 Time-frequency Decomposition based Methods
2.4 Feature Optimization for BCI
3 Sub ject-specific Reactive Band Identification with BMFLC-LMS
3.1 Overview
3.2 Methods
3.2.1 Band-limited Multiple Fourier Combiner with Least Mean Square
3.2.2 Time-frequency Decomposition
3.2.3 Subject-specific Reactive Band Identification
3.2.4 Normalized Power Between Events
3.3 Experiment Design
3.4 Results
3.5 Discussions
3.6 Summary
4 Single Trial ERD Detection with Multiple BMFLC-LMS
4.1 Overview
4.2 Methods
4.2.1 Time-delayed Multiple BMLFC-LMS
4.2.2 Time-frequency Decomposition of Multiple BMFLCs
4.2.3 Reactive Band Identification for Single Trial EEG
4.2.4 ERD Detection in Single Trial EEG
4.2.5 Data Set
4.3 Results
4.4 Discussions
4.5 Summary
5 Time-frequency Decomposition of EEG Signal with BMFLC Kalman Filter/Smoother
5.1 Overview
5.2 Methods
5.2.1 Uncertainty Principle of Time-frequency Methods
5.2.2 State-space Model of BMFLC
5.2.3 BMFLC with Kalman Filter/Smoother
5.2.4 Data Sets
5.3 Results
5.3.1 Parameter Selection
5.3.2 Estimation Accuracy
5.3.3 Temporal and Spectral Resolution: Comparison of All Five Methods
5.3.4 Computational Complexity
5.3.5 ERD Detection
5.3.6 Performance of BMFLC-KF under Noise Condition
5.4 Discussions
5.5 Summary
6 EA-based Feature Optimization for Multi-channel EEG Classification
6.1 Overview
6.2 Methods
6.2.1 Proposed BCI Architecture
6.2.2 EA-based Spatial Filter and Feature Selection Optimization .
6.3 Results
6.3.1 Experiment Settings
6.3.2 Performance of GLGA and CMA-ES-based Feature Optimization
6.3.3 Performance of the Final Classifier for All Configurations
6.4 Discussions
6.5 Summary
7 Conclusions and Future Work
7.1 Conclusions
7.2 Future Work
References
Abstract (Korean)
List of Figures
Figure 1.1 Work-flow of a typical BCI system
Figure 2.1 (a) International 10-20 and 10-10 system diagram, reproduced from 1; (b) Schematic diagram of pyramidal cells located at motor cortex and the origin of LFP and EEG signal; (c) Examples of LFP and EEG signal collected simultaneously, reproduced from
Figure 2.2 Examples of the EEG signal in each frequency band
Figure 2.3 (a) Motor cortex overlay with the EEG electrode according to the international 10-20 system. (b) Cortical homunculus represents the one-to-one correspondence of the motor cortex to the body part
Figure 2.4 Example of the ERD during motion execution
Figure 3.1 Structure of BMFLC: (a) BMFLC architecture; (b) Frequency distribution for BMFLC
Figure 3.2 Experiment time line
Figure 3.3 Estimation of EEG signal (a) Estimated EEG. (b) Estimation error
Figure 3.4 Subject #1, time-frequency decomposition with BMFLC weights (parameters for the proposed method Aw/2n = 0.5Hz, n=0.035)
Figure 3.5 Time-frequency decomposition for movement and rest EEG
Figure 3.6 Estimated subject-specific reactive band
Figure 3.7 Reactive band distribution in 30 subjects
Figure 3.8 Event classification with the weights obtained from BMFLC
Figure 4.1 Structure of time delayed BMFLC-LMS
Figure 4.2 Timing scheme of the experiment
Figure 4.3 Time frequency decomposition the proposed method for three subjects (S#1, S#3 and S#7, the dark line indicates onset of the stimulus); Reactive band identification for three subjects
Figure 4.4 Comparison of ERD % in alpha band and the identified reactive band
Figure 4.5 95% CI fro ERD in alpha band (a, b, c); 95% CI fro ERD in reactive band (d, e, f )
Figure 4.6 Average classification accuracy at four decision points for all subjects
Figure 5.1 Architecture of Kalman filter/smoother based BMFLC
Figure 5.2 Parameter tuning for Kalman filter: (a) Parameter q selection based on S1(t); (b) Parameter q selection based on S2(t); (c) Parameter R selection with S1 (t) for fixed q; (d) Subject #1 C3 (all trials); (e) Subject #1 C4 (all trials)
Figure 5.3 Time-domain estimation accuracy of BMFLC, Af = 0.5Hz
Figure 5.4 Time-frequency mapping for synthesized signals, S 1(t) and S2(t), Af = 0.5Hz
Figure 5.5 Time-frequency mappings of various methods for signal S3(t)
Figure 5.6 Modified CWT for S2(t) and S3(t)
Figure 5.7 Estimation performance for S4(t)
Figure 5.8 ERD mapping of 3 subjects for all methods. The ERD detection of various methods for S #1 right hand imagery, S #3 right hand imagery, S #7 right hand imagery is shown from first to third row respectively The magenta vertical line indicates the experiment cue onset
Figure 5.9 ERD detection comparison for all subjects of right hand imagery
Figure 5.10 ERD detection for subject 3 in one trial. The magenta vertical line indicates the experiment cue onset; (a1) Raw EEG signal; (b1) BMFLC-LMS; (b2) BMFLC-KF; (b3) BMFLC-KS; (b4) Short-time Fourier transform; (b5) Continuous-wavelet transform
Figure 5.11 Performance evaluation of BMFLC-KF under noise conditions. (a) Timefrequency mapping of the mixed signal with SNR = 190; (b) Timefrequency mapping of the mixed signal with SNR = 3.74; (c)Time- frequency mapping of the mixed signal with SNR = 0.54; (d) Estimation error
Figure 6.1 Proposed configurations
Figure 6.2 Evolution of training of CMA-ES for 300 generations. Shaded area indicates the standard deviation obtained from 10 cross validation runs
Figure 6.3 Evolution of training of GLGA-25 for 10,000 function evaluations. Shaded area indicates the standard deviation obtained from 10 cross validation runs.The vertical line indicates the transition from global searching to local searching
Figure 6.4 Classification accuracy improvement for BMFLC-CMAES, FFT-CMAES and BMFLC-GA
Figure 6.5 Selection of optimal number of spatial filter pair
Figure 6.6 Classification accuracy on testing set of all configurations
Figure 6.7 Performance comparison for various configurations
List of Tables
Table 2.1 Comparison of various time-frequency decomposition methods
Table 3.1 Accuracy with BMFLC
Table 3.2 Subjects with the reactive band % energy
Table 3.3 Comparison of r-power spectral density (rP/Hz)
Table 4.1 Subjects with optimal band power ratio
Table 4.2 Comparison of ERD (%) in alpha band and reactive band
Table 5.1 Estimation accuracy of BMFLC based methods
Table 5.2 Estimation accuracy for different frequency gaps
Table 5.3 Computational complexity
Table 5.4 Computational complexity of BMFLC-KF
Summary
Electroencephalograph (EEG) has been widely used for BCI applications due to its non-invasiveness, ease of implementation, and cost-efficiency. The collected EEG signal is non-stationary and has task-related information buried in the frequency and temporal domains. In this book, we focused on developing time-frequency decomposition methods for improving the feature extraction module in BCI systems. The obtained features are then optimized by identifying subject-specific reactive band and employing evolutionary algorithm-based methods for optimizing the obtained features, improving the BCI systems' performance.
A signal model named as band-limited multiple Fourier linear combiner (BMFLC) is employed to model the non-stationarity in the EEG signal for feature extraction. The non-stationary amplitude oscillation is presented as adaptive weights in the model and estimated with various adaptive filters such as least-mean square (LMS), Kalman filter (KF), or Kalman Smoother (KS). The estimated coefficients serve as features for classification. Our results suggest that the BMFLC-LMS, BMFLC-KF, and BMFLC-KS are all sufficient in modeling EEG signal in the band with average estimation accuracies of 93%, 99%, and 98%, respectively.
We modelled motion-induced EEG signal in frequency domain. We found that most subjects present a subject- specific reactive band during motion tasks. We then constructed features for a classifier that used only the frequency information in the subject-specific reactive band. As a result, the classification accuracy of the BCI system is improved compared to the system which uses the complete band information.
Features obtained from multiple EEG channels need to be optimized to enhance the performance of the BCI systems. Essentially, two problems need to be resolved: 1) volume conduction; 2) dimensionality. The volume conduction can be mitigated if spatial filter is employed, and the dimension of the feature vector can be reduced if a feature selection procedure is adopted. An evolutionary algorithm (EA) based approach was developed to estimate spatial filter and reduce feature dimension simultaneously. We show that the BMFLC-KF combined with the evolutionary al- algorithm has the highest classification accuracy compared to other BMFLC-KF based approaches and is superior to the traditionally employed band-power methods.
Chapter 1
Introduction
1.1 Background
Brain-computer interface (BCI) or brain-machine interface (BMI) is a collection of hardware and software systems which serves the purpose of translating measured brain activities into control commands. The output of BCI is then utilized for controlling an external device or software application 3.
Since the first brain electric activity was measured in laboratory by Hans Berger in 1938, researchers immediately started to look for methods that can utilize this brain generated signal 4. Hans' new finding had immense implications in gaining insight into brain functions 5 and developing new therapeutic methods for treating neurological disorders 6. One of the most profound applications of brain electric activity measurement was the development of BCI systems in which an direct link could be estbalished between the brain and the external world 7.
The BCI research in its earlier days focused primarily on confirming the possibility of employing ongoing brain activity to decipher thoughts or intents of humans [7-10]. Among the earlier attempts, the Brain-Computer Interface project initiated by Vadal confirmed the possibility of using human brainwave to generate meaningful control signal and also coined the term brain-computer interface that has been widely accepted by the research community [11, 12].
As pointed out in 7, the communication pathway linking brain to the external world is the ultimate form of man-machine interface. The research on BCI would be of great importance for the patients who suffer from the diseases such as amyotrophic lateral sclerosis (ALS), spinal cord injuries and stokes where the information transmission to the peripheral muscular system has been blocked 3. Over the last decade, a lot of promising BCI systems that would be helpful to the patients have been presented in literature. Some examples of such BCI systems include but are not limited to wheel chair control 13, P300 speller 14, artificial prosthesis 15, and neural rehabilitation 16.
The territory of the BCI also expands to the healthy subjects. A virtual reality game that can be controlled by the activity that is generated from the motor cortex of the subject has been reported in [17, 18]. The attempts of decoding human emotion directly from brain measurement are reported in 19. The versatile responses that can be collected from human auditory cortex also inspired the research on brain-computer music interfacing 20. With the miniaturization of the recording device, the brain-computer interface also shows the possibility of being used for computer encryption 21. The brain activity based fatigue detection is believed to be useful in detecting drowsiness in drivers 22.
The current available brain imaging techniques can be broadly categorized as invasive and non-invasive. The invasive methods usually involve a costly and painful surgical procedure to implant the sensor directly on the surface of the cortex or by penetrating few millimeters into the cortex to achieve a better signal-to-noise ratio 23. Thus the intracranial signal that is collected by invasive methods makes decoding complex mental tasks possible 24.
Non-invasive methods also play an important role in the development of BCI systems. In [25, 26], the blood oxygenation level dependent signal (BLOD) signal which is measured by functional magnetic resonance imaging (fMRI) was employed. The Magnetoencephalographic (MEG) is the measurement of magnetic field function distortion was caused by brain activity 27. With better spatial resolution, its application to BCI was reported in 28. In [29,30], a hybrid BCI system by binding signals from both EEG and functional near-infrared spectroscopy (fNIRS) were employed to decode motion imagery. Transcranial Doppler ultrasound (TCD) which measures the blood flow velocity in the cerebra is also tested for the application of BCI 31.
Among all the non-invasive methods, the EEG based BCI system has been in the spotlight since the beginning of the BCI research 7. Although EEG is easily contaminated by both internal and external noise as compared to the other non-invasive methods 32, its ease in conducting experiments and cost
makes it the most popular method for BCI applications.
This dissertation mainly focuses on developing new algorithms and techniques to decode the brain patterns that are employed in the EEG-based BCI systems. To better illustrate the needs and the objectives of this work, the working principle of the EEG-based BCI system will be given in the following sub-section.
1.1.1 EEG-based Brain-computer Interface
Despite the difference of the information sources that are employed in designing a BCI system, a general BCI system usually consists of the following six modules: brain activity measurement, signal pre-processing, feature extraction, feature post-processing, classification and feedback 33. The diagram of a typical BCI system is shown in Fig. 1.1.
Abbildung in dieser Leseprobe nicht enthalten
The BCI system first obtains the brain activity measurement, in this case the EEG signal from the brain. The recorded EEG signal in general has low SNR, thus pre-processing is employed to enhance the signal characteristics. Before generating control command by the classification module, features required by the classifier are obtained by the feature extraction module. Optionally, the obtained feature undergoes an extra feature optimization to reduce the dimensionality in order to improve the performance of the classifier. Then the classification module provides a decision or a real-valued control signal to the external effector. The action performed by the application serves as a feedback signal to the subject. The subject can then modify their mental strategy accordingly to improve performance.
1.2 Motivations
From the engineering perspective, the primary goal of the BCI research is to develop algorithms and methodologies that improves the performance of BCI which is quantified by certain performance indices such as classification accuracy, information transfer rate (ITR) 34 etc. Further, the developed methods could shed light on the mechanism of the functional brain that would in turn lead to improvement of BCI systems 35.
Among the building blocks that are identified in Fig. 1.1, the performance of a BCI system is primarily determined by the the classification module in which the control signal is generated. In current BCI systems, the classification module usually employs machine learning algorithms to learn the pattern from the training data and induces output from the testing data or on-line signals 33. The employed machine learning algorithms in literature assume that the features have already been processed to contain statistical meaningful patterns. Thus the reliability of the feature extraction and the feature optimization modules which precede the classification module plays a important role in the performance of the BCI.
Feature extraction module in BCI system extracts useful information from the EEG signal by utilizing the advanced signal processing algorithms 36. It processes and transforms the EEG signal into the form that is required by the machine learning module. To make the extracted information suitable for the BCI application, the understanding of the brain working mechanism is required.
Brain mainly generates two types of electric activity: the rapid depolarization of the neuronal membranes which produces the measured action potential and the relatively slower oscillation which is caused by the synaptic activation 37. These electric changes from a group of neurons can be acquired to produce the local field potential (LFP) 38. The EEG signal, however, is the spatial-temporal smoothed version of the recorded LFP over an area of approximatively 10 cm2 2.
The EEG signal is the mixture of both continuous membrane potential and discrete action potential with multiple frequencies. As the EEG signal also evolves with time, its temporal and spectral characteristics should be analyzed simultaneously 39. The analysis of EEG signal must also include the specific brain pattern that corresponds to the mental tasks that are involved in the design of the experiment. The existing methodology of analyzing the signal in temporal domain mainly employs grand-averaging to find the event-related potential 40. Fourier type of methods are popular for analysis in spectral domain 33. To obtain the joint estimation, namely time-frequency analysis, the short-time Fourier transform is usually employed 41.
As far as BCI systems are concerned, the averaged EEG signal or the Fourier transformed signal are employed for feature extraction. The existing methods have been successful to a certain extent as shown in 17. However, to further improve the performance of BCI systems, new methods are required to better identify the EEG characteristics in the joint temporal and spectral domain.
The development in the feature extraction is a double-edged sword. On one hand, the advanced methods would decode more detailed information from the EEG signal. While on the other hand, a high dimensional vector would be required to present such detailed information which will inevitably lead to degraded performance in the classification module. Thus, the obtained feature from the feature extraction module requires feature optimization to (i) identify the most relevant information related to the the tasks; (ii) reduce the dimensionality of the obtained feature vector.
1.3 Contribution of the Thesis
In this thesis, our main objective is to improve the performance of the BCI system by developing better feature extraction and feature optimization techniques for BCI as shown in Fig. 1.1.
To improve the feature extraction module, a signal model named as the band-limited multiple Fourier linear combiner (BMFLC) was adopted. The BMFLC uses a truncated Fourier series as the signal model and the model parameters are the Fourier coefficients. We estimated these Fourier coefficients by applying adaptive filters such as least-mean square (LMS), recursive leastmean square (RLS), Kalman filter and Kalman smoother. Thus, we improve the BMFLC model to estimate joint temporal-spectral distribution in real-time or quasi real-time.
To analyze the frequency characteristics of the EEG signal under motor execution, we applied the BMFLC with LMS algorithm for obtaining the timefrequency mapping from a large pool of subjects. We statistically demonstrate that a 3Hz subject-specific frequency band exists in most of the subjects during the motor execution tasks.
We extend the BMFLC with LMS method to the single trial EEG real-time analysis by adopting a time-delayed overlapping scheme. The results obtained from the EEG data under motor imagery tasks indicate that the previous identified 3Hz subject-specific band persists in the motor imagery tasks. It is also shown that, compared to the traditional employed band-power features, the amplitude estimated from BMFLC provides better performance.
BMFLC with Kalman filter and Kalman smoother are developed to improve the adaptation for better modeling of EEG signal in real-time or quasi realtime applications. A comprehensive comparison study of the performance for the time-frequency decomposition was conducted between the BMFLC model and traditional methods such as short-time Fourier transform (STFT) and continuous wavelet transform (CWT). We demonstrate that BMFLC based modeling outperforms the traditional methods in spectral resolution. As quantified by event-related amplitude detection, the BMFLC based model outperforms STFT and provides similar performance compared to CWT. With far less computation complexity compared to CWT, we show that the BMFLC based modeling is well suited for real-time BCI applications.
To tackle the feature optimization, we employed an evolutionary algorithm based approach. The Fourier coefficients obtained from the BMFLC modeling of all available EEG channels undergo a spatial filter for improving the spatial resolution. Then, a frequency selection filter is also employed to identify the features that significantly contributes to the performance. The spatial filter and the frequency selection are tuned to maximize the classification accuracy. With help of the evolutionary algorithm, we can optimize the spatial filter and the frequency selective filter simultaneously. Our results show that the evolutionary algorithm based feature optimization significantly improves the performance of the BCI system compared to the traditionally employed common spatial filter methods.
1.4 Organization of the Thesis
Chapter 2 reviews the neuroscience foundations of the brain patterns that emerge during motor tasks in BCI. It provides a detailed literature review on the existing methods of feature extraction and feature optimization methods for EEG signal. This chapter is mainly intended to introduce the basic terminology and concepts of BCI systems.
Chapters 3- 6 forms the main body of the thesis. In these four chapters, developed methodologies on feature extraction and feature optimization are presented.
In Chapter 3, we present the results on identifying the subject-specific reactive band. The identification of the subject-specific reactive band serves as the basis for the feature dimension reduction. BMFLC with LMS is employed for time-frequency decomposition of EEG signal for a large pool of subjects. The subject-specific reactive band identification technique is developed in this chapter.
Chapter 4 presents the time-delayed BMFLC for real-time single trial EEG data analysis. The subject-specific reactive band analysis is conducted on the motor imagery data obtained from BCI competition IV dataset. The coefficients in the BMFLC are employed as features for a classifier. We demonstrate that the time-delayed BMFLC combined with the identified subject-specific reactive band improves the classification accuracy compared to the existing popular band power methods.
In Chapter 5, the BMFLC with Kalman filter and Kalman smoother are developed. A comprehensive comparison study between BMFLC based models and the traditional methods is conducted for the temporal and spectral resolution. Further, we also analyzed the performance of different time-frequency decomposition techniques for ERD detection.
Chapter 6 presents the evolutionary algorithm based feature optimization methods. In this chapter, three different BCI configurations based on BMFLC Kalman filter are discussed. The evolutionary algorithms employed are the covariance matrix adaptation evolution strategy (CMAES) and the global and local genetic algorithm (GLGA) for optimizing the features obtained from BMFLC Kalman filter. A comparison study of the proposed BCI configurations with the existing methods is presented.
Finally in Chapter 7, we conclude the thesis with scope for future research.
Chapter 2
Literature Review
This chapter provides the necessary information that underpins the development of this thesis. We first provide a brief discussion on the brain patterns that are commonly used in brain-computer interface (BCI). We then review on the existing methodologies for feature extraction of the brain oscillatory pattern. Further, the feature optimization methods that are employed to improve the performance of the BCI system are reviewed.
2.1 EEG for BCI
2.1.1 Origin of EEG
The neuron cells in brain communicate by means of electrical activity that is generated at soma, passing through axons and finally released at the dendrite 42. The neurons that are thought to be responsible for the surface EEG signal
Abbildung in dieser Leseprobe nicht enthalten
are the pyramidal cells that align in parallel with each other in axons direction and perpendicular to the cortex surface 2. Thus, when the pyramidal cells fire synchronously, the electric current in the longitudinal direction adds up and cancels out in the transverse direction 37. A schematic diagram is given in Fig. 2.1(b).
The resultant net current of a group of pyramidal cell forms an open field which is then detectable far from its source. The extracellular measurement of this current is named as local field potential (LFP), and is considered as the source of the scalp EEG signal 42.
To make the LFP detectable at scalp level, the LFP generated from a large area of cortex cells, typically 10cm2 or more, is spatially and temporally integrated in the extracellular space. It then travels through the skull and is finally aggregated at scalp to be picked up by the EEG electrodes 43. An example is given in Fig. 2.1(c). The recorded EEG signal is also a macroscopic representation of the neuron firing activity. Experiments, for simultaneous recording of multi-unit activity, LFP and EEG in non-human primates indicates that the low frequency component in EEG signal is well correlated to the underlying neuron firing rate that is measured by multi-unit activity 44.
Although the temporal resolution of EEG is almost same to its LFP origin, as the conductivity and geometry of skull and scalp differs, the volume conduction effect causes the EEG signal be low in spatial resolution 38. The EEG acquisition system requires spatial sampling on the scalp surface to maintain spatial resolution for applications such as electrical brain imaging 45. Currently, the mainstream EEG acquisition system uses the international 10-20 or 10-10 system to place the EEG electrodes on the scalp surface 46. The international standard on the EEG electrodes placement scheme utilizes the anatomical landmark of brain to pinpoint the exact location for each electrode spatially. The placement is also considered indifferent among subjects 1. The name of each electrode and their respective location is given in Fig. 2.1(a).
2.1.2 Rhythmic Pattern in EEG
The recorded EEG signal can be understood either from its temporal morphological pattern or through its frequency transformation. In the temporal domain, the dominant EEG pattern is denoted as the event-related potential (ERP). ERP is defined as the negative-positive deflection of the EEG signal that is obtained by averaging over many experiment trials. The negative-positive deflection waveform is identified and named according to an external stimulus given during an experiment trial 47. Thus, the ERP is the time- and phase-locked EEG waveform that focuses on the stationary aspects of the brain function 40.
The EEG signal, after transforming into the frequency domain reveals significant information of the brain working as compared to its temporal counterparts. In the frequency domain, it is observed that the EEG signal contains many frequency components, ranging typically between 0.1Hz to a few hundred hertz, and at least ten oscillators working interactively 48.
The rhythmic pattern in EEG is an indicator of the status of the neuron cells on the cortex surface 49. To perform a complete task, the neuron cells needs to work locally and interact simultaneously with the functional cortex that is far apart 50. From communication point of view, the lower frequency components in EEG indicate a long range communication in the neuron pathway, and the higher frequencies indicate a more localized neuron activity 51.
In general, the frequency spectrum of the EEG signal is divided into several frequency bands. The frequency band division from lower end to the higher end of the spectrum is given as: 5 (0.2 to 3.5Hz), 0 (4 to 7.5Hz), a or p (8 to 13Hz), d (14 to 30Hz), y (30 to 90Hz) and the high-frequency oscillations (HFO 90Hz) 42. An example of EEG signal band pass filtered with all frequency bands is shown in Fig. 2.2. This simple division in spectral domain is important and offers a neuroscience basis for further studies of brain functions through the analysis of EEG signal.
Abbildung in dieser Leseprobe nicht enthalten
Figure 2.2 Examples of the EEG signal in each frequency band.
Although all the oscillatory patterns can be observed from each EEG electrode, their functional roles and generation mechanisms differ significantly 52. More importantly, the rhythmic pattern in EEG can be modulated by an external stimuli. For example, the steady-state visual evoked potential
Abbildung in dieser Leseprobe nicht enthalten
Figure 2.3 (a) Motor cortex overlay with the EEG electrode according to the international 10-20 system. (b) Cortical homunculus represents the one-to-one correspondence of the motor cortex to the body part.
(SSVEP) obtained from the visual cortex is a frequency modulated rhythmic pattern 53. The observed SSVEP is frequency modulated with respect to an external stimuli that flashes at a given frequency. Thus, a simple binary BCI can be achieved by detecting the existence of the frequency of the stimuli from the SSVEP 54.
2.1.3 Event-related De-synchronization and Synchronization
The BCI system relies on the underlying brain patterns that are stable 35. Among the EEG rhythmic patterns found in EEG signal, the one observed in the motor cortex has attracted a lot attention from the BCI community [55, 56].
Abbildung in dieser Leseprobe nicht enthalten
Motor cortex is a strap layer of neuron cells that are located around the central sulcus of the cerebral cortex. It consists of the primary motor cortex, pre-motor cortex and supplementary motor cortex 49. Functional imaging and electrophysiology studies have concluded that the motor cortex activates during the planning, control and conduction phase of the human voluntary motions 57. According to the international 10-20 system, the motor cortex lies underneath the electrodes C1 - C8 passing through the vertex Cz 1. A sketch of the motor cortex and approximate location of the EEG electrodes is shown in Fig. 2.3(a).
Earlier works have shown that there is a one-to-one correspondence of the body part to the motor cortex, especially the primary motor cortex. As depicted in Fig. 2.3(b), the volume of the cortex devoted to a body part determines the complexity of the motion that could be achieved by that part. Further, the motor cortex controls the motion cross the body midline, i.e. the left motor cortex controls the motion on the left side of the body and vice versa 49.
The EEG rhythm obtained from the motor cortex manifests a stable amplitude fluctuation behavior. It was shown in 57 that during execution, imagery and observation of a motion or motion sequence can attenuate the amplitude of alpha and beta rhythm in the motor cortex at the beginning of the motion and a rebound occurs in beta band after the termination of the motion. An example of alpha band attenuation during motion execution of right hand is shown in Fig. 2.4. This amplitude attenuation/rebound can be better quantified by the relative change of the energy in a frequency band with respect to a pre-defined reference period 58. Therefore, this phenomenon is also termed as Event-Related De-synchronization or Synchronization (ERD/ERS) 59.
It was hypothesized that ERD/ERS is the reflection of the activity of the thalamo-cortical system. The attenuation of the alpha band amplitude or ERD is related to the involvement of the corresponding cortex in a higher information processing chain 60. Recent studies have also demonstrated that the magnitude of the ERD/ERS is independent from the load of voluntary motion 61.
2.2 ERD/ERS-based BCI Systems
The salient feature of ERD/ERS is the stableness of its existence, thus making it very attractive for its application in BCI systems 57. ERD/ERS can be steadily identified in most subjects by averaging the EEG over the motor cortex across multiple experiment trials 59. In the on-line BCI systems, a study that involves ninety-nine BCI novice subjects, the authors concluded that around 93% subjects could achieve above 60 % accuracy by decoding their motion intention via detecting the ERD/ERS 62.
ERD/ERS pattern that corresponds to the hand imagery tasks has been the center of interest in the context of BCI systems 63. The employment of hand motion induced ERD/ERS facilitates a easy binary BCI system. In its earlier forms, the ERD based BCI system collected EEG signal on top of the motor cortex. Then, the collected EEG signal can be either band-pass filtered into the frequency band where ERD/ERS occurs 64, or taking the variance of the EEG signal to form the corresponding features 65. Finally, a binary decision is made through a classifier 23.
With this simple architecture, the ERD based BCI system has been successfully applied to various robotic systems 66. The ERD-based BCI for the direction control of a wheelchair was studied in 67. Its applications to control an avatar in a virtual reality environment was shown in 68.
Binary BCI system is useful only for the application that requires an on/off switch type of control or an one dimension continuous control. However, most daily activities are more complicated, thus requiring multiple control commands to be generated. To facilitate multiple class ERD/ERS based BCI, the intuitive way is to incorporate more motion tasks. In 69, a four classes motion imagery task is adopted for controlling a robot in a two-dimensional space. In 70, two pairs of motion imagery tasks are employed to move a helicopter in three dimensional space.
As pointed out in 71, subjects can learn to modulate the magnitude of ERD with the help of visual feedback. It is observed that subjects tend to intensify their corresponding rhythm when a positive feedback is obtained 72. Thus, like any other skills, modulating EEG rhythm can be acquired with sufficient feedback training 73. In 74, the subjects can attain multiple tasks by modulating their ERD pattern in one hemisphere with feedback training. Moreover, this finding also facilitates various new applications of the ERD/ERS based BCI systems including motor function facilitation after a stroke 75.
To identify the ERD pattern in the EEG signal, usually an experiment cue is provided 40. Then the subjects carry out the motion imagery for a few seconds after the cue is given. Such implementation of BCI system is named as synchronous BCI. The drawbacks of the synchronous BCI is that the system tends to have a very slow information transfer rate 3, as the BCI can only provides few control command per unit time. A solution to this is to design asynchronous BCI where BCI outputs a command only if subjects intends to do so 76.
In an asynchronous BCI, the system needs to determine when to give an output and when to stay idle. It is clear that the system performance is heavily affected by the false command that is generated during the idle state 77. Thus, the ERD pattern has been utilized as the means of differentiating idle and work states in various asynchronous BCI system 78.
Recently, the hybrid BCI systems which employ multiple brain activity measurement or combining different brain patterns are being developed 79. The ERD pattern is the backbone for all of these hybrid schemes 80. In 81, the
ERD pattern and the brain metabolism pattern that is measured by functional near infrared reflectance spectroscopy (fNIRS) are combined to achieve better classification performance. In 36, the ERD and SSVEP hybrid BCI is employed to help the BCI illiterate subject for enlarging the audience of the BCI systems.
2.3 Feature Extraction of the ERD in EEG
In the above section, we have discussed the brain rhythmic patterns that make the EEG based BCI possible, with special focus on the ERD/ERS pattern that is observed from motor cortex. BCI system employs advanced signal processing algorithms in the feature extraction module to decode the ERD patterns from EEG as shown in Fig. 1.1. Feature optimization and classification follows the feature extraction. In what follows, we discuss the existing methodologies for identifying these patterns.
2.3.1 Band Power based Methods
The ERD/ERS exists in a particular frequency band. Typical hand motion induced ERD/ERS can be found in the range of alpha and beta band 59. The hand motion activates the contralateral motor cortex. Thus, the intuitive way of estimating ERD/ERS is to calculate the power in alpha and beta band from the EEG signal that is collected from both motor cortex simultaneously 82.
Throughout this paper, the ongoing EEG from the i th electrode signal is denoted as x i (t). When the subscript is omitted, x(t) denotes a generic EEG signal in time domain. Therefore, the band power can be obtained by band-pass filtering the EEG signal into each frequency band and then squaring the obtained waveform [83]. To form a feature vector, the obtained band pass filtered EEG signal needs to be averaged over a time window [84]. This procedure can be described by the following equation.
Abbildung in dieser Leseprobe nicht enthalten
where f (t) is the feature vector at each time instant t, bandpass(•, •) is the band pass filter operator where the first argument is the signal and the second argument is the pass band of the filter. (•)L and (•)R denote the EEG channel corresponding to the left and right motor cortex respectively. The feature obtained from a time window can be calculated as:
Abbildung in dieser Leseprobe nicht enthalten
where win is the window length in the sample.
The signal variance can be also employed as an alternative measure of the energy contained in the signal. In 85, instead of squaring the EEG signal after passing through the band pass filter, the logarithm of the signal variance within a time window is calculated and employed as the feature. In an extreme case, the total range of frequency band is divided into few 2Hz sub-band. A band pass filter is assigned to each sub-band, thus forming a band pass filter bank 86.
2.3.2 Time-frequency Decomposition based Methods
The band pass filter based method is one way to obtain the frequency characteristics in the EEG signal. In the real-time BCI systems, the band pass filters usually employed are mostly fourth or fifth order Butterworth FIR type filters 87. The frequency and phase response of the this type of filter is far from ideal. Moreover, the peak frequency for alpha band also varies with the subjects 88, selecting a universal frequency band for the band pass filter may damage the performance of the ERD detection. Thus, frequency transformation based methods are often employed 89.
The Fourier transform and fast Fourier transform (FFT) have been employed in BCI for feature extraction. To apply FFT, the EEG signal is first partitioned by a time window of fixed length. Then, the signal is padded with zeros to achieve the desired spectral resolution. Afterwards, the obtained Fourier coefficients in the frequency band of interest can be directly employed as features 76, or it can be summed up to reduce the dimensionality of the feature vector 90.
The generation mechanism of the EEG signal as discussed above strongly suggests that the EEG signal is non-stationary, i.e. the signal characteristic changes both in time and frequency domain 91. Apart from being non- stationary, the EEG signal also presents nonlinear behavior both locally and globally 92. Moreover, the oscillatory pattern in EEG contains inter- and intra- frequency couplings 93. The detailed frequency energy distribution may contain extra information which could improve the overall performance of the BCI systems. Hence, the detailed information in EEG should be obtained with the time-frequency representation.
The first choice for deciphering the non-stationarity in a signal is to apply short-time Fourier transform (STFT) 94. In STFT, the original signal is first isolated in the temporal domain by multiplying the signal with a window function h(t), then FFT is applied upon the windowed signal 95. It can be written as the following.
Abbildung in dieser Leseprobe nicht enthalten
where x(T) is the EEG signal in time domain and STFT(w, t) is the timefrequency distribution of the signal at frequency w and time t.AstheSTFT(w, t) is a complex value, its absolute or absolute square value is usually applied, i.e,
Abbildung in dieser Leseprobe nicht enthalten
There are a large number of window functions that can be selected. The Gaussian function and the rectangular function are the popular ones in the analysis of EEG signal [41, 90]. The feature vector for STFT is formed by selecting the frequency components corresponding to the band of interest from the power estimates given in (2.4) and is shown as.
Abbildung in dieser Leseprobe nicht enthalten
where w1 and w n are the lower and upper bound of the interested frequency band.
In the literature of EEG signal analysis, the continuous wavelet transform (CWT) or its discrete version (DWT) is usually employed to achieve better time-frequency resolution 41. Similar to STFT, a mother wavelet function is employed by CWT as the window function. This mother wavelet in CWT can be dialed by a parameter “a”.
If the mother wavelet function is a complex function, the resultant CWT is also termed as complex CWT 96. The generally selected mother wavelet is the Morlet mother wavelet function defined as:
Abbildung in dieser Leseprobe nicht enthalten
where w0 is the basis frequency of the Morlet mother wavelet. The e-w0/2 in (2.6) is a correction term to make sure that the mother wavelet satisfies the admission property 96. This term can be omitted once the basis frequency w0 5 is selected 96.
The TFR that is obtained from CWT can be given as
Abbildung in dieser Leseprobe nicht enthalten
where * denotes the complex conjugate. The CWT(w,t) is obtained by translating the dilation parameter a to the frequency of the signal via the following
Abbildung in dieser Leseprobe nicht enthalten
Thus, the energy in the signal is given as the absolute squared version of CWT(w, t), and given by.
Abbildung in dieser Leseprobe nicht enthalten
Similar to the case in STFT, the feature vector in CWT is given as:
Abbildung in dieser Leseprobe nicht enthalten
The discretization of the wavelet transform depends on the selection of the dilation parameter a. Instead of continuously changing the value of a,this parameter is discretized on a dyadic grid 97. The application of CWT or DWT for feature extraction can be found in [98-100].
The CWT and STFT belongs to the non-parametric model based TFR methods. Their alternatives are the parametric based TFR methods such as the autoregressive (AR) model 101. The AR model of a signal can be defined as:
Abbildung in dieser Leseprobe nicht enthalten
where b(t) and p are the coefficients and the order of the autoregressive model respectively.
To apply AR model to the feature extraction of the EEG signal, the coefficients b(t) needs to be estimated. These coefficients can be estimated through linear square estimation 102, adaptive algorithms 101, and particle filter 91.
The AR methods are suitable for BCI due to its close relationship with the TFR of the signal. The TFR corresponding to the estimated AR and ARMA model can be given by.
Abbildung in dieser Leseprobe nicht enthalten
where fs is the sampling frequency of the EEG signal, 5e is the estimated variance of the noise in (2.11).
From (2.12), it is clear that the AR coefficients is a surrogate of the exact TFR in the EEG signal. It can be inferred from (2.12) that the time-frequency information details which can be obtained by the AR coefficients depends on the order of the AR model. In 103, the optimal order of AR model suitable for EEG signal analysis in BCI is analyzed. The results suggest that a 16th order AR model is sufficient for the BCI applications.
In the above section, we reviewed the most popular feature extraction methods in ERD pattern detection. However, each of these methods has its pros and cons. Band-pass filter is simple, but its incapability in obtaining TFR of EEG limited its application in BCI. TFR decomposition methods such as STFT and CWT are constrained by their spectral resolution and computational requirement. The AR based methods are simple for implementation, however the spectral resolution is limited by the order. In this thesis, we employ a parametric model, BMLFC, to solve the TFR decomposition problem for EEG signal. The least-mean square based BMFLC was first developed in 104. In chapter 3 and chapter 4, the BMFLC-LMS methods are employed for EEG signal decomposition and TFR analysis. To enhance the performance of the BMFLC, we then developed Kalman filter and Kalman smoother based BMFLC for real-time/quasi real-time BCI applications. In a nutshell, the comparison of various time-frequency mapping methods is given in Table 2.1.
2.4 Feature Optimization for BCI
The methods in the above section can only be employed to extract features from a single EEG channel. For a hand movement imagery BCI, minimum two channels' EEG is required to cover the left and right hemisphere of the cortex 74. Therefore, the above TFR methods should be applied to both the channels, then by cascading the obtained features, one feature vector is formed.
To obtain a stable classification performance, two EEG channels are usually insufficient. In 105, a stable performance is achieved after employing more
Table 2.1 Comparison of various time-frequency decomposition methods
Abbildung in dieser Leseprobe nicht enthalten
a : TR is the abbreviation of temporal resolution;
b : FR is the abbreviation of frequency resolution;
c : CC is the abbreviation of computation complexity; d : ◦ represents applicable; • represents not applicable.
than 9 channels of EEG signal. The feature extraction methods that are employed in 105 are simple, i.e, the EEG data for each channel is only band pass filtered between 8 to 35 Hz, which results in only 1 feature per channel. If the TFR methods were used, the dimensionality of the cascaded feature vector would make the employed classification scheme complicated and over trained 106. Thus, the obtained features should be further optimized for better classification performance.
The feature optimization in BCI usually takes two different technical paths. One way focuses on the neuroscience basis to interpret the EEG activity during the BCI tasks 59. Another way relies on machine learning techniques. Especially, the methodology on feature selection and dimension reduction attracts the most attention 107. Despite the different technical aspects, the outcome of two pathways are the same. The main goal is to reduce the dimension of the feature vector while preserving the important information that is related to the task.
We first look at the BCI feature optimization from the neuroscience perspective. The tasks that are employed by BCI assumes that a stable and detectable pattern should be elicited by the cortex. As in the case of motion imagery, we known that the ERD pattern exists in the alpha band 23. However, whether it is necessary to include the whole alpha band in the feature space is debatable.
According to 108, the alpha band where the ERD lies is subject-specific. Further, in 59, the ERD patterns in a large pool of subjects were analyzed. After comparing the frequency spectrum between the motion imagery trace and resting trace, it was observed that ERD patterns actually have a smaller bandwidth compared to the whole alpha band and its presence also differs from subject to subject. The identification of this subject-specific band plays an important role in improving the performance of the BCI systems. In 74, a 3Hz band has been selected for all subjects. Similar approach has been applied to an on-line BCI system in 109.
The frequency band where the ERD pattern of a subject is significant is named as the subject-specific reactive band. Its identification leads to a exclusion of the frequency components in (2.5) and (2.10), thus the reduction in the feature dimension is achieved. Although the existence of the subject-specific reactive band is certain, its distribution among subjects remains unknown. Moreover, a systemic way of identifying the subject-specific reactive band is yet to be developed. This issue will be studied in the Chapter 3 of this thesis.
[...]
-
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X.