Brain-computer interfaces (BCIs) based on electroencephalography (EEG) enable direct communication between humans and computers by analyzing brain activity. Specifically, modern BCIs are capable of translating imagined movements into real-life control signals, e.g., to actuate a robotic arm or prosthesis. This type of BCI is already used in rehabilitation robotics and provides an alternative communication channel for patients suffering from amyotrophic lateral sclerosis or severe spinal cord injury. Current state-of-the-art methods are based on traditional machine learning, which involves the identification of discriminative features. This is a challenging task due to the non-linear, non-stationary and time-varying characteristics of EEG signals, which led to stagnating progress in classification performance. Deep learning alleviates the efforts for manual feature engineering through end-to-end decoding, which potentially presents a promising solution for EEG signal classification.
This thesis investigates how deep learning models such as long short-term memory (LSTM) and convolutional neural networks (CNN) perform on the task of decoding motor imagery movements from EEG signals. For this task, both a LSTM and a CNN model are developed using the latest advances in deep learning, such as batch normalization, dropout and cropped training strategies for data augmentation. Evaluation is performed on a novel EEG dataset consisting of 20 healthy subjects. The LSTM model reaches the state-of-the-art performance of support vector ma- chines with a cross-validated accuracy of 66.20%. The CNN model that employs a time-frequency transformation in its first layer outperforms the LSTM model and reaches a mean accuracy of 84.23%. This shows that deep learning approaches deliver competitive performance without the need for hand-crafted features, enabling end-to-end classification.
Contents
1 Introduction
2 Technical Background
2.1 EEG-based Brain-Computer Interface
2.1.1 Sensory Motor Rhythms and Motor Imagery
2.1.2 EEG Data Acquisition
2.2 Artificial Neural Networks
2.3 Deep Learning
2.3.1 Feedforward Neural Networks
2.3.2 Convolutional Neural Networks
2.3.3 Vanilla Recurrent Neural Networks
2.3.4 Long Short-Term Memory Networks
2.3.5 Tuning Deep Neural Networks
2.4 Related Work
3 Methods
3.1 Experimental Design
3.1.1 Data
3.2 Implementation
3.2.1 Signal Preprocessing
3.2.2 Data Augmentation
3.2.3 Long Short-Term Memory Model
3.2.4 Convolutional Neural Network Model
3.3 Results
3.3.1 Proposed Long Short-Term Memory Model
3.3.2 Proposed Convolutional Neural Network Model
3.3.3 State-of-the-Art CNN Models
3.3.4 Summary
4 Discussion
5 Conclusion
Objectives & Core Topics
This thesis aims to investigate the potential of deep learning methods, specifically Long Short-Term Memory (LSTM) networks and Convolutional Neural Networks (CNNs), to decode binary motor imagery movements from EEG signals without the need for manual feature engineering. The core research question addresses whether these deep learning architectures can overcome the stagnation in BCI performance caused by the non-linear, non-stationary characteristics of EEG data.
- Development of LSTM and CNN models for end-to-end EEG decoding.
- Investigation of data augmentation strategies, such as sliding window cropping, to mitigate limited dataset size.
- Comparative benchmarking against state-of-the-art CNN architectures and traditional machine learning methods.
- Evaluation of spectrogram-based representations for improving CNN performance on time-series EEG data.
- Assessment of potential convertibility of developed models to power-efficient neuromorphic hardware (TrueNorth).
Excerpt from the Thesis
3.2.2 Data Augmentation
Small data sets limit the ability of deep learning models to learn discriminative features and lead to overfitting. In other domains such as computer vision, data augmentation is common practice to enlarge the dataset. While methods such as stretching, compressing, rotating or flipping works well for image-like data, it is unsuitable for time-series data like EEG. Crops generated by sliding a fixed-size window over each EEG trial, however, has been shown to efficiently increase the amount of training examples, leading to a better performance of CNN models [60].
In this work, the crops are created using a sliding window with the length of 1024 timesteps, i.e., 4 seconds given the sampling frequency of 256 Hz. The sliding window shifts by n timesteps to create next crop until the end of the trial. Formally, given an original trial Xj ∈ RE·T with E electrodes and T timesteps, the sliding window generates crops Cj with size T as slices of the original trial as follows:
Cj = {Xj [1,E],[t,t+T] |t ∈ [1, T − T]}
where j is the trial index. This cropping strategy forces the deep learning model to learn discriminative features that are present in all crops of the trial because the model can no longer rely on the global structural differences between the original trials. Each crop receives the same label yk as the original trial. Choosing a small value for the shift parameter n low leads to aggressive cropping, which in turn yields more but higher correlated new training examples. A shifting parameter of n = fs/8 = 32 (i.e., 125 ms) yielded the best results regarding model performance in terms of accuracy. The crops were collected starting 3 seconds prior to motor imagery onset until the end of the trial, which guarantees that a minimum of 1 second motor imagery is present within the crop.
Summary of Chapters
1 Introduction: Outlines the challenges of BCI design for disabled users and introduces deep learning as a promising approach for EEG signal classification.
2 Technical Background: Provides the theoretical foundation regarding BCI principles, EEG data acquisition, neural network architectures, and deep learning optimization techniques.
3 Methods: Details the experimental setup, data recording, preprocessing, the architecture of the proposed LSTM and CNN models, and the evaluation strategy.
4 Discussion: Interprets the classification results, compares the proposed models with state-of-the-art literature, and analyzes the trade-offs between LSTM and CNN architectures for EEG decoding.
5 Conclusion: Summarizes the contributions of the thesis and provides outlooks on future research directions, such as hybrid models and transfer learning.
Keywords
Brain-Computer Interface, BCI, EEG, Motor Imagery, Deep Learning, Recurrent Neural Networks, RNN, Long Short-Term Memory, LSTM, Convolutional Neural Networks, CNN, Spectrogram, Data Augmentation, Signal Processing, Neuromorphic Hardware
Frequently Asked Questions
What is the primary focus of this master thesis?
The thesis focuses on developing and evaluating deep learning models—specifically LSTM and CNN—to classify motor imagery EEG signals for brain-computer interfaces, aiming to replace manual feature engineering with end-to-end decoding.
What are the central themes discussed in this work?
The work covers EEG signal characteristics, the limitations of traditional machine learning in BCI, neural network architectures (MLP, RNN, CNN), data augmentation techniques, and methods for tuning deep neural networks.
What is the primary research goal or question?
The main goal is to determine if deep learning models can effectively decode non-stationary, noisy EEG data during motor imagery tasks to improve BCI classification accuracy and potential usability in assistive devices.
Which scientific methods are utilized?
The author employs deep learning techniques including LSTM and CNN, utilizing frameworks like Theano and Keras. The methodology involves data collection (NST dataset), signal preprocessing (Notch, Bandpass filtering), and performance evaluation using stratified k-fold cross-validation.
What topics are covered in the main body?
The main body treats the technical background of BCI systems, a review of related literature, the experimental implementation of custom LSTM and CNN models, and an extensive results analysis using visualizations like boxplots and loss curves.
How would you characterize this work through keywords?
The work is characterized by terms such as Brain-Computer Interface, Motor Imagery, EEG decoding, Deep Learning, LSTM, CNN, and data augmentation.
Why was the LSTM model performance compared against the CNN model?
While the LSTM was initially chosen for its capability to handle sequential time-series data, it performed lower than expected. The CNN was introduced and tested to see if providing image-like spectrogram representations could yield superior feature extraction and higher classification accuracy.
What is the significance of the TrueHigh neuromorphic hardware mentioned?
The author considers the convertibility of the developed algorithms to IBM’s TrueNorth chip. This is significant because the chip offers high energy efficiency, making it an ideal platform for real-world, long-term use in portable brain-machine interfaces.
- Arbeit zitieren
- Juri Fedjaev (Autor:in), 2017, Decoding EEG Brain Signals using Recurrent Neural Networks, München, GRIN Verlag, https://www.grin.com/document/455235
