Application of image processing in recognition

Contents

Abstract

List of Figures

List of Tables ix Abbreviations

1 Introduction
1.1 Biomertics
1.2 Gait as Biometrics

2 Related Literature Review
2.1 Model Free Approach
2.2 Model Based Approach

3 Gait Recognition System
3.1 Literature used for Implementations
3.2 Database Available
3.2.1 CASIA Gait Database
3.3 Pre-processing
3.4 Quality Factor Q (Our Minor Contribution)
3.5 Feature Extraction
3.5.1 Gait Cycle Measurement
3.5.2 Silhouette Vector Measurement
3.5.3 Rectangular Features(An Experiment!)
3.6 Principal Component Analysis

4 Recognition Scheme
4.1 Testing
4.1.1 Feature Extraction of Test Sequence
4.1.2 Template Forming
4.2 Template Matching

5 Performance Evaluation 29
5.1 Introduction
5.2 Biometrics Evaluation Terms
5.3 Other Methods

6 Results and Conclusions
6.1 Results
6.2 Conclusions

7 Future Work
7.1 Future Work Plan

Abstract

Lot of research in the field of human recognition is being carried out. Gait recognition is a relatively new approach which is gaining momentum in biometrics. We have proposed a simple approach as a solution to this problem and same is presented in this seminar. We have taken a feature which was proposed earlier i.e. the Silhouette Vector. This is the distance of boundary points from the centroid of the silhouette as it rotates 360 degrees. Additional to the silhouette vector, we divided the silhouette image into three equal parts vertically and computed some statistical properties of these parts. These properties were also added to the silhouette vector and given to the PCA training system. Training was performed using silhouette and rectangular vectors for each subject. For testing the system, nearest neighbor method was used which is one of the simplest algorithms used for classification problems. The test subject is assigned to the class which is the minimum Euclidean distance from it. Inclusion of the additional features has improved the system performance greatly. Cumulative match score was used to analyze the system performance.

List of Figures

3.1 Silhouette of inferior quality

3.2 Extracted complete gait cycle

3.3 Variation in width with no. of frames

3.4 Silhouette with centroid

3.5 Variation of silhouette vector for rotating angle 10 degree

3.6 Variation of silhouette vector for rotating angle 2 degree

3.7 Division of silhouette to measure arm and leg swing

6.1 SV of length 36 and tested on seq

6.2 SV of length 30 and tested on seq

6.3 SV of length 36 and tested on seq

6.4 SV of length 180 and tested on seq

6.5 Cumulative match score without RV . .

List of Tables

3.1 Database

Abbreviations

Abbildung in dieser Leseprobe nicht enthalten

Chapter 1

In troduction

1.1 Biomertics

Biometric is a multidimensional problem which is aimed at to recognize human being.It is expected from any biometric system that it could recognize people without errors,at a distance,while people are moving and without cooperation or even they are not aware about that the system is recognizing them.Obviously today’s commercial systems can- not cope with all these requirements.Investigations are needed in many applications of biometrics like border control, surveillance in critical infrastructures and ambient intelli- gence. Biometrics plays an important role in surveillance and security, such as in access control, e-passport and watch-list surveillance. Due to increased demands for security ,significant increase in biometric recognition applications has been reported in recent years. Biometric recognition systems automatically verify or identify person identity present in the input images and videos using human biometric traits. Human biometric traits which can be used for biometric recognition include face, iris, fingerprint, palm- print, and others. Multimodal biometrics combines several biometric modules to get more reliable recognition results. Biometric recognition is done by comparing template from probe images with those of gallary images which are already saved in the database. The comparison of a probe image with gallary image is performed in one of the two modes:

- Verification (or authentication)
- Identification (or recognition)

Verification is one-to-one in that the gallary image is compared against the test images to verify the claimed ID. This is the case of boarder control with e-passport where the ID is claimed by the claimant. Identification is one-to-many in that the test image is compared against those enrolled in the database(gallary images) to determine the identity of the probe. Another one-to-many scenario is watch-list surveillance, where only the found matches that are confident enough (above a preset threshold) should be shown to the system operator.Most commercial face recognition products and solutions are developed for cooperative user applications, including access control, e-passport, and national registration like Adhar Card where a user is required to cooperate with the camera to have his/her face image captured properly along with iris and figerprints in order to be granted for the national identity.

Even though an abundance of biometric sensing technologies exist as not all are equally suited for all applications. In particular, the required level of cooperation from the user strongly constraints the applicability of these devices in some operational environments. Todays most common biometric sensing modalities fall into one of three categories:

- Biometrics which requires subject contact
- Biometrics which is contactless
- Biometrics which can recognize subject at a distance.

The difference among these three categories is the required distance between the sen- sor and the user for effectively capturing the biometric data.Contact devices require the user to actually touch the biometric sensor. Some typical contact sensors are fingerprint, palm, and signature. These devices can hardly be hidden and require an active cooper- ation of the user. Contactless devices are all devices which do not require the user to be in physically contact with the sensor.This category includes all sensors which require a short distance, generally in the order of 1 cm to 1m, for obtaining a good sample of the biometric data. Iris capturing devices and touchless fingerprint sensors are among the most common devices of this type. Even though the user can keep a distance from the sensor, still an active cooperation is required. Also some kind of face recognition systems can be included in this category because the user must be close to the camera to be recognized.

Biometric systems capable of acquiring biometric data at a distance usually do not re- quire the active cooperation of the user. Gait recognition, some face recognition systems and the most recently developed iris recognition systems fall in this category.

Multibiometric systems generally uses a fusion of contact and contactless devices to reduce the false acceptance rate thus improving the recognition performances. In other cases, multiple modalities are exploited to facilitate the data acquisition for a wider population. A notable example of multibiometric system deployment is the ‘ADHAR CARD’issued by Govt. of India which exploits face,iris and fingerprint data to determine the identity of Indian Nationality.

Whenever the users are not cooperative or a high throughput is desired, a multimodal system where all modalities are contactless or at a distance is preferable.For example, a network of cameras can be used to acquire images of persons walking through a hallway and process their faces, iris, and walking dynamics at the same time and from different angles. Also in this case, by exploiting several modalities (namely gate, face, and iris biometrics) and fusing them either at feature, score, or decision level, can greatly improve the performances of the identification system and still allows a passive or non-cooperative behavior of the users.

1.2 Gait as Biometrics

Consider the task of recognizing someone from a distance. Such scenarios arise in wide- area monitoring and asset protection. What sources of biometrics one could use? Of course, the collection of fingerprints or iris scans at such distance is implausible. It is probable that face data can be captured, but resolution and outdoor sources of varia- tions, such as sunlight and shadows, would be hard issues to overcome. So instead of physical biometrics which are direct signatures of the physiology of the person, we turn to behavioral biometrics. One such behavioral biometric is gait, or more precisely, in our context, the pattern of shape and motion in the video of a walking person.

Biometric Identification Technology can be classified as physiological and behavioral by utilizing their biometric characters. Fingerprint, iris, face, hand, ear DNA are the physi- ological biometrics and gait, voice, signature, keystroke, etc are behavioral biometrics. It can be divided into the two parts, the first generation biometrics such as fingerprint, iris and face, and the second generation biometrics such as gait recognition. Gait recognition is used to identify individuals in image sequences by the way of their walking. Different from the first generation biometrics, gait recognition has many unique advantages, such as from the distance, no-touching, lower quality video and difficult to disguise. Therefore more and more researchers are interested in it.

Definition

“Gait can be defined as the coordinated, cyclic combination of movements that results in human locomotion.”

The movements are coordinated in the sense that they must occur with a specific tem- poral pattern for the gait to occur. The movements in a gait repeat as a walker cycles between steps with alternating feet. It is both the coordinated and cyclic nature of the motion that makes gait a unique phenomenon. Examples of motion that are gaits include walking, running, jogging, and climbing stairs. While sitting down, picking up and throwing objects are all coordinated motions, but they are not cyclic and hence does not constitute gait. Jumping jacks are coordinated and cyclic, but do not result in locomotion and thus, will not be considered in gait analysis process. Everybody has a unique pattern of walking, running and it can be distinguished manually very easily if the subject is familiar or known, but it is very tough task to recognize that subject by a computer. There are some basic methods or approaches for gait recognition, viz,

1. Moving Video based gait recognition: In this approach, gait is captured using a video-camera from a distance. Video and image processing techniques are employed to extract gait features for recognition purposes. For example stride, cadence, static body parameters, etc.
2. Floor Sensor based gait recognition: In this approach, a set of sensors or force plates are installed on the floor and such sensors enable to measure gait related features, when a person walks on them, e.g. maximum time value of heel strike, maximum amplitude value of the heel strike, etc.
3. Wearable Sensor based gait recognition: In this approach, gait is collected using body worn motion recording (MR) Sensors .The MR sensors can be worn at different locations on the human body. The acceleration of gait, which is recorded by the MR sensor, is utilized for authentication.

Among these, video based approach has unique advantage that it can be captured from a distance without subject’s willingness or without any physical contact. Once the video is captured, some distinct gait features are extracted from the video. These features then saved as templates and used for the identification. The complete authentication process requires some powerful classification and recognition methods. There are three major classes of approaches for video based gait recognition, like 1) Temporal alignment based approaches, 2) Shape based approaches,3) Static parameters based approaches. Most of these approaches can be divided into two categories as model free and model based. In model free approach, features are extracted from silhouette images directly and in model based approach, the human body is modeled in some mathematical model and then some parameters are analyzed for recognition.

Biometrics Technology is broadly classified into two categories, first is physiological such as face, iris , fingerprint and another is behavioral such as gait. Gait is the manner of walking of individual. The person can be identified by the manner of walking along with some features such as height, stride length, shape of body, optical flow etc. Gait biometrics has several advantages like non-touching, at a distance, lower video quality etc. It belongs to second generation of biometrics where the earlier technologies like face, iris, fingerprint etc are belonging to first generation. Gait biometrics is having another important advantage that it does not require willingness of subject where the first generation technologies need willingness of subject to get identified. Therefore more and more researchers are interested in it.

There are two basic approaches for gait recognition,

1. Model Free
2. Model Based

In model free approach the processing is directly done on image sequences where in model based approach the human body is modeled by some mathematical means.Considerable work has been done in human gait recognition,still there are many challenges and scope to improve the system performance. Investigation is needed regarding various features which vary with individual. Correct classification rate is always being a problem with any biometric system. In gait recognition covariate factors can certainly bring down the recognition performance. Efficient methods to remove covariate factors and best discrimination among classes are required. Recognition rate for outdoor video data is low as compared to indoor video data and hence significant efforts have to be made on robust segmentation in case of outdoor video data. Development of more flexible model based method to solve the conflicts between model complexity and model descriptive capability is required. Though model free approaches are more feasible than model based, fusion of model based and model free approaches can yield better results because of increased feature space and capability to fit model for feature extraction. Most of the work that have been reported in the literature is mainly concentrated on recognizing subject which makes a fixed angle with the camera, but the more efforts have to be made to develop the system which will recognize subject from the information obtained from multi- view angles.

Covariate F actors

There are some factors which affects the human gait and consequently on recognition. They can be categorized in two types.

1. External factors: Such factors mostly impose challenges to the recognition ap- proach (or algorithm). For example, viewing angles (e.g. frontal view, side view), lighting conditions (e.g. day/night), outdoor/indoor environments (e.g. sunny, rainy days), clothes, walking surface conditions (e.g. hard/soft, dry/wet grass/- concrete, level/stairs, etc.), shoe types (e.g. mountain boots, sandals), object carrying (e.g. backpack, briefcase) and so on.
2. Internal factors: Such factors cause changes of the natural gait due to sickness (e.g. foot injury, lower limb disorder, Parkinson disease etc.) or other physiological changes in body due to aging, drunkenness, pregnancy, gaining or losing weight and so on.

Chapter 2

Related Literature Review

We divided the literature overview in two categories, model free and model based re- spectively.

2.1 Model Free Approach

In this approach a binary silhouette is obtained first by background subtraction tech- nique. Some distinct features are then to be extracted like static and dynamic informa- tion of moving subject. An initial approach was done by Johnson [1] using moving light display (MLD) on the subject’s joints. The aim was to study the visual information from some typical motions patterns in the human body. The parametric eigen space representation for efficient image sequence comparison is proposed in [3]. They apply this idea to the recognition of people by their walk and to the lip reading problem. They extended the idea of eigen space to image sequence recognition. Authors [4] re- viewed the construction of a binary motion-energy image (MEI) which represents where motion has occurred in an image sequence then they generate a motion-history image (MHI) which is a scalar-valued image where intensity is a function of recency of motion. Taken together, the MEI and MHI can be considered as a two component version of a temporal template, a vector valued image where each component of each pixel is some function of the motion at that pixel location. These view-specific templates are matched against the stored models of views of known actions. P. S. Huang, C. J. Harris and M.S. Nixon [5] described the gait recognition by parametric canonical space. In this article, they proposed an approach which combines EST with CST for feature extraction from each image template. This method can reduce the data dimensionality and optimize the class separability of different gait sequences simultaneously. Myoelectric signals (or EMG) which are the measurement of the electrical activity in a muscle were used as features in [7] for gait analysis of an athletic. In this approach the classification and recognition was done using ANN.

A novel methodology for the efficient analysis of walking silhouettes is proposed [12] which can be used in a gait recognition system. The proposed system’s efficiency depends on an angular transform which calculates a metric of the silhouette in angular slices of various orientations with respect to the center of the silhouette. A new approach combining the model-free approach with the model-based approach for analyzing and extracting human gait is presented in [15]. The gait period is estimated by analyzing the variation of silhouette width and height. The static features extracted from human shape include body height, body width, stride length, etc. and kinematic information of gait is represented by joint angles.Study of the methods proposed in literatures reveal that PCA can successfully represent movement data in a low-dimensional space. The authors of [17] therefore used the low-dimensional manifold defined by the first few principal components to perform a second stage of principal component analysis that describes deviations in the manifold across individuals or types of gait. A paper [20] gives survey about some existing methods and approaches for gait recognition. In [22] for gender recognition various features like head, arm, trunk, thigh, front-leg, back-leg, and feet were extracted by strong segmentation method and SVM classifier was used for classification. Analysis shows the contribution of various features in recognition.

A novel algorithm of gait recognition is illustrated in [23]. It is a 3D approach for gait recognition where 3D silhouette contour called as stereo silhouette vector (SSV) is extracted as a feature. The LMS (Least Median of Squares) method is used for re- trieving the background image from image sequences captured by a stereo vision system then binary connected component analysis is used to extract a single-connective moving object. A contour of moving object can be accomplished by canny edge detection oper- ator. Stereo match on contour is performed in order to get the disparity of the pairs of stereo image. K-L transform is used for dimension reduction and classification based on nearest neighbor was done. In a recent paper [24], it is shown that how computational intelligence is used in gait research. Occlusion is always a challenge in gait recognition, Lee09 describes a missing data theory for coping with occlusion of body part. In hong09 GEI and MSI are extracted as feature by partitioning gait cycle and nearest neighbour classification is done for recognition. One of the papers boyd05 gives an overview of the factors that affect both human and machine recognition of gaits and uses of gait analysis beyond biometric identification. Another paper [27] discusses silhouette-based feature descriptor. Human silhouette geometry is generated by boundary tracking approach and resampled to a normalized format. Boundary-centroid distance is proposed to describe gait modality. Then, it applies wavelet transform to boundary centroid distance, and extracts wavelet descriptor. At the same time, system obtains the human skeleton model and extracts body’s dynamic parameters to express gait modality. Multiple feature fu- sion and multiple views fusion are carried out and the recognition results demonstrate that the performance of multiple features and multiple views recognition is better than any single feature and single view recognition. One of the relevant papers [28] provides the first demonstration of shadow biometrics.

One more interesting paper [29] presented a new gait identification and authentication method based on Haar wavelet and Radon transform. This method consists of two stages, gait modelling and recognition. In the first stage, images extracted from video sequences are pre-processed into binary silhouette. These sequences then divided into 4 states in terms of gait cycles. The horizontal and vertical features are acquired by Haar wavelet, and then feature vectors are obtained respectively by Radon transform. In the second stage, probe sequences are fed. After feature transform of image sequence, the value of similarity can be obtained by comparing probe vectors with gallery ones and op- timized to give gait recognition. One recent paper [30] describes a method for human gait recognition using Generalized Regression Neural Networks. The feature space is com- posed of a combination of dynamic (time-varying) gait signals and static body-shape parameters, extracted from binary silhouettes obtained after background subtraction from human gait sequences. The inputs to the neural network are obtained by per- forming Discrete Cosine Transform (DCT) on the feature space, followed by selection of transformed coefficients to construct compact vectors. Focusing on the application of Intelligent Surveillance, one of the existing research works [31] proposes a new approach in which fusion of face and gait is used for human recognition at a distance in video sequences. Hidden Markov Models and fisher faces method are primarily applied for gait and face classifier, respectively and the results obtained from the two classifiers are utilized and integrated at match score level.

The recent paper [32] propose a novel method that generates dynamic and static feature templates of the sequences of silhouette images called Dynamic Static Silhouette Tem- plates (DSSTs). Here the DSST is calculated from Gait Energy Images (GEIs). DSSTs capture the dynamic and static characteristics of gait. The experimental results show that this method overcomes the issues arising from differing clothing and the carrying of objects. Human gait is cyclic in nature and this characteristic exhibits itself in cyclic appearance changes in the images when taken from a side view. In one of the works [33], they extracted the gait signatures from the silhouette database. The area of the lower half of the silhouettes is calculated in each frame of the video sequence. Then it takes the I-D Discrete Wavelet Transform (DWT) of this area signal. Statistical fea- tures are extracted from low frequency and high frequency sub-bands to form the gait signature. This proposed feature is low dimensional and hence the recognition stage is computationally very efficient. In [34]. Factorial HMM and Parallel HMM is used for Gait Recognition where wavelet feature and frieze feature are extracted using FHMM.

In one of the most recent papers [36], author proposes a gait recognition method that the subject can walk at an arbitrary angle. The gait is detected through background subtraction technique. The contour is represented by a novel approach which includes not only the spatial body contour but also the temporal information. To prove be independent of view angle, the relationship model of the walking azimuth angle and the reference view has been modeled. It transforms the walking angle to the most similar canonical views angle. Three canonical views angles are adopted in this paper. In another recent paper [37], author presents a gait recognition system based on the fusion of multiple gait cycles using a new gait representation. First, a gait sequence is automatically partitioned into multiple gait cycles by finding the local minima of width signal. After gait cycle partitioning, system extract a new gait feature called motion contour image (MCI) that captures the contour of the binary silhouette image of a walking individual. Finally, for human identification, the outputs of nearest neighbor classifiers are fused at a decision level based on majority voting.

A novel gait recognition algorithm based on fuzzy principal component analysis (FPCA) for gait energy image (GEI) is proposed in [38]. Firstly, the original gait sequence is pre-processed and gait energy image is obtained. Secondly, the eigen values and eigenvectors are extracted by fuzzy principal component analysis, which are called fuzzy components. Then the eigenvectors are projected into lower-dimensional space. Finally, the NN classifier is utilized in feature classification. In a recent research a new gait recognition algorithm, the layered time series model (LTSM) [39], is proposed. LTSM is a two-level model which combines the dynamic texture model (DTM) and the hidden markov model (HMM). A gait cycle is divided into several temporally adjacent clusters and gait features of each cluster are modeled by the DTM. The HMM is built to describe the relationship among the DTMs, which are regarded as hidden states. Most recent paper [40] presents a new method for viewpoint independent gait biometrics. The system relies on a single camera, does not require camera calibration, and works with a wide range of camera views. Another paper [41] attempts to use ground reaction force (GRF) of continuous footfalls during at least one gait cycle for subject recognition. A simple and effective approach for gait recognition based on stereo vision is proposed in another interesting paper [42]. In this method, subject in gait sequences captured by stereo vision are detected and matched to produce stereo silhouette vector (SSV). Then stereo gait feature (SGF) is extracted for analysis and recognition. Authors of [43] present the gait recognition scheme of KPCA based on mean gait energy image (MGEI), which is a robust method. First, the MGEI is calculated from gait cycle. Then KPCA can make use of the high correlation between different MGEI for feature extraction by selecting the proper kernel function, and euclidean distance of covariance weighted reciprocal is designed as the classifier.

A new type of study is proposed in [44] which is a gait based ethnicity classification sys- tem. Another paper [45] proposes a novel scheme for the integration of two or even more soft biometric traits in a biometric recognition system using a stochastic framework. In particular, the ‘height’ soft biometric trait along with the ‘stride length’ are utilized to augment the information obtained by a gait recognition system and to ultimately advance its performance. If the intrinsic subspace dimension is either overestimated or underestimated, the recognition rate will decline. To overcome these problems, authors [47] propose to combine super resolution with multilinear tensor-based learning without tuning parameters (SRMTP). They first partition the gait frames of the training and test sets into a collection of training gait image patches and test gait image patches, respectively. They utilize manifold sampling techniques to remove redundant patches from the training set. Then, they apply super resolution with neighbor embedding to learn the high resolution counterpart of low resolution test gait images, followed by a back projection to make the low resolution and high resolution gait image pairs more consistent. Then, they tensorize both the training gait sequences and the learned high resolution test sequences. Finally linear discriminant analysis (LDA) is used further to improve the performance of the gait recognition algorithm. Another paper [48] presents gait recognition approach using a wearable motion recording sensor which is attached to the leg. This approach is referred to as WS (wearable sensor) based gait recogni- tion. In [49] correspondence-free motion features are extracted and finally KNN is used for recognition. In [52] Dynamic Time Warping (DTW) is used to generate dynamic template and finally depending on DTW distance recognition is done.

[...]