The Open Cybernetics & Systemics

: A novel approach of palmprint recognition using image reconstruction based on double DBNs (IR-DDBN) was proposed in this study, as principal component analysis (PCA) ignores the higher order statistics in feature extraction. Three main steps were involved in the algorithm. Firstly, whitening PCA was utilized to extract the prominent characteristics of the original palmprint image. The second step included reconstructing the original image and calculating the residual image for the residual features between the original and reconstructed images. Finally, the double DBNs were used for classification. The experimental results demonstrated better performance of the proposed algorithm by comparing with the traditional algorithms (PCA, LBP, HOG and DBN) with higher recognition rates, especially for relatively small training samples


INTRODUCTION
Palmprint identification is one of the key technologies in biometrics to identify a person based on palmprint images, containing creases, wrinkles and ridges, with advantages of low intrusion and easy use, etc [1].A typical recognition system is composed of five parts, including palmprint scanners, preprocessing, feature extraction, matcher and database [2].As the most important stage, feature extraction has an impact on the identification result directly.The numerous feature extraction techniques of palmprint recognition can be broadly classified into line-based approaches, subspacebased approaches, statistical approaches [2].Some researches supplemented appearance-based, texture based and hybrid approaches [3].Principle Component Analysis (PCA) [4] and Independent Components Analysis (ICA) [5] are the classical tools in subspace-based methods.Recently, deep learning was applied in palmprint recognition.Deep scattering convolutional network whose architecture and filters were predefined wavelet transform provided promising result [6].In our previous work, we also found that the appropriate DBN model can enhance the performance of palmprint recognition in accuracy [7].
PCA, one of the most popular appearance-based techniques, was applied in palmprint identification for Eigenpalm extraction by Lu et al. [4].Following this, the two-dimensional PCA, and the integration of PCA and other approaches have been proposed to improve the recognition performance [8 -10].The above PCA approaches decomposed the feature space orthogonally into principle components and residual components, in which the former parts were chosen as the projection matrix.However, the discarded data of residual space still kept the information for classification in the study of joint space theory [11], and were successfully applied in the reconstruction-based face recognition [12,13].Although manifold-based reconstruction [14] and sparse reconstruction [15] both presented better performance to capture the reconstruction residual/errors in the images, the calculation procedure is more complicated and timeconsuming than the subspace approach.Moreover, the palmprint images in our study do not involve the works of image occlusion and completion.So PCA reconstruction is efficient and feasible for our research.
Nevertheless, both the PCA principal space and the residual space cannot well solve the problem of training samples in an efficient way.With the decrease of training samples, PCA can not get efficient features for discrimination.Deep learning is a subfield of machine learning which attempts to learn high-level abstractions in data by utilizing hierarchical architectures [16].It allows computational models that are composed of multiple processing layers to learn representations of data with multiple levels of abstraction [17], which has made achievements in image recognition [18 -20], voice recognition [21 -23], natural language processing, etc. Various deep learning models were extensively investigated to enhance the learning performance.Deep learning concept was incorporated into robust information theoretic framework to reduce the uncertainties in general discriminative data representation tasks [24].A deep transfer metric learning method was as well proposed to learn a set of hierarchical nonlinear transformations for cross-domain visual recognition by transferring discriminative knowledge from the labeled source domain to the unlabeled target domain [25].It was reported that the concepts of fuzzy learning into deep learning can overcome the shortcomings of fixed representation [26].Furthermore, Multimodal deep learning was used to train deep networks for a shared representation between modalities and evaluate it on a unique task [27].A hybrid convolutional network-Restricted Boltzmann Machine (ConvNet-RBM) model was proposed for face verification to learn high-level relational visual features with rich identity similarity information [28].
An unsupervised layered training algorithm based on Deep Belief Network (DBN) was proposed by Hinton in 2006 [29], called as "the third generation of network" [30].DBN can well simulate layered visual processing system by learning the abstract features of different layers from up-to-down unsupervised learning and bottom-to-top supervised learning without manual selection during in the whole training procedure [29,31].Double DBNs in which two individual DBNs were combined in the last layer to eliminate the ambiguity was proposed in an object recognition and pose estimation system [32].
Taking together both procedures, we developed a new algorithm for palmprint recognition using image reconstruction based double DBNs (IR-DDBN), in which the features of the prominent component space and residual space, extracted by the former PCA image reconstruction, were trained by two separated DBNs with unsupervised learning and combined at the last layer for supervised learning and classification.Three main steps were involved in the algorithm.Firstly, whitening PCA method was utilized to extract the principal components characteristics of ROI palmprint image.Secondly, reconstructing the original image and calculating the residual image for the residual features between the original and reconstructed images.Finally, the double DBNs were used for classification.
The rest of this paper is organized as follows: Sections 2 and 3 introduce image reconstruction based on PCA and double DBNs, respectively.Section 4 presents the proposed algorithm based on image reconstruction and double DBNs.Experimental results are given in Section 5.

IMAGE RECONSTRUCTION BASED ON WHITENING PCA
Different from the traditional PCA, in which the discarded higher-order statistical components still contributes to image classification, image construction based on whitening PCA can utilize both principal features and higher-order features [13].Generally, image reconstruction based on whitening PCA can be divided into two stages, whitening PCA and image reconstruction.

Whitening PCA
PCA is one of the most commonly used unsupervising learning methods for dimension reduction.Given n images of m classes i 1 ,i 2 ,……,i n (arranged to column vector), the average palm is

(1)
The corresponding covariance matrix is The eigenvalues of C can be sorted in a descending order, and the eigenvectors corresponding to the former eigenvalues [λ 1 , λ 2 , ........, λ r ] are chosen when the dimension of transformed data is required to reduced to r-dimension.Generally, W n× r composed of the projection vectors W 1 ,W 2 ,……, and W r is referred to as projecting matrix.Although PCA can reduce the dimension of the data effectively; the relevance of the neighborhood cannot be changed.Regulation the extracted feature by whitening can not only keep the same covariance of all the features, but also reduce the relevance of the data [13]. Suppose ,then the whitening process can be represented as follows [13] (4) Where x and y correspond to a given image and the whitened image, respectively.

Image Reconstruction
The reconstruction of x can be obtained according to Eq.4 in Section 2.1 as follows [13]: The residue error is the difference of the original image and reconstruction image:

DOUBLE DBNs
In 2006, Hinton from the University of Toronto proposed an unsupervised training algorithm based on Deep Belief Network (DBN), a probabilistic generative model superimposed by multi-layer Restricted Boltzmann Machine (RBM), for more abstract higher-level representation by arranging the lower-layer features [29 -31].The architecture of double DBNs is shown in Fig. (2), where two individual single DBNs can be combined in the last layer for supervised classification, so as to obtain more information of an object [32].Different kinds of images or features can be trained as the input of visible-layer of DBN 1 and DBN 2 , respectively.In that followed, the output of hidden previous-layer RBM became the input of the next-layer RBM.Finally, the output of the whole DBN was the output of the last-layer RBM, in which a logistic regression layer was superimposed for classification.

PALMPRINT RECOGNITION BASED ON IMAGE RECONSTRUCTION AND DOUBLE DBNs
In order to make full use of residue error of whitening PCA and explore more robust features in deep learning, we proposed a palmprint recognition method using image reconstruction and double DBNs in this paper.Having decomposed the palmprint images onto PCA principal space and residue error space, two individual DBNs were used to train the features, respectively.Then the integration in the last layer for supervised learning to obtain the labels of the input data were obtained by the classification of BP network regression and Softmax regression [31].Hence, more intrinsic features were trained effectively for palmprint recognition.The flowchart of the algorithm is illustrated as Fig.
(3), and accordingly the concrete procedure of the algorithms can be described as follows.
Firstly, whitening PCA method was utilized to extract the principal components characteristics of ROI palmprint image.Palmprint ROI images preprocessed by alignment and normalized to a size of 128x128 were divided into training set and test set.Each training sample was stored in row to constitute the training space.Projection matrix W 1 can be obtained by Whitening PCA (see Section 2.1).We set the dimension feature 100, each training sample and test sample can be projected onto the projection matrix to yield feature vector F 1 and Test 1 , respectively.
Secondly, reconstruct the original image to get the residual features.The residual error can be obtained by calculating the difference between original image and reconstructed image.Then transform the residue error space composed of training samples with whitening PCA to get the projection matrix W 2 .Project the residue error image of training samples and test samples onto W 2 to obtain feature vector F 2 and Test 2 , respectively.
Finally, the double DBNs are applied for classification.Input the two groups of features, Principal Feature1 [F 1 , Test 1 ] and Residue Feature2 [F 2 , Test 2 ] to DBN 1 , DBN 2 for independent training, respectively, and combine them in the upper-layer for supervised regressive training, and eventually get the recognition results.

EXPERIMENTAL RESULTS
In order to testify the effectiveness of the algorithm, we conducted 2 groups of experiments.One group was used to test the performance of single DBN and double DBNs with principal components and residue components, the other group was used to compare the performance of the proposed algorithm with other classical approaches.The Beijing Jiaotong University (BJJTU) Palmprint Database contains 3460 palmprint images of 292×413 pixels of 72 dpi from 346

Comparison of Recognition Performance of Traditional DBN and Double DBNs
In this group, we compared the running time and recognition accuracy of single DBN with principal components (DBN(P)) and residue components (DBN(R)) and image reconstruction based double DBNs (IR-DDBN) taking advantages of both components of whitening PCA.Both DBN and double DBNs consisted of two layers of RBM, and the numbers of DBN and double DBNs were not set too high for an efficient calculation.The numbers of 4 hidden layer units of DBN were 100,500,100,and 100, respectively.The numbers of 4 hidden layer units of double DBNs were 100,500,200,and 100 (Fig. ( 2)), respectively.We set batchsize 100 in the experiments because at least one image of each class was required in every batch processing [31].The iteration times and learning rate were set 40 and 0.0003, respectively, according to our previous study [7].
In order to reduce the total running time, 1000 palm images of 100 classes were chosen randomly from BJJTU Palmprint Database in the experiments.The training samples and test samples were both 500, i.e., each class has 5 training samples and 5 test samples, respectively.Owing to the recognition accuracies trained by DBN model varied each time, the average of 10 outputs was used to represent the experimental results.(Table 1) listed the recognition accuracy (%) and running time(ms) of traditional DBN and double DBNs based on image reconstruction in PCA space and residue error space, where DBN(P) and DBN(R) represent the recognition rate of PCA and residue error space, respectively.T 1 ,T 2 and t were the training time, test time and total time, respectively.As can be seen from Table 1, the recognition accuracy of DBN(P) was 92.88%, 44.36% higher than that of DBN(R) (48.52%), suggesting that whitening PCA prominent space extracts most information of the image, on the contrast, residue error space only kept partial discriminant information.The recognition accuracy of double DBNs reached 95.56%, 2.37% higher than that of DBN(P).The improvement was originated from the integration of PCA space and residue error space, which can not only extract the PCA space feature but also keep the residue error for discrimination.The total time consumption of IR-DDBN (22.6550ms) and DBN(R) (20.1298ms) was greater than DBN(P) (13.0549ms) because the image reconstruction and second whitening PCA training based on the residue space.Compared with DBN(R), the total time of IR-DBN based on image reconstruction was slightly longer for more features were trained in the model.

Performance Comparisons with Other Classical Approaches
We compared the proposed IR-DDBN with some methods, such as PCA, histogram of gradient (HOG) [33,34], and local binary (LBP) and convolutional neural networks(CNN) [35] using different number of training samples and classes.HOG was proposed for human detection [33], and was applied block based HOG for multispectral palmprint recognition [34].The feature dimension of PCA was 100, and Euclidean distance and nearest classifier were used for classification.The block size of HOG was 3x3.The block size of LBP was 2x2, the histogram of LBP was used for identification.In CNN model (codes available from https://github.com/rasmusbergpalm/DeepLearnToolbox), the learning rate, batchsize and epoch iteration times were set 2, 50, and 300, respectively.In the 6 feature maps (size 28×28), each neural cell is connected with the input of 5×5 neighborhood.The architectures and parameter settings of traditional DBN and IR-DDBNs were the same as those of above experiments in Section 5.1.
(Table 2) listed the identification performance on BJJTU Palmprint Database.The experiments were executed on different classes of 100, 150, and 200 with different numbers of training samples 3,5,7 per class for training, and the rest images per class were used for validation.100 classes were taken for example, the training set contained 300, 500 and 700 images, respectively, then the corresponding test set contained 700, 500 and 300 images, respectively.The corresponding ratios of training/validation were7/3, 5/5 and 3/7, respectively.As observed from the table, the recognition accuracies of all the approaches increased with the number of training samples.The enhancement was mainly due to more discriminant information learned from more training samples.IR-DDBN outperformed other classical methods and traditional DBN based on prominent components DBN(P) and residue components DBN(R), especially when the training samples are 3 and 5.The experimental results suggested that the proposed approach can extract more sufficient information of whitening PCA and residue components when the number of training samples is not abundant.Another factor contributed to the improvement comes from the learning ability of DBN as compared with other shallow learning methods.With the number of the classes increase from 100 to 150 and 200, the overall recognition accuracies of all approaches decreased.The identification accuracies of IR-DDBN were 89.27%, 87.15%, and 84.38% when the test classes were 100, 150 and 200, using 3 training samples per class.However, the recognition performance of the proposed algorithms was still superior to PCA, HOG, LBP and traditional DBN approaches.LBP, HOG and PCA showed relatively lower recognition rates than the proposed approach, especially when the training samples are 3 and 5, suggesting that DBN can extract the substantive feature from a relatively small number of training samples.Other DBN approaches can not exceed the proposed method for they did not use the whitening PCA principal components and residue components in the same time.The inferior identification performance of CNN as compared with IR-DDBN may come from the complicated optimizing of parameter setting for CNN requires excessive training to explore.To sum up, the identification performance of the proposed IR-DDBN outweighed the other approaches in the table when the numbers of training samples and test classes varied.
(Table 3) listed the identification performance of IR-DDBN, CNN, PCA, HOG and LBP executed on PolyU Palmprint Database.Higher identification accuracies than those on BJJTU Palmprint Database may come from the image quality acquired in different circumstances.The identification accuracies of IR-DDBN were 99.44%, 99.25%, and 98.75% when the number of classes were 100, 150 and 200, using 3 training samples per class.When the training samples were 5 and 7 per class, the identification accuracies of IR-DDBN exceeded 99%.The outperformance of IR-DDBN as compared with other algorithms on different databases witnessed its robustness in palmprint identification.

CONCLUSION
In this paper, we proposed an algorithm based on image reconstruction and double DBNs for palmprint recognition.The features extracted by whitening PCA based on original image and residue space were trained by two separated DBNs and integrated in the last layer for classification.The experimental results witnessed the effectiveness of the algorithm, which proved that the double DBNs based on image reconstruction can not only extract the effective features from a relatively small number of training samples, but is also benefited from the relatively complete information from whitening PCA space and residue error space.

( 6 )
Figs. (1a, b and c) correspond to a palmprint ROI image, the reconstruction image, and the residue error image (PCA dimension 100), respectively.
palms by a small-scale image scanner, each has 10 samples.The central part of 128×128 pixels cropped from original palms constitutes a ROI palmprint database (see Fig. (4)).Another Palmprint Database of Hong Kong Polytechnic University (PolyU), available from http://www.comp.polyu.edu.hk/~biometrics, was used to test the robustness of the algorithm.PolyU Palmprint Database contained 8000 palmprint images from 800 palms, and the ROI image size is 128×128.All the experiments were executed on a computer system of Intel(R)Core(TM) 3.20GHz and 4GB RAM with Matlab R2011a.
a) Original palmprint image b) ROI image