Abstract: As a kind of physical signals that could be easily acquired in daily life, photoplethysmography (PPG) signal becomes a promising solution to biometric identification for daily access management system (AMS). State- of-the-art PPG-based identification systems are susceptible to the form of motions and physical conditions of the subjects. In this work, to exploit the advantage of deep learning, we developed an improved deep convolutional neural network (CNN) architecture by using the Gram matrix (GM) technique to convert time-serial PPG signals to two-dimensional images with a temporal dependency to improve accuracy under different forms of motions. To ensure a fair evaluation, we have adopted cross-validation method and “training and testing” dataset splitting method on the TROIKA dataset collected in ambulatory conditions. As a result, the proposed GM-CNN method achieved accuracy improvement from 69.5% to 92.4%, which is the best result in terms of multi-class classification compared with state-of-the-art models. Based on average five-fold cross-validation, we achieved an accuracy of 99.2%, improved the accuracy by 3.3% compared with the best existing method for the binary-class.