In this paper, we present Neural Physiological Encoder (NPE), a simple module that leverages decomposed convolutional filters to automatically generate effective features that can be used with a downstream neural network for blood glucose prediction.