In indoor environments, such as smart homes, the number of occupants within the space and their moving directions can provide a rich set of contextual information about the surroundings and occupants themselves, which can enable systems to adapt their services according to the occupants’ situation. Therefore, significant effort has been devoted to the development of variable sensing systems and learning methods. In this study, we introduce a pyroelectric infrared (PIR) sensor-based sensing system for counting moving people and detecting directions using convolutional neural networks (CNNs) and generative adversarial networks (GANs). PIR output signals were collected from four multiple-subject scenarios: single-, two-, three-, and four-subject groups in the experiments. We propose a novel time sequence sensor data augmentation algorithm, namely, auxiliary-classifier conditional GAN. This algorithm embeds the input data to reflect the condition to which the generated data should be transformed and its class information to which the generated data should be classified. The algorithm aims to build a model that works well in cases where multiple people move together (like to occur less than the cases when a single person moves alone). The experimental results show that when compared with the original model without augmentation, our multitask learning model combined with the proposed sample augmentation method increases the precision of counting moving people by 7.9%, 9.7%, 26%, and 37.5% for the one-, two-, three-, and four-subject groups, respectively, when compared with the original model without augmentation.