Cardiovascular diseases are considered the number one cause of death across the globe which can be primarily identified by the abnormal heart rhythms of the patients. By generating electrocardiogram (ECG) signals, wearable Internet of Things (IoT) devices can consistently track the patient’s heart rhythms. Although Cloud-based approaches for ECG analysis can achieve some levels of accuracy, they still have some limitations, such as high latency. Conversely, the Fog computing infrastructure is more powerful than edge devices but less capable than Cloud computing for executing compositionally intensive data analytic software. The Fog infrastructure can consist of Fog-based gateways directly connected with the wearable devices to offer many advanced benefits, including low latency and high quality of services. To address these issues, a modular one-dimensional convolution neural network (1D-CNN) approach is proposed in this work. The inference module of the proposed approach is deployable over the Fog infrastructure for analysing the ECG signals and initiating the emergency countermeasures within a minimum delay, whereas its training module is executable on the computationally enriched Cloud data centers. The proposed approach achieves the F1-measure score ≈1 on the MIT-BIH Arrhythmia database when applying GridSearch algorithm with the cross-validation method. This approach has also been implemented on a single-board computer and Google Colab-based hybrid Fog-Cloud infrastructure and embodied to a remote patient monitoring system that shows 25% improvement in the overall response time.