Traditionally the machine learning assisted quality assessment of biomedical signals (such as electrocardiogram - ECG, photoplethysmography - PPG) have classified a signal segment quality as ”good” or ”bad” and used this assessment to determine if the segment is usable for further processing steps, such as heart beat estimation. In principle, this is a suitable approach and can be justified by its straightforward implementation and applicability. However, in the case of body sensor networks with multiple simultaneously operating units, such as IMUs (Inertial Measurement Units) there is a need to select the best performing axes for further processing, instead of processing the data among all axes (which can be computationally intensive). For a single IMU, there are already six separate acceleration and angular velocity axes to be evaluated. In this paper, instead of classifying the signal segments simply as ”good” or ”bad” quality we propose a learning to rank based approach for the quality assessment of cardiac signals, which is able to determine the relative importance of a signal axis or waveform. We illustrate that the method can generalize between multiple human experts annotated ground truths in automated best axis selection and ranking of signal segments based on their quality.