Breast cancer is one of the most common cancers in women worldwide, and its early diagnosis is essential for successful treatment. However, diagnosing breast cancer from MRI images is challenging because of the similarities with other benign breast conditions. Our study proposes a straightforward deep learning model and evaluates its diagnostic performance using transfer learning and various machine learning techniques. Our approach can extract distinctive graphical features of breast cancer and classify them, enabling early clinical diagnosis before the pathological test. Our deep learning architecture for transfer learning is a simple modification of 5 or 11 new layers on the pretrained CNNs of ImageNet, which resulted in the highest test accuracy (97.30%), F1 score (0.9730), AUC (0.9732), and kappa value (0.946) after training. We also used this architecture for feature extraction and studied the performance of various classifiers, achieving the highest test accuracy (97.97%) with k-NN. Furthermore, we compared multiple CNNs and machine learning models for their potential in breast cancer detection and proposed a faster and automated disease detection methodology. We found that smaller and memory-efficient architectures are just as good as deep and heavy ones at predicting breast cancer. DenseNet and VGG architectures were the overall best for this task. In conclusion, our study proposes an efficient and accurate method for diagnosing breast cancer using deep learning and machine learning techniques. Our approach can save valuable time for disease management and control, enabling early clinical diagnosis before the pathological test. Our findings provide insights into the potential of various CNNs and machine learning models for cancer detection, paving the way for faster and automated disease detection methodologies.