Advances in artificial intelligence have facilitated the automated grading of histopathology slides. Yet, the magnification of whole slide scanners (WSS) has restrained the accuracy of patch-based grading. In this work, we found that augmentation can significantly promote grading performance under this issue, even when the data volume is large (>140 K). With augmentation and a multi-dilation model, the CovXNet, we yielded a Balanced Accuracy of 92.13%, which is the current highest for the Breast Histopathology Dataset (40X magnification) also the first time both sensitivity and specificity >90%. However, in this focused grading task, augmentation only improves models with high invariance (the CovXNet and BCA-CNN). Pre-trained ResNet has lower invariance in this task, but fine-tuning can significantly improve both accuracy and invariance. For the CropNet attention model, adapting with max pooling but not augmentation offers promotions. Additionally, this work also found two types of common errors in high-starred codes, when using random.shuffle for data-label composited array, or the integrated shuffle function of ImageDataGenerator, which fake a higher accuracy by masking class 0 as class 1. Using Sklearn.shuffle instead is safer. All codes are available on our GitHub.