The existence of incompatible measurements is a fundamental phenomenon having no explanation in classical physics. Intuitively, one considers given measurements to be incompatible within a framework of a physical theory, if their simultaneous implementation on a single physical device is prohibited by the theory itself. In the mathematical language of quantum theory, measurements are described by POVMs (positive operator valued measures), and given POVMs are by definition incompatible if they cannot be obtained via coarse-graining from a single common POVM; this notion generalizes noncommutativity of projective measurements. In quantum theory, incompatibility can be regarded as a resource necessary for manifesting phenomena such as Clauser-Horne-Shimony-Holt (CHSH) Bell inequality violations or Einstein-Podolsky-Rosen (EPR) steering which do not have classical explanation. We define operational ways of quantifying this resource via the amount of added classical noise needed to render the measurements compatible, i.e., useless as a resource. In analogy to entanglement measures, we generalize this idea by introducing the concept of incompatibility measure, which is monotone in local operations. In this paper, we restrict our consideration to binary measurements, which are already sufficient to explicitly demonstrate nontrivial features of the theory. In particular, we construct a family of incompatibility monotones operationally quantifying violations of certain scaled versions of the CHSH Bell inequality, prove that they can be computed via a semidefinite program, and show how the noise-based quantities arise as special cases. We also determine maximal violations of the new inequalities, demonstrating how Tsirelson's bound appears as a special case. The resource aspect is further motivated by simple quantum protocols where our incompatibility monotones appear as relevant figures of merit.