Relatively few X-ray sources are known that have low-mass galaxies as hosts. This is an important restriction on studies of active galactic nuclei (AGNs), hence black holes, and of X-ray binaries (XRBs) in low-mass galaxies; addressing it requires very large samples of both galaxies and X-ray sources. Here, we have matched the X-ray point sources found in the XXL-N field of the XXL survey (with an X-ray flux limit of  ∼6 x 10^-15 erg s^-1cm^-2 in the [0.5-2] keV band) to galaxies with redshifts from the Galaxy And Mass Assembly (GAMA) G02 survey field (down to a magnitude limit r = 19.8) in order to search for AGNs and XRBs in GAMA galaxies, particularly those of low optical luminosity or stellar mass (fainter than M_r = -19 or M_*  ≲ 10^9.5 M_⊙). Out of a total of 1200 low-mass galaxies in the overlap region, we find a total of 28 potential X-ray source hosts, though this includes possible background contaminants. From a combination of photometry (optical and infrared colours), positional information, and optical spectra, we deduce that most of the ≃ 20 X-ray sources genuinely in low-mass galaxies are high-mass X-ray binaries in star-forming galaxies. None of the matched sources in a low-mass galaxy has a BPT classification as an AGN, and even ignoring this requirement, none passes both criteria of close match between the X-ray source position and optical galaxy centre (separation <= 3 arcsec) and high [OIII] line luminosity (above 10^40.3ergs^-1).