Correctly identifying emotions of other species is central to the welfare of animals in our care; yet, the factors underlying variation in our ability to recognise animal emotions remain unclear. Here, we investigated the human ability to recognise emotional valence and arousal in zoo-living Barbary macaques, tigers, and markhors, in short video clips from which contextual information and other clues were removed. Visitors at Korkeasaari Zoo were recruited to rate valence and arousal, and their ratings were analysed for correct identification of positive vs. negative valence and low vs. high arousal. Overall, arousal was more accurately recognised than valence, low arousal more accurately than high arousal, and negative valence more accurately than positive valence. Moreover, recognition accuracy varied among species being rated. Valence was recognised most accurately in macaques, arousal most accurately in markhors, and tigers’ emotional states were recognised the least accurately, both for valence and arousal. The results suggest that while we can recognise non-domesticated species’ emotional states, accuracy varies depending on the species and the emotional state in question, which highlights the importance of training in assessing animal welfare. Overall, considering animal welfare, it is crucial that we improve education for the identification of animal emotions, both positive and negative ones.