Cognitive-linguistic functions are an essential part of adequate communication competence. Cognitive-linguistic deficits are common after traumatic diffuse axonal injury (DAI). We aimed to examine the integrity of perisylvian white matter tracts known to be associated with linguistic functions in individuals with DAI and their eventual association with poor cognitive-linguistic outcomes. Diffusion tensor imaging (DTI) results of 44 adults with moderate-to-severe DAI were compared with those of 67 controls. Fractional anisotropy (FA) values of the superior longitudinal fasciculus (SLF), arcuate fasciculus (AF), SLF with frontal connections to the lower parietal cortex, and AF with temporal connections to the lower parietal cortex were measured using tractography. The associations between white matter integrity FA values and cognitive-linguistic deficits were studied in the DAI group. Cognitive-linguistic deficits were determined based on our earlier study using the novel KAT test. No previous studies have examined the associations between white matter integrity and cognitive-linguistic deficits determined using the KAT test. Patients with DAI showed lower FA values in all left-side tracts than the controls. Unexpectedly, the poor cognitive-linguistic outcome in the language comprehension and production domains was associated with high FA values of several tracts. After excluding five cases with the poorest cognitive-linguistic performance, but with the highest values in the DTI variables, no significant associations with DTI metrics were found. The association between white matter integrity and cognitive-linguistic functioning is complex in patients with DAI of traumatic origin, probably reflecting the heterogeneity of TBI.