The rapid expansion of natural language processing (NLP) and the widespread adoption of large language models—including Bidirectional Encoder Representations from Transformers (BERT), Generative Pre-trained Transformer (GPT), and Large Language Model Meta AI (LLaMa)—have intensified global concerns about energy consumption and computational sustainability. Data centers consume a significant and growing share of global electricity and AI workloads are projected to substantially increase energy demands in coming years. Training large transformer models generates considerable carbon emissions, raising critical questions about the long-term scalability of current AI paradigms. Spiking Neural Networks (SNNs) offer a biologically inspired pathway toward energy-efficient NLP, utilizing event-driven processing and temporal coding to approach the extraordinary efficiency of the human brain's remarkably low power consumption. This paper surveys the interface of SNNs and modern NLP systems, with a specific focus on spiking architectures, learning algorithms, and neuromorphic hardware implementations. We examine spiking architectures, learning rules, and neuromorphic hardware, analyzing how key transformer components—attention mechanisms, tokenization strategies and pre-training objectives—can be adapted to the spiking domain while maintaining linguistic capability. The work critiques recent advances in neuromorphic language processing, including spike-based token representations, temporal sequence modeling, and energy-aware training protocols. We critically assess the salient limitations facing spiking language models, including training complexity, memory scalability constraints on neuromorphic hardware, and temporal encoding problems for discrete language tokens. Furthermore, we identify pivotal research directions: hybrid spike-rate coding schemes, biologically plausible learning rules, hardware-software co-design strategies, and standardized benchmarking frameworks. By bridging brain-inspired computation with state-of-the-art language models, this work provides a comprehensive roadmap toward sustainable and energy-efficient AI systems for natural language understanding.