Comparative Analysis of Transformer-Based Models for Extracting Skills from Information Technology Job Posting using Named Entity Recognition

Abstract

Skill extraction from job postings is critical for labor market analytics and curriculum design, yet it remains challenging due to unstructured text and diverse terminology. This study conducts a comparative evaluation of five Transformer-based Named Entity Recognition (NER) models—BERT, RoBERTa, DistilBERT, ALBERT, and XLM-RoBERTa—using a curated bilingual dataset of information technology job postings annotated with hard and soft skill entities. The methodology involved standardized preprocessing, BIO tagging, and a 70/15/15 train-validation-test split, with model fine-tuning carried out using optimized hyperparameters. Evaluation metrics included precision, recall, and F1-score, alongside computational efficiency measures such as training time, inference speed, and memory usage. Experimental results indicate that BERT achieved the highest F1-score (0.90–0.91), while DistilBERT delivered a favorable balance between accuracy and efficiency, and ALBERT offered moderate performance with reduced resource demands. In contrast, RoBERTa and XLM-RoBERTa underperformed in this domain-specific context. The contributions of this work include establishing a systematic benchmark for Transformer-based NER in IT skill extraction, providing insights into the trade-offs between accuracy and efficiency, and demonstrating practical implications for human resource analytics and curriculum mapping. These findings advance the understanding of model suitability for domain-specific and bilingual skill extraction tasks and lay the foundation for future cross-lingual and cross-domain applications.