Fine-Tuning BERT for Context-Aware Phrasal Verb Disambiguation

Abstract

Transformer-based models like BERT have significantly advanced natural language processing by enabling deep contextualized language representations. While powerful in token prediction and structural understanding, their success does not inherently imply a strong grasp of semantic meaning, particularly for ambiguous expressions. Semantic disambiguation is a core challenge, especially with phrasal verbs, whose meanings are often non-compositional and highly context-dependent, making them difficult for models trained primarily on surface-level patterns. This research addresses the gap where foundational BERT models are not optimally designed for meaning classification. We investigate whether fine-tuning BERT can improve its performance in phrasal verb meaning classification. To this end, we created a context-rich dataset focused on phrasal verb semantic interpretation, augmenting it with LLM-generated sentences for robustness. We compare a baseline bert-base-uncased model with a fine-tuned version, utilizing QLoRA for efficient training given hardware constraints. Model performance before and after fine-tuning is evaluated using a diverse set of metrics including cosine similarity, BLEU, ROUGE-L, Jaccard similarity, and METEOR. The results consistently demonstrate that fine-tuning significantly enhances the model’s ability to capture the semantic intent behind phrasal verbs, confirming the effectiveness of contextual specialization through targeted training for nuanced language understanding. The code implementation can be found at: https://github.com/Abishethvarman/bert-semantic-classification