Large Language Models (LLMs) have revolutionized natural language processing, demonstrating remarkable versatility across various tasks. However, task and domain specialization is crucial to maximize their potential and achieve higher accuracy and relevance in specific fields while remaining efficient. This talk delves into innovative domain adaptation for LLMs through three models. We begin with SaulLM, the first LLM tailored for the legal field, excelling in legal research, document drafting, and contract analysis. Next, we introduce CroissantLLM, a bilingual model proficient in French and English, enabling seamless translation and bilingual text generation with cultural and linguistic precision. Lastly, we present TowerLLM, designed for translation tasks, offering high-fidelity translations while maintaining contextual integrity. In this talk, we will showcase the methodologies and breakthroughs that enable large-scale specialization of LLMs in their respective domains/applications.