Small Language Models & FineTuning
Master Small Language Models and advanced fine-tuning techniques. Learn efficient model training, optimization strategies, and deployment of specialized language models for specific domains and tasks.
Introduction to small language models, architecture design, and efficiency principles
Pruning, quantization, knowledge distillation, and optimization techniques
PEFT methods, LoRA, adapters, and parameter-efficient training strategies
Model deployment, inference optimization, and real-world applications
Build a small language model from scratch for a specific domain or task
Implement compression techniques to reduce model size while maintaining performance
Develop a parameter-efficient fine-tuning system using LoRA and QLoRA
Deploy optimized models for edge devices and mobile applications
Strong understanding of neural networks, transformers, and PyTorch
Prior experience with language models, training, or our LLM course
Advanced programming skills and familiarity with model deployment pipelines
Duration: 12 weeks • Focus: Efficiency & Specialization
Build powerful, efficient language models that run anywhere.