Finetuning Transformer Models
Master the art of LLM finetuning with LoRA, QLoRA, and Hugging Face. Learn how to prepare, train and optimize models for specific tasks efficiently.
Skill level
IntermediateTime to complete
Average based on combined completion rates — individual pacing in lessons, projects, and quizzes may vary3 hoursPrerequisites
1 courseWe suggest you complete the following courses before you get started with Finetuning Transformer Models:- Intro to AI Transformers
About this course
Finetuning is an essential skill in the world of Large Language Models (LLMs), allowing you to customize pre-trained transformer models for specific tasks. This course will guide you through the practical process of finetuning using popular tools like LoRA, quantization, and QLoRA, using Hugging Face libraries and the popular Mistral series of open-weight LLMs. You’ll learn about the life cycle of finetuning, GPU usage in deep learning, and parameter-efficient finetuning (PEFT).
Skills you'll gain
Learn how to prepare training data for finetuning
Understand how to perform full finetunes using Hugging Face
Get hands-on experience with PEFT and 4-bit quantization techniques
Syllabus
3 lessons • 2 quizzesCertificate of completion available with Plus or Pro
Earn a certificate of completion and showcase your accomplishment on your resume or LinkedIn.
Earn a certificate of completion
Show your network you've done the work by earning a certificate of completion for each course or path you finish.- Show proofReceive a certificate that demonstrates you've completed a course or path.
- Build a collectionThe more courses and paths you complete, the more certificates you collect.
- Share with your networkEasily add certificates of completion to your LinkedIn profile to share your accomplishments.
Reviews from learners
- The progress I have made since starting to use codecademy is immense! I can study for short periods or long periods at my own convenience - mostly late in the evenings.ChrisCodecademy Learner @ USA
- I felt like I learned months in a week. I love how Codecademy uses learning by practice and gives great challenges to help the learner to understand a new concept and subject.RodrigoCodecademy Learner @ UK
- Brilliant learning experience. Very interactive. Literally a game changer if you're learning on your own.John-AndrewCodecademy Learner @ USA
Our learners work at
Frequently asked questions about Finetuning Transformer Models
Finetuning is the process of training a pre-trained deep learning model further using your own custom data, resulting in a customized (“finetuned”) model.
Join over 50 million learners and start Finetuning Transformer Models today!
Looking for something else?
Related resources
- Article
How to Fine Tune Large Language Models (LLMs)
Learn how to fine tune large language models (LLMs) in Python with step-by-step examples, techniques, and best practices. - Article
Prompt Engineering vs Fine Tuning: When to Use Each
Compare LLM fine-tuning vs prompt engineering for costs, complexity, and when to use each approach for your AI projects. - Article
How to Use Hugging Face: Beginner's Guide to AI Models
Learn Hugging Face fundamentals to train transformer models, tokenize text, and deploy AI with Google Colab. Complete beginner tutorial.
Related courses and paths
- Learn about the Hugging Face AI and machine learning platform, and how their tools can streamline ML and AI development.
- Beginner Friendly.< 1 hour
- Explore fine-tuning AI models like GPT-3 and 4 with OpenAI APIs. Learn to utilize the Assistants API and understand the creation and comparison of text embeddings.
- Intermediate.1 hour
- Learn how to use PyTorch in Python to build text classification models using neural networks and fine-tuning transformer models.
- With Certificate
- Intermediate.4 hours
Browse more topics
- Python4,273,222 learners enrolled
- AI2,536,836 learners enrolled
- Data science5,284,907 learners enrolled
- Machine learning800,746 learners enrolled
- Code foundations8,480,489 learners enrolled
- Computer science6,972,650 learners enrolled
- Web development5,690,504 learners enrolled
- For business4,073,541 learners enrolled
- JavaScript3,192,677 learners enrolled
Unlock additional features with a paid plan
Practice Projects
Guided projects that help you solidify the skills and concepts you're learning.Assessments
Auto-graded quizzes and immediate feedback help you reinforce your skills as you learn.Certificate of Completion
Earn a document to prove you've completed a course or path that you can share with your network.