Finetuning Transformer Models
Master the art of LLM finetuning with LoRA, QLoRA, and Hugging Face. Learn how to prepare, train and optimize models for specific tasks efficiently.
Skill level
IntermediateTime to complete
Average based on combined completion rates — individual pacing in lessons, projects, and quizzes may vary3 hoursPrerequisites
1 courseWe suggest you complete the following courses before you get started with Finetuning Transformer Models:- Intro to AI Transformers
About this course
Finetuning is an essential skill in the world of Large Language Models (LLMs), allowing you to customize pre-trained transformer models for specific tasks. This course will guide you through the practical process of finetuning using popular tools like LoRA, quantization, and QLoRA, using Hugging Face libraries and the popular Mistral series of open-weight LLMs. You’ll learn about the life cycle of finetuning, GPU usage in deep learning, and parameter-efficient finetuning (PEFT).
Skills you'll gain
Learn how to prepare training data for finetuning
Understand how to perform full finetunes using Hugging Face
Get hands-on experience with PEFT and 4-bit quantization techniques
Syllabus
3 lessons • 2 quizzesCertificate of completion available with Plus or Pro
Earn a certificate of completion and showcase your accomplishment on your resume or LinkedIn.
Earn a certificate of completion
Show your network you've done the work by earning a certificate of completion for each course or path you finish.- Show proofReceive a certificate that demonstrates you've completed a course or path.
- Build a collectionThe more courses and paths you complete, the more certificates you collect.
- Share with your networkEasily add certificates of completion to your LinkedIn profile to share your accomplishments.
Reviews from learners
- The progress I have made since starting to use codecademy is immense! I can study for short periods or long periods at my own convenience - mostly late in the evenings.ChrisCodecademy Learner @ USA
- I felt like I learned months in a week. I love how Codecademy uses learning by practice and gives great challenges to help the learner to understand a new concept and subject.RodrigoCodecademy Learner @ UK
- Brilliant learning experience. Very interactive. Literally a game changer if you're learning on your own.John-AndrewCodecademy Learner @ USA
Our learners work at
Frequently asked questions about Finetuning Transformer Models
Finetuning is the process of training a pre-trained deep learning model further using your own custom data, resulting in a customized (“finetuned”) model.
Join over 50 million learners and start Finetuning Transformer Models today!
Looking for something else?
Related resources
- Article
Getting Started With Hugging Face
An introduction to one of the most popular new ML education and model-building resources. - Article
Getting Started with OpenAI Models
Learn how to use an OpenAI model and fine-tune it tailoring it to better fit tasks or topics. - Article
What Is Deep Learning?
A quick overview of deep learning and its applications
Related courses and paths
- Free course
Intro to Hugging Face
Learn about the Hugging Face AI and machine learning platform, and how their tools can streamline ML and AI development.Beginner Friendly< 1 hour - Free course
Using OpenAI APIs: Fine-tuning Models, the Assistants API, & Embeddings
Explore fine-tuning AI models like GPT-3 and 4 with OpenAI APIs. Learn to utilize the Assistants API and understand the creation and comparison of text embeddings.Intermediate1 hour - Course
Learn Text Classification with PyTorch
Learn how to use PyTorch in Python to build text classification models using neural networks and fine-tuning transformer models.With CertificateIntermediate1 hour
Browse more topics
- Python3,719,188 learners enrolled
- AI2,035,383 learners enrolled
- Data science4,606,613 learners enrolled
- Machine learning681,501 learners enrolled
- Code foundations7,586,805 learners enrolled
- Computer science6,048,517 learners enrolled
- Web development5,083,061 learners enrolled
- For business3,477,461 learners enrolled
- JavaScript2,917,821 learners enrolled
Unlock additional features with a paid plan
Practice Projects
Guided projects that help you solidify the skills and concepts you're learning.Assessments
Auto-graded quizzes and immediate feedback help you reinforce your skills as you learn.Certificate of Completion
Earn a document to prove you've completed a course or path that you can share with your network.