What is web scraping?

Web scraping is a way for programmers to learn more about websites and users. Sometimes you'll find a website that has all the data you need for a project — but you can't download it. Fortunately, there are tools like Beautiful Soup (which you'll learn how to use in this course) that let you pull data from a web page in a usable format.

What does web scraping do?

There's a lot of information on the internet, but it's not all easily accessible. For example: manually pulling listing prices from eBay would take a ton of time and effort. But with web scraping, you can write a program that can automatically request web pages from the internet, search them for specific terms or HTML elements, and store the contents.

What kind of jobs use web scraping?

Data Scientists and Analysts can use web scraping to collect data for valuable insights. And it can also be helpful for Product Managers and UX Designers who scrape websites to learn more about their industries and products.

What else should I study if I am learning web scraping?

After learning how to extract data, the next step is putting it to good use. Try building your data analytics and visualization skills by learning to use Python libraries like NumPy and seaborn — or brush up on R, which was made specifically to work with data.

Course

Learn Web Scraping with Beautiful Soup

Can’t download the data you need? Learn how to pull data right from the page by web scraping with the Python library Beautiful Soup.

This course includes

22,523 learners enrolled

This course includes

Skill level
Intermediate
Time to complete
Average based on combined completion rates — individual pacing in lessons, projects, and quizzes may vary

2 hours
Projects
1
Prerequisites
2 courses

We suggest you complete the following courses before you get started with Learn Web Scraping with Beautiful Soup:
Learn Python 3
Learn Data Analysis with Pandas

About this course

Many of your coding projects may require you to pull a bunch of information from an HTML or XML page. This task can be really tedious and boring, that is until you learn how to scrape the web with an HTML Parser! That’s where Beautiful Soup comes in. This Python package allows you to parse HTML and XML pages with ease and pull all sorts of data off the web.

Say you want to pull all of the tweets from your favorite movie star and run some analysis on their word usage — scrape em! Maybe you want to make a digital collage all the images you’ve posted of your dog to Instagram — parse em! And if you want to pull a list of all of your friend’s favorite books from Goodreads — Beautiful Soup em!

Syllabus

Beautiful Soup
Learn how to take data that’s displayed on websites and put it into Python using the Beautiful Soup library!

lesson Web Scraping with Beautiful Soup
quiz Beautiful Soup
project Chocolate Scraping with Beautiful Soup
Certificate of completion available with Plus or Pro
Earn a certificate of completion and showcase your accomplishment on your resume or LinkedIn.

The platform

Hands-on learning

Animated GIF of an AI provided error explanation within Codecademy's learning environment

An AI-generated hint within the instructions of a Codecademy project

Animated GIF of Codecademy's Job Readiness Checker tool generating a compatibility report for a senior software engineer role

Animated GIF of building a phone screen interview using Codecademy's Interview Simulator

A fill-in-the-blank JavaScript question in a Codecademy assessment

Explore features

Projects in this course

practice Project
Chocolate Scraping with Beautiful Soup
Use BeautifulSoup to scrape a site that contains over 1700 expert ratings of different chocolate bars. Then, put the data you find into Pandas and analyze the results!

Earn a certificate of completion

Show your network you've done the work by earning a certificate of completion for each course or path you finish.

Show proofReceive a certificate that demonstrates you've completed a course or path.
Build a collectionThe more courses and paths you complete, the more certificates you collect.
Share with your networkEasily add certificates of completion to your LinkedIn profile to share your accomplishments.