eBay listing prices with Python: a beginner's scraper

In plain English: if you sell a few things online, or you're hunting for a bargain, you've probably copied eBay prices into a spreadsheet by hand at some point. This script does the copying for you. It searches eBay for a keyword, reads the title, price and link of each listing across a few pages of results, and saves everything in a CSV file, a plain spreadsheet file that Excel, LibreOffice or Google Sheets can open. You'll use Python with requests to fetch the pages and beautifulsoup4 to read them. A proxy means the requests come from an address other than your own, and a short pause between pages keeps the load on eBay light.

Quick start in 3 steps

1. Install the two libraries (Step 1).
2. Swap your own proxy login into the code in Step 2.
3. Run the finished script from Step 5 and open the ebay.csv file it creates.

Is this allowed? Play fair

A handy tool needs sensible handling, and that's also what keeps you out of trouble. Before collecting anything, read eBay's robots.txt (the file where a site tells automated programs which areas to avoid) and its Terms of Service. Only take what any visitor can see without signing in. Don't work your way around sign-in pages, don't try to defeat anti-bot checks, and don't gather personal details about buyers or sellers. Pace yourself the way a person browsing would, leave a gap between pages, and keep a copy of what you fetch so you never download the same page twice. For bigger or business projects, eBay publishes official APIs, and they're worth checking first.

Which proxy type fits eBay?

eBay tailors results to where you appear to be, and it watches for traffic that doesn't behave like a person. Fire off dozens of requests from one address and you'll soon see a CAPTCHA or a short block. For a few pages now and then, datacenter proxies are the quicker, lower-cost option. If pages start coming back empty or odd, move up to residential proxies, which use household broadband addresses, and pick a rotating one so each request gets a different address. The proxies page lays out all four types side by side. Whichever you choose, run it through the proxy checker before you start, just to be sure it answers.

What you need

• Python 3, run from Command Prompt, PowerShell or Terminal.
• Proxy details in host, port, username and password form.
• A search word or two, the same as you'd type into eBay's search box.

Step 1: Install the tools

Run this once in your terminal:

pip install requests beautifulsoup4

Step 2: Proxy details and headers

Each of our proxies comes as a host:port address with a username and password. Put those into the proxies dictionary below. The headers are extra details sent with every request: User-Agent names the browser type, and Accept-Language asks for English pages.

import requests
from bs4 import BeautifulSoup

proxies = {
    "http": "http://USER:PASS@HOST:PORT",
    "https": "http://USER:PASS@HOST:PORT",
}

headers = {
    "User-Agent": (
        "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
        "AppleWebKit/537.36 (KHTML, like Gecko) "
        "Chrome/124.0 Safari/537.36"
    ),
    "Accept-Language": "en-US,en;q=0.9",
}

Step 3: Fetch one page of results

eBay's search addresses follow a simple pattern. _nkw holds your search words and _pgn holds the page number. This function asks for one page through the proxy, waits up to 20 seconds for an answer, and raises an error if eBay replies with a problem instead of a page:

def fetch_page(keyword, page=1):
    url = "https://www.ebay.com/sch/i.html"
    params = {"_nkw": keyword, "_pgn": page}
    response = requests.get(
        url,
        params=params,
        headers=headers,
        proxies=proxies,
        timeout=20,
    )
    response.raise_for_status()
    return response.text

Step 4: Read the listings

On the results page, each listing sits in a list item with the class s-item, and inside it you'll find the title, the price and a link. eBay reworks its page layout now and again, so the code checks that each piece exists before reading it. It also skips the placeholder "Shop on eBay" item that sometimes appears at the top of the list:

def parse_listings(html):
    soup = BeautifulSoup(html, "html.parser")
    items = []
    for card in soup.select("li.s-item"):
        title_el = card.select_one(".s-item__title")
        price_el = card.select_one(".s-item__price")
        link_el = card.select_one("a.s-item__link")

        if not title_el or not price_el:
            continue
        title = title_el.get_text(strip=True)
        if title.lower() in ("shop on ebay", ""):
            continue

        items.append({
            "title": title,
            "price": price_el.get_text(strip=True),
            "url": link_el["href"] if link_el else None,
        })
    return items

Step 5: Go through a few pages, politely

Now join the two functions together and leave a gap between pages. The time.sleep line isn't decoration. It pauses the script for 4 seconds after each page, and it's the habit that keeps your scraping considerate. At the end, the results go into a CSV file:

import time
import csv

def scrape(keyword, pages=3):
    results = []
    for page in range(1, pages + 1):
        html = fetch_page(keyword, page)
        page_items = parse_listings(html)
        print(f"Page {page}: {len(page_items)} listings")
        results.extend(page_items)
        time.sleep(4)  # be polite between requests
    return results

if __name__ == "__main__":
    data = scrape("mechanical keyboard", pages=3)
    with open("ebay.csv", "w", newline="", encoding="utf-8") as f:
        writer = csv.DictWriter(f, fieldnames=["title", "price", "url"])
        writer.writeheader()
        writer.writerows(data)
    print(f"Saved {len(data)} rows to ebay.csv")

The script prints how many listings it found on each page, so you can see straight away if a page came back empty.

Step 6: A different address for each request

For bigger jobs you may want each request to leave from a different IP address. A rotating residential endpoint does this on its side, so your settings never change. If instead you have a handful of fixed endpoints, put them in a list and pick one at random for every request:

import random

proxy_pool = [
    "http://USER:PASS@HOST1:PORT",
    "http://USER:PASS@HOST2:PORT",
    "http://USER:PASS@HOST3:PORT",
]

def get_proxies():
    choice = random.choice(proxy_pool)
    return {"http": choice, "https": choice}

To use it, change fetch_page so that its requests.get call receives proxies=get_proxies() instead of the fixed dictionary. Each fetch then starts from a different address.

When pages come back empty

An empty page or an unexpected error usually means eBay wants you to slow down. Take the hint. Wait longer between pages, try once or twice more with a growing pause (programmers call this backoff), and stop for the day if it keeps happening. Pressing on harder isn't the answer. You can check at any point which address your traffic is using with our my IP tool, to be sure the proxy really is in play.

Common mistakes

• Taking out the pause. It makes the run slightly quicker and much more likely to be blocked.
• Re-running the whole script whenever you want to look at the data. Open the CSV file instead; it's already saved.
• Expecting the layout to last forever. When the counts suddenly drop to zero, the class names have probably changed.
• Confusing asking prices with selling prices. Search results show what sellers are asking right now, not what items actually sold for.

Beginner questions

Can I search eBay in another country?

Yes. Each country has its own eBay site, such as ebay.co.uk or ebay.de. Change the address in fetch_page to the site you want, and use a proxy in that country so the results match what local shoppers see.

Why do some listings show a range instead of one price?

Listings with options, like different sizes or colours, often show the lowest and highest price together. The script saves the price text exactly as eBay displays it.

How many pages should I collect?

As few as answer your question. The example stops after three pages, which is plenty for getting a feel for prices on a single item.

That's a complete, working eBay scraper in well under a hundred lines. Start small, respect eBay's rules and rate limits, and add more proxy capacity only when your project really needs it. When you want to match a plan to the size of your project, the pricing page shows every option side by side.