Why should I care?
Repeatedly crawling a website using the same IP will get you blocked or cloaked. To prevent this, send requests through proxy networks, which replace your source IP with different IPs. You can either use a data-center, a network of servers that has its own IPs, or a residential network, a network of individuals’ IPs.
Advantages to web scraping with residential IPs:
- Residential IPs are real people’s IPs, so they’re almost impossible for a website to detect
- Datacenter IPs generally have the same subnet block range, making them easily detectable, while residential IPs do not
- Datacenter IPs can only be as specific as the data-center’s location, while residential IPs are available in almost any city around the world
How do I use data-center or residential IPs?
Luminati has data-center IPs in addition to 30 million residential IPs. Here’s how to get started:
- Sign up for Luminati and get $5 of free bandwidth to test our data-center IPs
- Send your first request through the data-center using your free $5
- Ask for residential permissions to gain access to our residential IPs