Using Residential Proxies with Python: A Simple Example

Lewis Kerr — Tue, 06 Aug 2024 01:58:30 +0000

In this post, we'll explore how to use residential proxies with Python to make requests while masking your IP address. Residential proxies can help you access web content with a more authentic IP address, which can be useful for web scraping.

What is a Residential Proxy?

A residential proxy routes your requests through an IP address provided by an Internet Service Provider (ISP), rather than a data center. This makes it appear as though your requests are coming from a regular home user, which can be beneficial for avoiding IP-based rate limiting.

Example: Using a Residential Proxy with Python

Here’s a simple example of how to use a residential proxy in Python using the requests library:

import requests

if __name__ == '__main__':
    # Define the proxy details
    proxyip = "http://username_custom_zone_US:password@us.swiftproxy.net:7878"

    # The URL to which the request will be made
    url = "http://ipinfo.io"

    # Set up the proxies dictionary
    proxies = {
        'http': proxyip,
        'https': proxyip,  # Include HTTPS if you plan to use secure URLs
    }

    # Make a GET request through the proxy
    response = requests.get(url=url, proxies=proxies)

    # Print the response text
    print(response.text)

Breaking Down the Code

Proxy Details: Replace username_custom_zone_US, password, us.swiftproxy.net, and 7878 with your actual proxy credentials and details.

Proxies Dictionary: The proxies dictionary maps both HTTP and HTTPS protocols to your proxy. If you only need HTTP, you can remove the https entry.

Making Requests: The requests.get function is used to make a GET request to the specified URL through the proxy.

Printing the Response: The response from the server is printed out. In this example, we’re using http://ipinfo.io to show the IP address information of the proxy.

Important Notes

Handle Credentials Securely: Be cautious with sensitive information like usernames and passwords. Avoid hardcoding them in production code. Consider using environment variables or secure vaults for storing credentials.
Error Handling: For robustness, consider adding error handling to manage cases where the proxy might fail or the request might not succeed.
Legal and Ethical Use: Ensure that your use of proxies complies with legal regulations and the terms of service of the websites you are accessing.

Conclusion