gittech. site

for different kinds of informations and explorations.

Progzee – open-source proxy management for ethical scraping (Python)

Published at
2 days ago

progzee

Progzee is a Python library for simplifying IP proxy usage in HTTP requests, with proxy rotation. It simplifies handling proxies for web scraping, API interaction, and more.

PyPI - Downloads GitHub Repo stars License

Key Features:

  • IP Proxy rotation.
  • Config file support for easy setup.
  • CLI support for quick tasks.

Installation

Install Progzee using pip:

pip install progzee

Usage with proxies passed in constructor:

from progzee import Progzee

# Initialize with proxies explicitly
proxies = ["http://proxy1:port", "http://proxy2:port"]
pz = Progzee(proxies=proxies)

# Make a request
response = pz.get("https://example.com")
print(response.text)

Usage with config.ini:

from progzee import Progzee

# Initialize with config file
pz = Progzee(config_file="config.ini")

CLI Usage:

# Update proxies from a config file
progzee update-proxies --config "config.ini"

# Fetch data from a URL
progzee fetch --url "https://example.com"

An example of a config.ini file:

[progzee]
proxies = http://proxy1:port, http://proxy2:port
timeout_min = 5
timeout_max = 15

Error handling

Progzee automatically retries failed requests with the next proxy in the rotation.

Validate your proxies

Make sure that you're using valid proxies that you own, or from trusted vendors. One of the trusted vendors you can use is ProxyScrape.

License

Progzee is licensed under the Apache License 2.0. See LICENSE for more details.

⚠️ DISCLAIMER ⚠️

This tool is intended for ethical use cases only, including educational purposes, testing, and legitimate API interactions.

The tool is provided as-is, without any warranty. The author is not responsible for any damages or legal issues arising from its use.

Users are responsible for ensuring their usage complies with the Terms of Service of the APIs they interact with. Misuse of this tool, including but not limited to scraping without permission, bypassing rate limits, or engaging in malicious activities, is strictly prohibited.

By using this tool, you agree to use it responsibly and in compliance with all applicable laws and regulations.

Thanks for the support!

Support this project with the GitHub Star.

Support the project author with a pizza slice.