
gittech. site
for different kinds of informations and explorations.
Progzee β open-source proxy management for ethical scraping (Python)
progzee
Progzee is a Python library for simplifying IP proxy usage in HTTP requests, with proxy rotation. It simplifies handling proxies for web scraping, API interaction, and more.
Key Features:
- IP Proxy rotation.
- Config file support for easy setup.
- CLI support for quick tasks.
Installation
Install Progzee using pip:
pip install progzee
Usage with proxies passed in constructor:
from progzee import Progzee
# Initialize with proxies explicitly
proxies = ["http://proxy1:port", "http://proxy2:port"]
pz = Progzee(proxies=proxies)
# Make a request
response = pz.get("https://example.com")
print(response.text)
Usage with config.ini
:
from progzee import Progzee
# Initialize with config file
pz = Progzee(config_file="config.ini")
CLI Usage:
# Update proxies from a config file
progzee update-proxies --config "config.ini"
# Fetch data from a URL
progzee fetch --url "https://example.com"
An example of a config.ini
file:
[progzee]
proxies = http://proxy1:port, http://proxy2:port
timeout_min = 5
timeout_max = 15
Error handling
Progzee automatically retries failed requests with the next proxy in the rotation.
Validate your proxies
Make sure that you're using valid proxies that you own, or from trusted vendors. One of the trusted vendors you can use is ProxyScrape.
License
Progzee is licensed under the Apache License 2.0. See LICENSE for more details.
β οΈ DISCLAIMER β οΈ
This tool is intended for ethical use cases only, including educational purposes, testing, and legitimate API interactions.
The tool is provided as-is, without any warranty. The author is not responsible for any damages or legal issues arising from its use.
Users are responsible for ensuring their usage complies with the Terms of Service of the APIs they interact with. Misuse of this tool, including but not limited to scraping without permission, bypassing rate limits, or engaging in malicious activities, is strictly prohibited.
By using this tool, you agree to use it responsibly and in compliance with all applicable laws and regulations.
Thanks for the support!
Support this project with the GitHub Star.
Support the project author with a pizza slice.