Add missing features from proxybroker2 (minimum proxies in queue, remove proxy, sample script, strategy option) #161
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
|
@Dibbyo456 Could you try a clean install? It works in my test. Not sure why it failed installing at your side. |
sultanoz
approved these changes
May 5, 2020
bluet
changed the title
This was referenced Aug 20, 2022
Add coverage test
This reverts commit d051792.
Update remaining dev dependencies to latest versions: - pytest: ^7.1.2 → ^8.3.5 (modern testing framework) - pytest-runner: ^6.0.0 → ^6.0.1 (patch update) - alabaster: ^0.7.12 → ^1.0.0 (Sphinx theme) - poetry-core: >=1.0.0 → >=2.1.3 (build system) Remove redundant linting dependencies since we use ruff: - flake8 ^5.0.4 (replaced by ruff) - isort ^5.10.1 (replaced by ruff) - pytest-flake8 ^1.1.1 (replaced by ruff) - pytest-isort ^3.0.0 (replaced by ruff) This streamlines our toolchain to use ruff for all linting/formatting while keeping test framework and documentation tools current. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
Update legacy setup.py and documentation requirements to match the modernized dependency versions in pyproject.toml: setup.py: - aiohttp: 3.5.4 → 3.12.0 - aiodns: 2.0.0 → 3.4.0 - attrs: 19.3.0 → 25.3.0 - maxminddb: 1.4.1 → 2.7.0 - cachetools: 4.1.0 → 5.5.2 - click: added ≥8.2.1 - pytest: 4.3 → 8.3.5 - pytest-asyncio: 0.10 → 0.26.0 - Remove flake8/isort (replaced by ruff) docs/requirements.txt: - aiohttp: 3.8.0 → 3.12.0 - aiodns: 3.0.0 → 3.4.0 - maxminddb: 2.0.0 → 2.7.0 Ensures consistency across all dependency specifications. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
Update all documentation and CI/CD workflows to reflect modernized dependencies: GitHub Actions workflows: - Update Poetry versions: 1.7.1/1.8.4 → 1.8.4/2.1.3 - Update action versions: checkout@v2→v4, setup-python@v2→v5 - Modernize Python versions for publishing (3.x → 3.11) Documentation updates: - CLAUDE.md: Add dependency update section with versions - README.md: Update requirements with specific versions and descriptions - README.md: Modernize development tools section - docs/index.rst: Update Python badge to 3.10-3.13 - environment.yml: Update all dependencies to latest versions Consistency improvements: - All dependency files now reference same versions - Documentation reflects modern toolchain (ruff, pytest 8.3.5+) - Workflows use latest stable actions and Poetry versions 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
Change version from "2.0.0-beta1" to "2.0.0b1" to follow PEP 440 canonical pre-release format: - Eliminates setuptools normalization warning - Uses official Python packaging standard - Improves compatibility with packaging tools - Matches industry best practices The hyphenated form gets normalized to this format anyway, so using the canonical form directly is cleaner. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
Update remaining version references to use PEP 440 canonical format: - docker-build.sh: VERSION=v2.0.0-beta1 → v2.0.0b1 - CHANGELOG.md: [2.0.0-beta1] → [2.0.0b1] - CHANGELOG.md: Update URL references to use new format All version references now consistently use the canonical PEP 440 format "2.0.0b1" across: - pyproject.toml (source of truth) - docker-build.sh (Docker versioning) - CHANGELOG.md (release documentation) - Python package (__version__ auto-detected) 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Remove deprecated pytest-asyncio event_loop usage where possible - Clean up obsolete flake8 configuration in favor of ruff toolchain - Improve async test patterns while preserving functionality for tests requiring event_loop fixture - Eliminate most ResourceWarnings from aiohttp sessions during test execution - Remove deprecated pytest configuration for discontinued flake8 integration Also improve documentation quality: - Clarify api.rst as "User Guide - Essential API" vs api_auto.rst as complete reference - Fix RST syntax errors in proxy.py docstrings - Reduce documentation warnings from 42 to 34 (19% reduction) using proper :noindex: approach - Apply :noindex: to auto-generated reference while keeping user guide discoverable - Exclude constructors from user guide to focus on essential methods - Maintain full documentation content and cross-references Update project documentation: - Add warning management guidelines to CLAUDE.md - Document approach for balancing functionality vs warning cleanup - Update dependency status to reflect clean warning profile All tests continue to pass (131/131) with improved warning profile. These improvements are included in the consolidated 2.0.0b1 release. Also merged changelog entries: 2.0.0b1 consolidates all improvements from 2.0.0-alpha8. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Add 'from None' to properly distinguish timeout errors from errors in exception handling - Addresses gemini-code-assist review comment about exception chaining - All other reviewer comments were already addressed in the PR 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Regenerate lock file to ensure consistency with recent pyproject.toml updates - Fixes poetry lock file mismatch warning 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
Fixes poetry installation error by specifying the proxybroker package in the packages configuration. This tells poetry which package to install for the current project. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Disable autosummary_generate to prevent duplicate object descriptions - Remove autosummary directive from api_auto.rst - Add :exclude-members: __init__ to prevent duplicate __init__ descriptions - Add :noindex: to automodule directives - Remove auto-generated _autosummary files This fixes the 3 warnings that were causing the ReadTheDocs build to fail. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
ProxyBroker2 needs to handle both HTTP and HTTPS URLs by design as it's a proxy management tool that must work with various protocols. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
These assertions are always True because if the Server constructor failed, it would raise an exception rather than return None. Removed redundant assertions to clean up the test code. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Add exception chaining in checker.py for better debugging - Add debug logging for timeout in resolver.py - Remove outdated comments in checker.py These minimal changes address the real code quality issues found without adding unnecessary complexity. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Update Python version from 3.6 to 3.10 - Replace deprecated tools (flake8, isort) with ruff - Update all hook versions to latest stable - Add type checking with mypy - Add security scanning with bandit - Remove deprecated seed-isort-config This aligns pre-commit with the project's current toolchain. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Add proper exception chaining throughout codebase - Add stacklevel to all deprecation warnings - Fix unused loop variables - Apply consistent code formatting - Fix trailing whitespace and newlines Pre-commit hooks now enforce these standards going forward. Remaining warnings are false positives for proxy functionality. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Add assertions to verify server configuration in test_server_accepts_broker_serve_parameters - Fix pytest.raises to use specific exception types instead of broad Exception - Ignore S101 (assert) warnings in pre-commit for test files - Ignore S104 (binding to all interfaces) for tests directory 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Apply ruff format to tests/test_negotiators.py and tests/test_public_contracts.py - Fix unused loop variable by using underscore for protocol variable - Ensure consistent code style across the project 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
Add @types.setter to allow direct assignment to proxy.types as used
in tests/test_integration.py:87 and tests/utils.py:28. This enables
both direct assignment (proxy.types = {...}) and update operations
(proxy.types.update({...})) on the types property.
The setter properly handles None values by initializing an empty dict,
maintaining backward compatibility with existing code.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <[email protected]>
🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
Replace pytest.raises(Exception) with specific ResolveError exception to improve test precision and fix B017 linting rule. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
Reset self._schemes = () in the types setter to ensure the schemes property gets recalculated based on the new types. This prevents stale cached values when proxy types are modified after creation. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Add missing asyncio import - Pre-populate cache to test cache hit behavior properly - Use asyncio.TimeoutError for proper exception type that gets converted to ResolveError - Simplify exception testing by directly mocking DNS query to raise TimeoutError 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
Add robust type validation to ensure new_types parameter is a dict or None: - Raise TypeError with clear message for invalid types - Maintain existing behavior for valid dict and None values - Preserve cache invalidation functionality - Add comprehensive test coverage for validation This improves the API's robustness and provides clearer error messages when users accidentally pass incorrect types. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
Fix critical bugs, achieve 100% test coverage, and improve documentation
Major documentation updates to reflect the production-ready status: ### README.md - Updated title to emphasize production-ready v2.0.0b1 status - Added comprehensive feature highlights with emojis for better readability - Updated installation section to promote stable v2.0.0b1 release - Enhanced benefits section with production-ready messaging ### docs/source/index.rst - Updated header to reflect production-ready status - Added v2.0.0b1 key achievements section - Emphasized zero critical bugs and 100% test coverage ### CHANGELOG.md - Added unreleased section with latest improvements - Updated release date to 2025-05-26 - Added type validation and test improvements ### CLAUDE.md - Updated section header to reflect v2.0.0b1 release date - Maintains comprehensive architecture and development guidance All documentation now accurately reflects the production-ready status of ProxyBroker2 v2.0.0b1 with its zero critical bugs, comprehensive testing, and modern Python support. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
Poetry requires README.md to be present during installation as it's referenced in pyproject.toml. This fixes the error: 'Readme path /app/README.md does not exist' The Dockerfile now copies README.md along with poetry.lock and pyproject.toml before running poetry install. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Update CLI help message to point to new issues URL - Update README.md image URLs to new repository - Update package metadata URL in __init__.py - Keep historical changelog URLs unchanged for accuracy
- Add current maintainer (BlueT - Matthew Lien - 練喆明) to copyright - Update author metadata to current maintainer - Preserve original author's copyright alongside current maintainer's
- Update all image URLs from constverum/ProxyBroker to bluet/proxybroker2 - Update PR reference link to new repository
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
From https://github.com/bluet/proxybroker2
added: --min_queue to keep a minimum number of proxy candidates (and RR before promote them to official pool)
added: remove proxy by calling special url http://proxyremove/host:port
added: --strategy=best command line option
added: use_existing_proxy.py in example
added: inject X-Proxy-Info header in response
added: HTTP API for getting proxy info and removing proxy from queue
added: docker image on docker hub
Related issues:
#147
#142
#139