Skip to content

Latest commit

 

History

81 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Free-proxy

Version 1.3.0

The FreeProxy class scrapes proxies from https://www.sslproxies.org/, https://www.us-proxy.org/, https://free-proxy-list.net/uk-proxy.html, and https://free-proxy-list.net and checks to make sure that it works. You can filter proxies by country, and specify an acceptable timeout. You can also randomize the list of proxies, rather than going in the order that they are scraped in.

You can use this to send requests through a custom proxy, with Selenium, or with anything else.

Returns proxy as string:

'http://113.160.218.14:8888'

Requirements

Installation

pip install free-proxy

With async support (installs aiohttp):

pip install "free-proxy[async]"

asciicast

Usage examples

First import Free Proxy that way:

from fp.fp import FreeProxy

Options

Parameter Type Example Default value
country_id list ['US', 'BR'] None
timeout float > 0 0.1 0.5
rand bool True False
anonym bool True False
elite bool True False
google bool,None False None
https bool True False
url str '' google.com
request_timeout float > 0 5 10
proxy = FreeProxy().get()
proxy = FreeProxy(country_id=['US', 'BR']).get()
proxy = FreeProxy(country_id=['US']).get()
proxy = FreeProxy(country_id=['GB']).get()
  • timeout parameter Timeout is the parameter for checking if a proxy is valid. If the server does not respond in specified time, the script will mark the proxy as invalid. Default timeout=0.5. You can change it by specifying a timeout eg. timeout=1.
proxy = FreeProxy(timeout=1).get()
  • rand parameter Shuffles the order of the proxy list from https://www.sslproxies.org/ instead of going from newest to oldest (as listed on the website). Defaults to rand=False
proxy = FreeProxy(rand=True).get()
  • anonym parameter Return only proxies marked as anonymous. Defaults to anonym=False
proxy = FreeProxy(anonym=True).get()
  • elite parameter Return only proxies marked as 'elite proxy'. Defaults to elite=False.
proxy = FreeProxy(elite=True).get()

Please note: elite proxies are always anonymous. If you set elite=True, you will also be eliminating any non-anonymous proxies.

  • google parameter If True it will only return proxies marked as "google". If False, it will not return proxies marked as "google". Defaults to google=None, which returns all proxies.
proxy = FreeProxy(google=True).get()
  • https parameter If True it will only return proxies marked as HTTPS. Defaults to https=False - i.e. HTTP proxy (for HTTP websites).

    Please note: HTTPS proxies work for both HTTP and HTTPS websites.

proxy = FreeProxy(https=True).get()

You can combine parameters:

proxy = FreeProxy(country_id=['US', 'BR'], timeout=0.3, rand=True).get()

If there are no working proxies with the provided parameters, the script will raise FreeProxyException with the message There are no working proxies at this time..

  • url parameter The url parameter allows you to set a custom URL for testing purposes. If left empty, it defaults to 'https://www.google.com'.

Using default URL:

proxy = FreeProxy().get() 

Using custom URL, if test on different endpoint is needed:

proxy = FreeProxy(url='http://httpbin.org/get').get() 
  • request_timeout parameter Number of seconds to wait when downloading the proxy list from the source websites (https://www.sslproxies.org/ etc.). If the website does not respond in that time, the script raises FreeProxyException instead of hanging indefinitely. Defaults to request_timeout=10. Not to be confused with timeout, which is used for checking individual proxies.
proxy = FreeProxy(request_timeout=5).get()

Async usage

AsyncFreeProxy is an asyncio variant of FreeProxy. It accepts the same options, but checks proxies concurrently (up to max_concurrent at a time, default 20) and returns the first working one, which is usually much faster than the synchronous version. Remaining checks are cancelled as soon as a winner is found.

Async support requires the async extra:

pip install "free-proxy[async]"

In a plain script, run it with asyncio.run:

import asyncio

from fp.fp import AsyncFreeProxy

proxy = asyncio.run(AsyncFreeProxy().get())

Inside your own async application (or in a Jupyter cell), just await it:

proxy = await AsyncFreeProxy(country_id=['US'], max_concurrent=50).get()
  • max_concurrent parameter (AsyncFreeProxy only) Maximum number of proxies checked at the same time. Defaults to max_concurrent=20. Raise it for faster scanning on a strong connection, lower it to be gentler on your network and the test URL.

CHANGELOG


[1.3.0] - 2026-09-06

  • Added AsyncFreeProxy — an asyncio variant of FreeProxy that checks proxies concurrently and returns the first working one (based on the idea from #39, thanks @NerdzzyDev)
  • Async support is an optional extra: pip install "free-proxy[async]" (installs aiohttp)
  • Added max_concurrent parameter to AsyncFreeProxy (default: 20) limiting how many proxies are checked at once
  • Raised minimum Python version to 3.9

[1.2.3] - 2026-08-24

  • Fixed the proxy check silently bypassing the proxy when the test URL scheme did not match the https setting. With default settings (HTTP proxy mapping + HTTPS test URL) requests selected no proxy at all, the peername verification then rejected every candidate and get() always failed. The proxies mapping now covers both http and https schemes. Broken since 1.2.1. (#61)
  • Added a CI workflow running the test suite on Python 3.9–3.13

[1.2.2] - 2026-07-04

  • Fixed get_proxy_list hanging indefinitely when a source website stalls — the proxy list request now has a timeout (#58)
  • Added request_timeout parameter (default: 10 seconds) to control the proxy list download timeout

[1.2.1] - 2026-06-09

  • Fixed proxy check failing due to a duplicated URL schema (http://https://...), introduced in 1.1.3. This broke https=True and custom url usage (#52, #43).

[1.2.0] - 2026-06-09

  • Updated lxml to version 6.1.1
  • Updated requests to version 2.34.2
  • Updated pip-chill to version 1.0.5
  • Pinned minimum dependency versions in install_requires (lxml>=5.0,<7, requests>=2.31,<3)
  • Raised minimum Python version to 3.8

[1.1.3] - 2024-11-07

  • Added url paramameter

[1.1.2] - 2024-09-07

  • Updated lxml to version 5.3.0
  • Updated pip-chill to version 1.0.3
  • Updated requests to version 2.32.3

[1.1.1] - 2023-02-18

  • Fixed https parameter error
  • Fixed additional loop issue when no proxy was found

[1.1.0] - 2022-11-12

[1.0.6] - 2022-01-23

  • Added google parameter
  • Added https parameter

[1.0.5] - 2022-01-07

  • Added elite parameter
  • Add exception class and raise exception instead of system exit
  • Change lxml version to 4.6.5

[1.0.4] - 2021-11-13

  • Fix proxy list default length

[1.0.3] - 2021-08-18

  • Change XPatch due to SSL proxies page update
  • Change lxml version

[1.0.2] - 2020-09-03

  • Added anonym parameter

[1.0.1] - 2020-03-19

  • Fix typos in readme
  • Fix urrlib3 exception urllib3.exceptions.ProxySchemeUnknown: Not supported proxy scheme None,
  • Fix imports

[1.0.0] - 2019-02-04

  • Initial release

Disclaimer

The authors of this repository are not responsible for any consequences, damages or losses arising from the use or misuse of this repository or content. Users are solely responsible for their actions and any consequences that may follow.

License


MIT

Free Software!

About

Free proxy scraper written in python. It is pypi library - free to use.

Topics

Resources

Stars

403 stars

Watchers

8 watching

Forks

Releases

Sponsor this project

Packages

Used by

Contributors

Languages