ScrapeUnblocker Ruby client

Official Ruby client for the ScrapeUnblocker web scraping API.

Every request is fully JavaScript-rendered in a real browser and routed through premium proxies, so it bypasses Cloudflare, DataDome, PerimeterX, Akamai, Kasada and similar anti-bot systems - from one simple call. You are only billed for successful requests.

• Highest success rate on the market (95%+ on live production traffic)
• Rendered HTML or parsed JSON - no per-site parsers to maintain
• Zero dependencies (uses the standard library), typed errors

Install

gem install scrapeunblocker

Or in a Gemfile:

gem "scrapeunblocker"

Requires Ruby 2.7+.

Quickstart

require "scrapeunblocker"

su = ScrapeUnblocker::Client.new # reads SCRAPEUNBLOCKER_KEY, or Client.new(api_key: "YOUR_API_KEY")

# Rendered HTML for any URL
html = su.get_page_source("https://example.com")

# Structured JSON instead of HTML (products, listings, search results, ...)
product = su.get_parsed("https://www.amazon.com/dp/B08N5WRWNW")
puts product.page_type # "product"
p product.data

Get your API key at app.scrapeunblocker.com. The free trial does not require a credit card.

Authentication

Set an environment variable and the client picks it up:

export SCRAPEUNBLOCKER_KEY="YOUR_API_KEY"

su = ScrapeUnblocker::Client.new # reads SCRAPEUNBLOCKER_KEY

Fetch rendered HTML

html = su.get_page_source(
  "https://www.nordstrom.com/browse/women/clothing/dresses",
  proxy_country: "US", # route through a specific country
  time_sleep: 3        # wait extra seconds after load
)

Browser steps

Drive the page after it loads with an ordered list of browser actions - fill a form, click through, wait for content, then capture the rendered result. Steps run once in the real browser, in order.

html = su.get_page_source(
  "https://example.com/search",
  steps: [
    { action: "type", selector: "input#q", value: "web scraping", clear: true },
    { action: "press_key", value: "Enter" },
    { action: "wait_for", selector: "#results" },
    { action: "scroll", value: "bottom" }
  ]
)

Supported actions and their fields:

Action	Fields
wait_for	selector, selector_type (css default, xPath, className, tagName), timeout_ms
wait_for_text	value (text to await), timeout_ms
wait	value (milliseconds)
click	selector, selector_type, timeout_ms
type	selector, selector_type, value, clear (bool), timeout_ms - types like a human
select	selector, selector_type, value, timeout_ms
press_key	value - one of Enter, Tab, Escape, Backspace, Delete, Space, ArrowUp, ArrowDown, ArrowLeft, ArrowRight, Home, End, PageUp, PageDown
scroll	value - "bottom" or a pixel count

Steps are not idempotent (they submit forms, click buttons), so a call is not safe to blindly retry. If a step fails, the API answers HTTP 422 and the client raises ScrapeUnblocker::ValidationError; its body is the JSON describing which step failed (error: "step_failed", step_index, action, reason, selector, html).

List elements

Pass list_elements: true to get a JSON summary of the matched elements instead of the full HTML. The method then returns a parsed Hash ({"url", "count", "elements"}) rather than an HTML String.

result = su.get_page_source("https://example.com", list_elements: true)
puts result["count"]
result["elements"].each { |el| p el }

Get parsed JSON

result = su.get_parsed("https://www.walmart.com/ip/12345")
puts result.page_type # e.g. "product"
puts result.source    # how it was extracted
p result.data         # the fields

# If a parse ever comes back wrong, force a fresh set of rules:
fresh = su.get_parsed(url, refresh_rules: true, rules_hint: "price is missing")

Google search (SERP)

serp = su.serp("web scraping api", pages_to_check: 2, proxy_country: "US")

Google Local (Maps)

local = su.google_local("coffee shops in chicago", proxy_country: "US", gl: "us")
local["results"].each { |biz| puts "#{biz['name']} #{biz['rating']} #{biz['address']}" }

Meta Ad Library

ads = su.meta_ad_library("Nike", country: "US")
ads["results"].each { |ad| puts "#{ad['id']} #{ad['media_type']}" }

advertiser is required. Optional filters: country, active_status, media_type and max_ads. Unset filters are dropped and the API applies its own defaults.

Oopbuy goods search

goods = su.oopbuy_search("running shoes", channel: "1688", page: 1, page_size: 20, sort: "default")
goods["results"].each { |item| puts "#{item['title']} #{item['price']} #{item['url']}" }

channel is one of "1688" (default), "taobao" or "official". sort is one of "default", "price_asc", "price_desc" or "best_selling". page_size max is 60. Oopbuy trademark-blocks brand keywords at its own backend: those come back as a successful 200 with keywordRejected: true and an empty results array, not an error.

Amazon

Product and search data as a Hash, priced in the marketplace's own currency:

# One product by ASIN (or url: "https://www.amazon.de/dp/B0BSHF7WHW")
product = su.amazon_product(asin: "B0BSHF7WHW", marketplace: "amazon.com")
puts [product["title"], product["price"], product["currency"], product["rating"]].join(" | ")

# Keyword search
results = su.amazon_search("wireless headphones", sort: "price_asc")
results["results"].each do |item|
  puts [item["title"], item["price"], item["currency"], item["asin"]].join(" | ")
end

proxy_country defaults to the marketplace's home country (amazon.com -> US, amazon.de -> DE), so prices come back in the right currency with no configuration.

eBay search

items = su.ebay_search("iphone 13", marketplace: "ebay.com", condition: "used", sort: "newly_listed")

if items["exactMatches"]
  items["results"].each do |item|
    puts [item["title"], item["price"], item["currency"], item["condition"]].join(" | ")
  end
end

marketplace is any of the 19 regional eBay hosts (ebay.com default). condition is one of "new", "open_box", "refurbished", "used" or "for_parts"; sort is one of "best_match" (default), "newly_listed", "ending_soon", "price_asc" or "price_desc"; page_size is 60, 120 or 240. exactMatches is false when eBay found nothing for the keyword and answered with its own loosely-related suggestions instead, so check it before using the listings.

TikTok

profile = su.tiktok_profile("nasa", max_videos: 5)      # exact stats + newest videos
video = su.tiktok_video("https://www.tiktok.com/@nasa/video/7665075736742530317", include_transcript: true)
tag = su.tiktok_hashtag("nasa", max_videos: 10)
results = su.tiktok_search("space telescope", max_results: 25)   # TikTok's own ranking
comments = su.tiktok_comments("https://www.tiktok.com/@nasa/video/7665075736742530317", max_comments: 40)
puts profile["stats"]["followers"], video["stats"]["plays"], tag["stats"]["views"], comments["totalComments"]

Profiles and hashtags list up to 10 videos in a couple of seconds from TikTok's server-rendered widget; ask for more (up to 200) and the real grid is scrolled in a browser session.

Cookies and the serving proxy

page = su.get_page_with_cookies("https://example.com")
puts page.html
p page.cookies
puts page.proxy

Images

bytes = su.get_image("https://example.com/photo.jpg")
File.binwrite("photo.jpg", bytes)

Skyscanner plugins

Flights, hotels and car hire as JSON:

locations = su.skyscanner.flight_locations("London")

flights = su.skyscanner.flights(
  origin: "London", dest: "New York",
  depart_date: "2026-09-01", adults: 1, currency: "USD"
)

hotels = su.skyscanner.hotels(destination: "Madrid", checkin: "2026-09-01", checkout: "2026-09-03")
cars = su.skyscanner.carhire(pickup: "Madrid", pickup_datetime: "2026-09-01T10:00", dropoff_datetime: "2026-09-03T10:00")

Error handling

Non-2xx responses raise typed errors, all subclasses of ScrapeUnblocker::Error.

begin
  html = su.get_page_source("https://example.com")
rescue ScrapeUnblocker::BlockedError
  # 403: the target blocked every bypass path (not billed)
rescue ScrapeUnblocker::PaymentRequiredError
  # 402: quota, credit limit, or a failed payment - fix billing
rescue ScrapeUnblocker::RateLimitError
  # 429: slow down
rescue ScrapeUnblocker::UpstreamOutageError
  # 503: the target site itself is down - retry later
end

Error	Status	Meaning
InvalidRequestError	400	Bad URL, unsupported scheme, or the API key header was not sent
AuthenticationError	401	Key not recognised - typo, stray whitespace, or a rotated key
NoSubscriptionError	401	Key is fine, but the account has no active plan
PaymentRequiredError	402	Billing block - base class for the three below
QuotaExceededError	402	The plan's requests for this period are used up
CreditLimitExceededError	402	Unpaid balance is past the account's credit limit
PaymentFailedError	402	A card payment was declined three times
BlockedError	403	Blocked by bot protection on every path
NotFoundError	404	Page loaded but held no image (get_image only)
BrowserTimeoutError	408	Our browser run timed out before the page was ready
UnsupportedContentError	415	The URL serves something other than HTML
ValidationError	422	Missing or wrong-typed parameter; body holds the detail array
RateLimitError	429	Too many requests
UpstreamOutageError	503	The target origin is down
ServerError	5xx	Unexpected server error, including a 504 upstream timeout
TimeoutError	-	This client gave up locally before the API answered
ConnectionError	-	Could not reach the API

Transient failures (429, 502, 503, 504 and network errors) are retried automatically with exponential backoff. A 401 or 402 is never retried - it clears when the key or the billing state changes, not on another attempt. Neither is billed or counted against your quota, because the request is refused before anything is scraped.

Billing errors (402)

The three billing blocks share a status code and differ only in their message, so the client raises a dedicated error for each:

begin
  html = su.get_page_source("https://example.com")
rescue ScrapeUnblocker::QuotaExceededError
  # plan quota (plus any overage allowance) is used up for this period
rescue ScrapeUnblocker::CreditLimitExceededError
  # unpaid balance passed the account credit limit
rescue ScrapeUnblocker::PaymentFailedError
  # card declined three times - update the payment method
end

When more than one applies, the most serious wins: failed payment outranks credit limit, which outranks quota. All three lift by themselves once the billing state changes - access returns within about a minute, and the API key stays the same. One catch worth knowing: subscribing to a new plan does not clear PaymentFailedError, because the old unpaid invoice stays open until it is paid.

Full details for every status code: docs.scrapeunblocker.com/errors.

Configuration

ScrapeUnblocker::Client.new(
  api_key: nil,        # or SCRAPEUNBLOCKER_KEY env var
  base_url: "https://api.scrapeunblocker.com",
  timeout: 180,        # seconds; protected pages can be slow
  max_retries: 2
)

Links

• Documentation: docs.scrapeunblocker.com
• Website: scrapeunblocker.com
• Dashboard: app.scrapeunblocker.com

License

MIT