Add IPInfo mmdb files support - #7
Conversation
|
Thanks for the PR. What's the license of IPInfo? the goal of MMDB was to support only open source data such as GeoOpen https://cra.circl.lu/opendata/geo-open/ - https://data.public.lu/en/datasets/geo-open-ip-address-geolocation-per-country-in-mmdb-format/ |
|
@adulau CC-BY-SA 4.0 |
|
Do you have a reference? because you need a token in order to download the database https://ipinfo.io/developers/database-download |
|
@adulau, please use the IPinfo Lite Database: https://ipinfo.io/developers/ipinfo-lite-database If you want to support the download mechanism, I highly recommend signing up to get a access token yourself. It is free to sign up, and once you have your token, you can redistribute it as part of the project. |
|
@abdullahdevrel Thanks a lot for the clarification of the license. Would it be possible to make it available without the token? as the content is open data. That would be a huge advantage for the users. |
|
@adulau If the project owner or maintainer opens a single free account with us, they can integrate the access token as a project-specific token. They can then distribute the project access token along with the project like embed/hardcode the access token directly in project codebase if they want. |
Brings the /raw source-IP endpoint, the Dockerfile, README docker notes, and updated GeoOpen databases. No conflicts with the IPInfo support; verified the merged server keeps both the ipinfo metadata handling and the new /raw route.
Exports the loaded mmdb as collapsed CIDR lists (IPv4+IPv6) in three formats: plain (HAProxy acl file), apache (Require ip lines), json. The index is built once in a daemon thread at startup so the lookup endpoints are never blocked; /cidr answers 503 until ready. Requires maxminddb >= 2.5 for Reader iteration.
The startup index kept every network of every country and ASN as ipaddress objects (~1.4M for GeoOpen-Country-ASN, +300 MB RSS) and never finished on a small host. Now nothing is indexed at startup: the first request for a key scans the database in a background thread, keeping only that key's networks (a few seconds, ~10 MB), answers 503 Retry-After meanwhile, and caches the result until restart. The scan uses its own mmap reader (C extension when available) so the in-memory lookup readers are untouched. Also recognizes the GeoOpen-Country-ASN ASN field and answers 404 immediately for country codes not in country.json.
… endpoint Covers /geolookup (ipinfo and GeoOpen record shapes, jq field extraction, 422 on bad input), / and /raw, /cidr/country and /cidr/asn in all three formats with the 503/Retry-After flow, and HAProxy/Apache integration snippets with a sanity-checked refresh script.
4 participants
No description provided.