Skip to content

Add 74 Taiwan-specific sites to tw.csv - #2267

Merged
agrabeli merged 1 commit into
citizenlab:masterfrom
anoni-net:tw-highweight-2026
Sep 18, 2026
Merged

agrabeli merged 1 commit into
citizenlab:masterfrom
anoni-net:tw-highweight-2026

Conversation

@toomore

@toomore toomore commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Following up on #1444, which we closed because its scope made it unreviewable. This is the first of the smaller, scoped batches we described there. It touches one file and adds entries only, with no deletions and no edits to existing rows.

The Taiwan list was 392 entries, 361 of them added in 2017, and its category mix had drifted away from the categories that matter most for censorship monitoring. POLR had a single entry, ANON had none, while COMM and GAME together were 28% of the list. This batch adds 74 Taiwan-specific, Chinese-language sites to fill the thinnest categories.

Before and after

hk.csv and kr.csv are shown alongside because we used them to work out which types of site a Taiwan list was missing.

Category Before After hk kr
POLR Political Criticism 1 21 58 49
REL Religion 9 24 84 4
HOST Hosting and Blogging Platforms 3 11 23 35
MMED Media sharing 5 9 8 15
LGBT LGBT 5 8 18 24
GRP Social Networking 3 7 16 1
PUBH Public Health 1 5 16 12
COMT Communication Tools 1 5 7 23
SRCH Search Engines 3 5 12 7
CULTR Culture 16 19 44 20
HUMR Human Rights Issues 42 45 30 50
XED Sex Education 1 3 12 13
NEWS News Media 94 95 97 87
ANON Anonymization and circumvention 0 1 13 2
Total 392 466 630 510

POLR is mostly political parties. None were in the list before: the only party entries were freetaiwanparty.tw under MILX and two green parties under ENV. REL now covers Buddhist orders, folk religion temples, Protestant and Catholic churches and media, Islam and new religious movements, following the range that hk.csv covers.

ANON is still 1. Taiwan has no significant locally hosted circumvention or anonymity service, and we decided against the approach hk.csv takes of listing localized marketing pages of commercial VPN vendors. We would rather leave the category thin than fill it with vendor pages.

How the entries were checked

116 candidates were considered and 74 kept. Every entry was checked for:

  • Duplicates, at the full-URL level against both lists/tw.csv and lists/global.csv, matching what scripts/lint-lists.py enforces.
  • Reachability, following redirects with a browser user agent.
  • Recent content, from the newest of four signals: the latest item in an RSS or Atom feed, the largest lastmod in sitemap.xml, the Last-Modified header, and dates in the page body. A single stale signal is not enough to call a site dormant, so all four are collected and the newest is used.
  • Identity, by reading the page <title> of every candidate that passed the checks above, to confirm the domain belongs to the organisation we assumed.

Candidates that failed were dropped rather than guessed at. Some examples of what this caught:

  • opinion.udn.com returns 200 and its domain is healthy, but the publication announced in August 2023 that it had stopped publishing.
  • vpntaiwan.com returns 200 and its registration runs to 2027, but the site now serves an unrelated SEO page for a plumbing business in Texas.
  • kktv.me returns 200 and had content dated within the year, but it redirects to a /sunset notice: the service closed on 2026-01-01 and moved to another platform.
  • 1980.org.tw, lungshan.org.tw, matsu.org.tw, roodo.com and ces.org.tw are all reachable but have not published anything for between one and nine years.
  • tas.org.tw passed every automated check, but its <title> showed it is a cardiology society rather than the gender education group we had assumed, so the category we had assigned would have been wrong.

scripts/lint-lists.py passes over the whole repository.

A small number of entries return 403 to non-browser clients because of bot protection. They are live, widely used services, and the behaviour is recorded in the notes column so it is not mistaken for a dead site later.

About us

We are anoni.net, a Taiwanese community working on anonymity networks and censorship measurement. We intend to keep maintaining lists/tw.csv on an ongoing basis. Two further batches are planned: a per-URL review of the dead and http:// entries from 2017, answering the review questions raised on #1444 for each one, and a small set of domains subject to administrative blocking orders in Taiwan.

One disclosure: anoni.net itself is included under ANON. It is our own site, added because the category was empty and it is a Taiwan-based, Chinese-language resource on anonymity tools. We are flagging it here rather than letting it pass unnoticed, and we will drop it without argument if you would prefer we did not list ourselves.

Adds POLR, REL, HOST, CULTR, MMED, GRP, LGBT, PUBH, SRCH, COMT, XED,
HUMR, NEWS and ANON entries specific to Taiwan. Additions only, no
deletions.

All URLs were checked for duplicates against both lists/tw.csv and
lists/global.csv, and verified on 2026-09-12 for reachability and for
recent content, using feed dates, sitemap lastmod, the Last-Modified
header and dates in the page body. Entries whose content had not been
updated for over a year, whose domain now serves unrelated content, or
which redirect to a service shutdown notice were dropped.

A few entries return 403 to non-browser clients because of bot
protection. They are live services and the behaviour is recorded in the
notes column.
@agrabeli

Copy link
Copy Markdown
Collaborator

Thank you @toomore ! <3

@agrabeli
agrabeli merged commit 180e33a into citizenlab:master Sep 18, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants