orwall prevents internet detection when using wifi

[eighthave]

I just set up my Nexus 7 2013 flo with CyanogenMod 12.1-20151117 and orWall. The wifi manager now thinks that my wifi does not have internet, and refuses to automatically connect to known wifi. I believe this internet detection is based on a "DNS ping" to the currently configured DNS server, which is probably blocked by orWall since it is DNS and its UDP. There should be a way to opt-in to this detection since it greatly improves the usability, especially on wifi-only devices.

My guess is that there is some process that can be whitelisted, but I haven't been able to find any info on it.

[eighthave]

There is a tiny bit more info about this in the com.android.server.wifi.WifiWatchdogStateMachine class

[cjeanneret]

hello,

hmm, there is indeed a weird thing in there: my nexus 4 can auto-connect to some wifi. But not all the time. Not all wifi.

The main issue with the "DNS Ping" is: guess who get the ping ;).
There is a part in orwall code doing some stuff with the dns ping — but it's for the captive portal detection — not sure it's related, although the detection process is probably the same :/.

Plus, since 4.4, there isn't per-user process separation for DNS queries, all is using the kernel cache, hence all is "proxified" to root :(.
That means: all DNS queries are passing through Tor.

Another tricky issue… But I think it's secondary compared to the "not-working-init" I still didn't shoot down…

[eighthave]

I wish we had more info on how the internet detection worked, it would make it a lot easier to figure this out. My guess is that the DNSPinger would be fine if the DNS traffic was being forced through Tor, as long as it gets the appropriate reply and does not timeout. Can you point me to the source code in Android that does the network probe mentioned in the comment in `enableCaptiveDetection()`?

[blahdeblahblah]

Just did a small test. And it seems if you connect to the wireless point with orwall disabled, and then later reconnect with all applications forced via orwall the captive portal detection passes. not sure why this is. Tested on cyanogenmod 12. totally breaks anonymity, but still. would be interesting to do a packet capture on a closed network to see if the portal detection is sent via tor in the second instance. it even works if you forget the network and reconnect.

[infinity0]

Hi, I'm having this issue too with Nexus 5X / CM 13. This appears to be a new thing in Android, you can't switch it off, and is separate from the "Avoid poor connections" setting which you can switch off.

The annoying thing about it, is that if your WiFi cuts out momentarily, your phone will switch to using data, and (due to this bug) won't automatically connect when WiFi comes back up, eating up your data plan. That just happened to me when I left my phone unattended to download some big blobs over the internet. :(

[eighthave]

if we can solve this issue, then orWall will be quite solid and easy to use.

[cjeanneret]

some help might be cool, really. For now, I'm unable to take back this project. I have some "big" plans, like complete rewrite using TDD in order to ensure people can actually add their own patches while ensuring there is no regression — the current UI is a mess and in fact, the app just doesn't seem to compile with new android sdk…
I guess I'll have some long, really long nights with that, and this will take a really long time before I can squash those bugs :(.

[eighthave]

I'll try to help where I can. I spent a chunk of time trying to track down the code in Android that actually does this "internet detection" of wifi points, but haven't yet found time to test adding the DNS ping stuff to the firewall rules. If we get this issue settled, there will be more users, and that usually brings more contributors. If we can figure out the mechanism, I'll happily code it up.

[infinity0]

I'm not sure if WifiWatchdogStateMachine is the culprit - that seems to be in charge of the "avoid poor connection" feature that one can disable, rather than the "no Internet" feature that prevents auto-join that one can't disable.

Anyway, following a long trail of grepping for MESSAGE_CONSTANTS, I found this: https://android.googlesource.com/platform/frameworks/base/+/master/services/core/java/com/android/server/connectivity/NetworkMonitor.java#103

    // After a network has been tested this result can be sent with EVENT_NETWORK_TESTED.
    // The network should not be used as a default internet connection.  It was found to be:
    // 1. a captive portal and the user is prompted to sign-in, or
    // 2. a captive portal and the user did not want to use it, or
    // 3. a broken network (e.g. DNS failed, connect failed, HTTP request failed).

which looks promising, since it's a specific list of things that they test for. On the downside, we might have to get orWall to unblock all of them, which will be a massive pain. :(

I will pause for now, and continue investigating when I next have some time. As a parallel line of thought, perhaps we can email these guys for help? :p

android/platform/frameworks/base$ git log --pretty=format:"%ae <%an>" services/core/java/com/android/server/connectivity/NetworkMonitor.java | head -n100 | sort -u

[infinity0]

I believe I have a workaround. Reading NetworkMonitor.java reveals that one way to transition to a validated state is to have isCaptivePortal() return 204 (line 465), which occurs if (but not only if) mIsCaptivePortalCheckEnabled is false (line 638). You can force this by going into a root shell, then execute

# settings put global captive_portal_detection_enabled 0

then turning WiFi off and on again. The exclamation mark and "Connected, no Internet" warning no longer appears, and Android automatically connects to my known networks again. This setting appears to get reverted when I restart the phone, but one can probably fix that by messing with /data/data/com.android.providers.settings/databases/settings.db using sqlite3.

This post warns that what I just did won't work in Android M Developer Preview - however I tested this on a Nexus 5X with CM 13 (based on Android 6.0.1) and it does work. It's also consistent with the source code (from master) that I linked above. So I'm not sure what that's about, but perhaps someone else can investigate further.

Of course, it's not clear how to turn this into an actual solution for orWall, but I'll leave that for later and/or other people and go to bed myself. :)

[infinity0]

BTW, this entire time I had "CaptivePortalLogin" itself in orWall set to "TransProxy via Bypass" and this didn't affect anything I did above. Only setting captive_portal_detection_enabled 0 made Android auto-connect to WiFi again. This seems strange to me, but I haven't had time to investigate further.

[eighthave]

@infinity0 awesome find! Sounds like you've got us headed in the right direction. I think that settings put global captive_portal_detection_enabled 0 will be a good short term solution, and something that should be easy to include in orWall. I think orWall should just set it on boot, since it is already doing stuff on boot. Modifying other app's data files could easily cause trouble, we have no idea what the lifecycle of that data is, or when its read or written to.

In the long run, we'll want to leave captive portal detection on, it greatly improves usability, especially for a device where everything is forwarded through Tor. Captive portal detection can be used to prompt the user whether they want to go to the captive portal, bypassing Tor. Otherwise, the user will just wonder why wifi isn't working.

[cjeanneret]

Hello,

already implemented in fact:
https://github.com/EthACKdotOrg/orWall/search?utf8=%E2%9C%93&q=captive_portal_detection_enabled

there might be some issue when the script is called though, have to squash this for some times now -.-'

[eighthave]

On my orWall device, that script does not seem to be working:

shell@flo:/ $ settings get global captive_portal_detection_enabled
1

Manually setting it made the ! go away on the wifi notification.

[infinity0]

I tried running this from /data/local/userinit.sh and it didn't work; logcat shows an selinux error:

01-26 01:18:18.663  4300  4300 W sh      : type=1400 audit(0.0:35): avc: denied { getattr } for path="/system/bin/app_process64" dev="mmcblk0p41" ino=336 scontext=u:r:sysinit:s0 tcontext=u:object_r:zygote_exec:s0 tclass=file permissive=0

I also tried adding it to /system/bin/oem-iptables-init.sh as described in #79#issuecomment-174730305 but got the same error. So it would be hard to do this automatically, unless we mess with selinux policies. The other option (making orwall allow captive portal queries) might be the better option.