[ISSUE] Waydroid shipped broken

[ISSUE] Waydroid shipped broken

[notramo]

[-- Attachment #1: Type: text/plain, Size: 710 bytes --]

New issue by notramo on void-packages repository

https://github.com/void-linux/void-packages/issues/41363

Description:
### Is this a new report?

Yes

### System Info

Void x64 musl

### Package(s) Affected

waydroid

### Does a report exist for this bug with the project's home (upstream) and/or another distro?

_No response_

### Expected behaviour

Waydroid starts, or the guide is complete.

### Actual behaviour

Waydroid doesn't start. The attached guide (`README.voidlinux`) is incomplete.

### Steps to reproduce

1. Install `waydroid`
2. Follow the steps in `/usr/share/doc/waydroid/README.voidlinux`
3. Start the system service.
4. `waydroid session start`
5. `waydroid log` to view the error

Re: Waydroid shipped broken

[paper42]

[-- Attachment #1: Type: text/plain, Size: 158 bytes --]

New comment by paper42 on void-packages repository

https://github.com/void-linux/void-packages/issues/41363#issuecomment-1368005868

Comment:
@JamiKettunen 

Re: Waydroid shipped broken

[JamiKettunen]

[-- Attachment #1: Type: text/plain, Size: 1170 bytes --]

New comment by JamiKettunen on void-packages repository

https://github.com/void-linux/void-packages/issues/41363#issuecomment-1368266779

Comment:
@notramo I couldn't reproduce this in a fresh Void x86_64-musl VM, `weston-x11` launched as the compositor on Xfce4 image copied from ISO to disk and updated, and https://docs.waydro.id/faq/get-waydroid-to-work-through-a-vm done to get graphical output since `glxinfo | grep 'OpenGL renderer'` showed `llvmpipe`.

What does `waydroid status` show? There should be *A LOT* more details in `waydroid log` than that either way, run it before `waydroid session start`. You could possibly even down `waydroid-container` service and start it after starting to tail the log too to get even more details.

Curious if this could be a clash with another service trying to use some same address etc, perhaps `ls /var/service` would be useful (assuming you don't start system services elsewhere).

Btw `waydroid session start` also won't show a GUI, see `waydroid show-full-ui` instead for example. `.desktop` entries for Waydroid and the installed apps should also be created assuming you have an app launcher iterating those.

Re: Waydroid shipped broken

[notramo]

[-- Attachment #1: Type: text/plain, Size: 5360 bytes --]

New comment by notramo on void-packages repository

https://github.com/void-linux/void-packages/issues/41363#issuecomment-1370246273

Comment:
Full log (`waydroid log` started before launching container):
```
(013323) [22:18:55] % modprobe -q ashmem_linux
(013323) [22:18:55] % chmod 666 -R /dev/anbox-binder
(013323) [22:18:55] % chmod 666 -R /dev/anbox-vndbinder
(013323) [22:18:55] % chmod 666 -R /dev/anbox-hwbinder
(013323) [22:18:55] Container manager is waiting for session to load
(013716) [22:19:12] Save session config: /var/lib/waydroid/session.cfg
(013716) [22:19:12] UserMonitor service is not even started
(013716) [22:19:12] Clipboard service is not even started
(013323) [22:19:12] % /usr/lib/waydroid/data/scripts/waydroid-net.sh start
vnic is waydroid0

dnsmasq: failed to open pidfile /run/waydroid-lxc/dnsmasq.pid: Permission denied
Failed to setup waydroid-net.
Failed to setup waydroid-net.
(013323) [22:19:12] % umount /var/lib/waydroid/rootfs/vendor/waydroid.prop
(013323) [22:19:12] % umount /var/lib/waydroid/rootfs/vendor
(013323) [22:19:12] % umount /var/lib/waydroid/rootfs
(013323) [22:19:12] % mount /var/lib/waydroid/images/system.img /var/lib/waydroid/rootfs
(013323) [22:19:13] % mount -o remount,ro /var/lib/waydroid/images/system.img /var/lib/waydroid/rootfs
(013323) [22:19:13] % mount /var/lib/waydroid/images/vendor.img /var/lib/waydroid/rootfs/vendor
(013323) [22:19:13] % mount -o remount,ro /var/lib/waydroid/images/vendor.img /var/lib/waydroid/rootfs/vendor
(013323) [22:19:13] % mount -o bind /var/lib/waydroid/waydroid.prop /var/lib/waydroid/rootfs/vendor/waydroid.prop
(013323) [22:19:13] Save config: /var/lib/waydroid/waydroid.cfg
(013323) [22:19:13] % chmod 777 -R /dev/dri
(013323) [22:19:13] % chmod 777 -R /dev/fb0
(013323) [22:19:13] % chmod 777 -R /dev/video1
(013323) [22:19:13] % chmod 777 -R /dev/video0
(013323) [22:19:13] % lxc-start -P /var/lib/waydroid/lxc -F -n waydroid -- /init
(013323) [22:19:13] New background process: pid=13787, output=background
(013323) [22:19:13] waiting 10 seconds for container to start...
lxc-start: waydroid: ../src/lxc/network.c: netdev_configure_server_veth: 711 No such file or directory - Failed to attach "vethrKzhjZ" to bridge "waydroid0", bridge interface doesn't exist
lxc-start: waydroid: ../src/lxc/network.c: lxc_create_network_priv: 3427 No such file or directory - Failed to create network device
lxc-start: waydroid: ../src/lxc/start.c: lxc_spawn: 1840 Failed to create the network
lxc-start: waydroid: ../src/lxc/start.c: __lxc_start: 2107 Failed to spawn container "waydroid"
lxc-start: waydroid: ../src/lxc/conf.c: run_buffer: 321 Script exited with status 126
lxc-start: waydroid: ../src/lxc/start.c: lxc_end: 985 Failed to run lxc.hook.post-stop for container "waydroid"
lxc-start: waydroid: ../src/lxc/tools/lxc_start.c: main: 306 The container failed to start
lxc-start: waydroid: ../src/lxc/tools/lxc_start.c: main: 311 Additional information can be obtained by setting the --logfile and --logpriority options
(013323) [22:19:14] waiting 9 seconds for container to start...
(013323) [22:19:15] waiting 8 seconds for container to start...
(013323) [22:19:16] waiting 7 seconds for container to start...
(013323) [22:19:17] waiting 6 seconds for container to start...
(013323) [22:19:18] waiting 5 seconds for container to start...
(013323) [22:19:19] waiting 4 seconds for container to start...
(013323) [22:19:20] waiting 3 seconds for container to start...
(013323) [22:19:21] waiting 2 seconds for container to start...
(013323) [22:19:22] waiting 1 seconds for container to start...
(013323) [22:19:23] ERROR: container failed to start
(013323) [22:19:23] See also: <https://github.com/waydroid>
(013323) [22:19:23] Traceback (most recent call last):
  File "/usr/lib/waydroid/tools/__init__.py", line 68, in main
    actions.container_manager.start(args)
  File "/usr/lib/waydroid/tools/actions/container_manager.py", line 131, in start
    raise OSError("container failed to start")
OSError: container failed to start

(014031) [22:19:23] UserMonitor service is not even started
(014031) [22:19:23] Clipboard service is not even started
(013716) [22:19:23] UserMonitor service is not even started
(013716) [22:19:23] Clipboard service is not even started
(014032) [22:19:23] WayDroid container is STOPPED
(014043) [22:19:23] % modprobe -q ashmem_linux
(014043) [22:19:23] % chmod 666 -R /dev/anbox-binder
(014043) [22:19:23] % chmod 666 -R /dev/anbox-vndbinder
(014043) [22:19:23] % chmod 666 -R /dev/anbox-hwbinder
(014043) [22:19:23] Container manager is waiting for session to load
```

There are no messages in `dmesg` regarding the interface, or anything network-related.

The `/run/waydroid-lxc` directory is 777 permissioned and empty.

I have `dnscrypt-proxy` running on port 53, but it's complaining about a pidfile, not the TCP socket. I also tried to relaunch Waydroid container and session with `dnscrypt-proxy` stopped.

I tried to run `dnsmasq` standalone from a `root` shell, with the following command:
```
dnsmasq -x /run/waydroid-lxc/dnsmasq.pid
```
It didn't work, and failed with the same error message as is in the log. It started successfully when the pidfile was not specified (it was created at the default location `/run/dnsmasq.pid`).

Re: Waydroid shipped broken

[notramo]

[-- Attachment #1: Type: text/plain, Size: 455 bytes --]

New comment by notramo on void-packages repository

https://github.com/void-linux/void-packages/issues/41363#issuecomment-1370252275

Comment:
I tried editing the pidfile option in `/usr/lib/waydroid/data/scripts/waydroid-net.sh` to a random one in `/tmp`, and it still did not work.

Then I completely deleted the pidfile option, and it successfully boots up the Android runtime, and `waydroid app list` gives meaningful result by listing actual apps.

Re: Waydroid shipped broken

[notramo]

[-- Attachment #1: Type: text/plain, Size: 206 bytes --]

New comment by notramo on void-packages repository

https://github.com/void-linux/void-packages/issues/41363#issuecomment-1370263807

Comment:
However, there is no network access inside the Android system.

Re: Waydroid shipped broken

[notramo]

[-- Attachment #1: Type: text/plain, Size: 316 bytes --]

New comment by notramo on void-packages repository

https://github.com/void-linux/void-packages/issues/41363#issuecomment-1370263807

Comment:
However, there is no network access inside the Android system. I can't tell if it is a DNS configuration problem, or a routing problem, because I don't know how to test it.

Re: Waydroid shipped broken

[notramo]

[-- Attachment #1: Type: text/plain, Size: 388 bytes --]

New comment by notramo on void-packages repository

https://github.com/void-linux/void-packages/issues/41363#issuecomment-1370263807

Comment:
However, there is no network access inside the Android system. I can't tell if it is a DNS configuration problem, or a routing problem, because I don't know how to test it. How can I get a shell without ADB? (Or how can I open ADB to Waydroid?)

Re: Waydroid shipped broken

[JamiKettunen]

[-- Attachment #1: Type: text/plain, Size: 282 bytes --]

New comment by JamiKettunen on void-packages repository

https://github.com/void-linux/void-packages/issues/41363#issuecomment-1370368824

Comment:
`waydroid shell` gives you a root shell in the LXC container, I'll have to check later if I've ever had this `dnsmasq.pid` in my logs

Re: Waydroid shipped broken

[JamiKettunen]

[-- Attachment #1: Type: text/plain, Size: 393 bytes --]

New comment by JamiKettunen on void-packages repository

https://github.com/void-linux/void-packages/issues/41363#issuecomment-1370370729

Comment:
As for the network issue it could be various things really including a running service such as `docker` causing the issue or a firewall setup, unfortunately https://docs.waydro.id/debugging/networking-issues doesn't really have too much info :/

Re: Waydroid shipped broken

[notramo]

[-- Attachment #1: Type: text/plain, Size: 2053 bytes --]

New comment by notramo on void-packages repository

https://github.com/void-linux/void-packages/issues/41363#issuecomment-1370385454

Comment:
I tried flushing nftables, no success. Also tried it with Docker stopped.
`ip route` inside the container gives no output.
`ip addr show` output inside the container:
```
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0@if71: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:f9:d3:03 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::a758:6767:86f8:7586/64 scope link stable-privacy
       valid_lft forever preferred_lft forever
```
It seems it got an IPv6 address only, but the host system has only an IPv4 gateway.

`ip addr show` on the host system shows these 2 interfaces that are created by Waydroid:
```70: waydroid0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:00:00:01 brd ff:ff:ff:ff:ff:ff
    inet 192.168.240.1/24 brd 192.168.240.255 scope global waydroid0
       valid_lft forever preferred_lft forever
    inet 169.254.106.76/16 brd 169.254.255.255 scope global noprefixroute waydroid0
       valid_lft forever preferred_lft forever
    inet6 fe80::ba14:a2bc:d8:3e6e/64 scope link
       valid_lft forever preferred_lft forever
71: vethvPyc2L@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master waydroid0 state UP group default qlen 1000
    link/ether fe:98:50:59:58:6a brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 169.254.59.163/16 brd 169.254.255.255 scope global noprefixroute vethvPyc2L
       valid_lft forever preferred_lft forever
    inet6 fe80::6625:f411:d258:7207/64 scope link
       valid_lft forever preferred_lft forever
```

Re: Waydroid shipped broken

[notramo]

[-- Attachment #1: Type: text/plain, Size: 275 bytes --]

New comment by notramo on void-packages repository

https://github.com/void-linux/void-packages/issues/41363#issuecomment-1370386728

Comment:
I found the dnsmasq one. https://docs.waydro.id/debugging/known-issues

However, there is still no internet inside the container.

Re: Waydroid shipped broken

[JamiKettunen]

[-- Attachment #1: Type: text/plain, Size: 797 bytes --]

New comment by JamiKettunen on void-packages repository

https://github.com/void-linux/void-packages/issues/41363#issuecomment-1370411878

Comment:
You mentioned you had `dnscrypt-proxy` running on port 53 and I found https://github.com/waydroid/waydroid/issues/117#issuecomment-950303832 as well as https://github.com/waydroid/waydroid/issues/588 which could be relevant.

Since you said you had docker running, if restarting `waydroid-container` service didn't help solve networking issue you maybe `reboot`ing would? I previously filed https://github.com/waydroid/waydroid/issues/509 for the nftables + docker setup causing networking issues.

If you end up getting it working and have ideas on implementing the Waydroid network setup better do please contribute or file an issue upstream!

Re: Waydroid shipped broken

[CameronNemo]

[-- Attachment #1: Type: text/plain, Size: 220 bytes --]

New comment by CameronNemo on void-packages repository

https://github.com/void-linux/void-packages/issues/41363#issuecomment-1384581100

Comment:
Is AppArmor in use here? That could explain a dnsmasq permissions error.

Re: Waydroid shipped broken

[JamiKettunen]

[-- Attachment #1: Type: text/plain, Size: 582 bytes --]

New comment by JamiKettunen on void-packages repository

https://github.com/void-linux/void-packages/issues/41363#issuecomment-1384614603

Comment:
Upstream actually does have AppArmor profiles too since v1.3.4 (https://github.com/waydroid/waydroid/tree/1.3.4/data/configs/apparmor_profiles), we just don't package them (yet); simply additionally calling `make install_apparmor` should get those in (https://github.com/waydroid/waydroid/blob/1.3.4/Makefile#L37), and I suppose it shouldn't need any extra dependencies as people who want to use AppArmor set it up themselves on Void

Re: Waydroid shipped broken

[CameronNemo]

[-- Attachment #1: Type: text/plain, Size: 717 bytes --]

New comment by CameronNemo on void-packages repository

https://github.com/void-linux/void-packages/issues/41363#issuecomment-1384728603

Comment:
I was actually referring to this, which it seems could still be an issue:

https://docs.waydro.id/debugging/known-issues#one-cause-for-this-issues-is-apparmor-the-fix-is-as-follows

If the maintainer of waydroid (you, @JamiKettunen?) can test with AppArmor that would be ideal, otherwise we should not install the Waydroid profiles ... I am not sure that they tested it! Maybe in a separate package for the atypical use case. Most people should understand that Waydroid provides a relatively medium-soft security boundary... still on the same kernel and hypervisor.

Re: Waydroid shipped broken

[CameronNemo]

[-- Attachment #1: Type: text/plain, Size: 827 bytes --]

New comment by CameronNemo on void-packages repository

https://github.com/void-linux/void-packages/issues/41363#issuecomment-1384728603

Comment:
I was actually referring to this, which it seems could still be an issue:

https://docs.waydro.id/debugging/known-issues#one-cause-for-this-issues-is-apparmor-the-fix-is-as-follows

If the maintainer of waydroid (you, @JamiKettunen?) can test with AppArmor that would be ideal, otherwise we should not install the Waydroid profiles ... I am not sure that they tested it! Maybe in a separate package for the atypical use case. Most people should understand that Waydroid provides a relatively medium-soft security boundary... still on the same kernel and hypervisor.

Edit: Made an MR for the apparmor profile change https://gitlab.com/apparmor/apparmor/-/merge_requests/969

Re: Waydroid shipped broken

[HadetTheUndying]

[-- Attachment #1: Type: text/plain, Size: 296 bytes --]

New comment by HadetTheUndying on void-packages repository

https://github.com/void-linux/void-packages/issues/41363#issuecomment-1501034195

Comment:
I'm also experiencing the issues of just no networking within the container. I've followed everything talked about here and still nothing works.