colima @0.5.5_0 container runtime fails to start

[a01fe (Aoife Finch)]

Colima container runtime fails to start.

$ colima start debug --verbose
INFO[0000] starting colima [profile=debug]              
INFO[0000] runtime: docker                              
INFO[0000] preparing network ...                         context=vm
INFO[0000] creating and starting ...                     context=vm
> Terminal is not available, proceeding without opening an editor
> "Attempting to download the image" arch=aarch64 digest="sha512:84c93e8aaa09446618bf87daa993e260da69b50e95670aed5df6671b2cff9464810752cbf70f6ee5ddf9d3e1c91d98104b3c573cc024c5f0687ad3f4d2e93ebc" location="https://github.com/abiosoft/alpine-lima/releases/download/colima-v0.5.5/alpine-lima-clm-3.18.0-aarch64.iso"
> Using cache "/Users/finchr/Library/Caches/lima/download/by-url-sha256/bbac3cc01786365dbff7aa3e7c1dc2dcc8ee0aeacd6df51bce9840c8feeca75f/data"
> [hostagent] Starting QEMU (hint: to watch the boot progress, see "/Users/finchr/.lima/colima-debug/serial.log")
> SSH Local Port: 49689
> [hostagent] Waiting for the essential requirement 1 of 5: "ssh"
> [hostagent] Driver stopped due to error: "exit status 1"
> [hostagent] Shutting down the host agent
> [hostagent] failed to exit SSH master
> [hostagent] Shutting down QEMU with ACPI
> [hostagent] failed to open the QMP socket "/Users/finchr/.lima/colima-debug/qmp.sock", forcibly killing QEMU
> [hostagent] QEMU has already exited
> exiting, status={Running:false Degraded:false Exiting:true Errors:[] SSHLocalPort:0} (hint: see "/Users/finchr/.lima/colima-debug/ha.stderr.log")
FATA[0000] error starting vm: error at 'creating and starting': exit status 1 
$ 

macOS 13.4, XCode 14.3 (14E222b), MacPorts 2.8.1, MacBook Pro 16" 2021 M1 Max

It works after reactivating colima@0.5.4_0.

[fmaerkl-sw]

I just experienced the same issue. As reported in ~/.lima/colima-debug/ha.stderr.log, the issue was that colima passed -netdev stream,... to qemu, which is only supported since qemu 7.2, but macports only has 7.1.0. Updating the qemu Portfile to 8.0.2, adding --enable-slirp and libslirp as a dependency made it work for me, so this would be fixed by updating the qemu port.

Alternatively, colima start --network-driver slirp seems to work as a workaround too.

[a01fe (Aoife Finch)]

Replying to fmaerkl-sw:

Alternatively, colima start --network-driver slirp seems to work as a workaround too.

This workaround works for me, thanks!

[herbygillot (Herby Gillot)]

In ef95e8d8614e5aad0f141fb1ecaedbbe7bc9f687/macports-ports (master):

qemu: update to 8.0.2

• support numerous image formats (qcow1, parallels, vdi, etc.) by default
• build with compression formats (lzo, snappy, lzfse & zstd) by default
• build with slirp support
• use Python 3.11 Sphinx to build docs
• disable linking against the jack audio library

Fixes: #67409
Fixes: #64246
See: #67549

[herbygillot (Herby Gillot)]

MacPorts qemu has been updated to 8.0.2 and slirp support enabled.

[herbygillot (Herby Gillot)]

Is this still happening after the qemu port update?

[MrBouncy]

Still unable to launch with my environment after upgrading qemu and activating colima@0.5.5 once again, specifying network driver as slirp or gvproxy (as before) seems to make no difference, running on MacOS 12.6.7

[klanchman (Kyle Lanchman)]

I'm also seeing issues with colima when using qemu@8.0.2. The symptoms are different than the original issue: colima start hangs for a long time and serial.log remains empty, like the VM isn't booting at all.

➤ time colima start --verbose
INFO[0000] starting colima
INFO[0000] runtime: docker
INFO[0000] preparing network ...                         context=vm
INFO[0000] starting ...                                  context=vm
> Using the existing instance "colima"
> [hostagent] Starting QEMU (hint: to watch the boot progress, see "/Users/klanchman/.lima/colima/serial.log")
> SSH Local Port: 59135
> [hostagent] Waiting for the essential requirement 1 of 5: "ssh"
> [hostagent] Waiting for the essential requirement 1 of 5: "ssh"
> [hostagent] Waiting for the essential requirement 1 of 5: "ssh"
> [hostagent] Waiting for the essential requirement 1 of 5: "ssh"
> [hostagent] Waiting for the essential requirement 1 of 5: "ssh"
> [hostagent] Waiting for the essential requirement 1 of 5: "ssh"
> [hostagent] Waiting for the essential requirement 1 of 5: "ssh"
> [hostagent] Waiting for the essential requirement 1 of 5: "ssh"
> did not receive an event with the "running" status
FATA[0600] error starting vm: error at 'starting': exit status 1

________________________________________________________
Executed in  600.26 secs      fish           external
   usr time  108.81 millis    0.12 millis  108.69 millis
   sys time  139.39 millis    1.96 millis  137.44 millis

---

Using colima@0.5.5 with qemu@8.0.3 from Homebrew seems to work. I tried editing the qemu Portfile to install v8.0.3, but colima start hangs and serial.log is empty with that build too.

[a01fe (Aoife Finch)]

Replying to klanchman:

I'm also seeing issues with colima when using qemu@8.0.2. The symptoms are different than the original issue: colima start hangs for a long time and serial.log remains empty, like the VM isn't booting at all.

I see the same behavior here.

[klanchman (Kyle Lanchman)]

I had some time to play with Lima and QEMU manually and I think I figured out the general problem and a workaround.

I found that using Lima directly was also not working for me. I dug around trying to figure out how to run something directly on QEMU aarch64 using -accel hvf (like Lima does). Most incantations I could find led to the same "no serial output, QEMU pegged at 100% CPU usage" behavior that Lima/Colima had. I eventually found ​a helpful Super User answer explaining that aarch64 QEMU machines always need to be pointed to an EFI firmware image. Once I did that, the VM booted up just fine.

I dug a bit through Lima and found where they handle giving QEMU a firmware image: ​https://github.com/lima-vm/lima/blob/148b686f0b1f7a0ef752be613286142bd287a91e/pkg/qemu/qemu.go#L1068

It seems like Lima should be looking in the right place for this firmware already, unless there's some subtle issue I'm missing (I don't write Go). But I copied the firmware to one of the other paths, and somehow that made Lima/Colima work for me!

---

So, this is my workaround:

Copy the EFI firmware image to one of the other directories where Lima tries to find it:

mkdir -p ~/.local/share/qemu
cp /opt/local/share/qemu/edk2-aarch64-code.fd  ~/.local/share/qemu/

Symlinking did not work for me, I had to copy it.

After this, the latest versions of colima/lima/QEMU from MacPorts are working for me.

---

This isn't a long-term solution as I assume the firmware images might change between QEMU versions, and MacPorts users shouldn't need to do this extra step manually.

Does anybody sees an issue in how Lima is looking for firmware that it'd work for Homebrew but not MacPorts? At some point I'll try to spend some time debugging that code to see if I can figure anything else out.

[thestr4ng3r (Florian Märkl)]

Background of this issue: comment:ticket:67731:14 The issue in this ticket is the same.

tl;dr workaround: Install afscompress from macports and do sudo afscompress -d /opt/local/share/qemu/edk2-aarch64-code.fd

The reason why the workaround suggested by klanchman works is that by copying the file, the APFS compression gets stripped.

[MrBouncy]

Replying to thestr4ng3r:

Background of this issue: comment:ticket:67731:14 The issue in this ticket is the same.

tl;dr workaround: Install afscompress from macports and do sudo afscompress -d /opt/local/share/qemu/edk2-aarch64-code.fd

You are a lifesaver (or at least a hair saver, my scalp thanks you)! I decompressed the whole firmware path there for now and everything is working locally now with current port versions of colima/lima/qemu.