osquery @4.3.0: Failed to retrieve the file from the given url

[PeteToscano (Pete Toscano)]

Trying to install osquery on my 11.5.2 laptop. Build failed, but couldn't find an obvious answer as to how or why. The log file that was pointed to in the error message showed that the make was running 16 parallel jobs at once:

:info:build make[2]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_sysutils_osquery/osquery/work/build'
:info:build [ 26%] Built target openssl
:info:build make[1]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_sysutils_osquery/osquery/work/build'
:info:build make: *** [all] Error 2
:info:build make: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_sysutils_osquery/osquery/work/build'
:info:build Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_sysutils_osquery/osquery/work/build" && /usr/bin/make -j16 -w all VERBOSE=ON
:info:build Exit code: 2
:error:build Failed to build osquery: command execution failed
:debug:build Error code: CHILDSTATUS 16813 2
:debug:build Backtrace: command execution failed
:debug:build     while executing
:debug:build "system {*}$notty {*}$callback {*}$nice $fullcmdstring"
:debug:build     invoked from within
:debug:build "command_exec -callback portprogress::target_progress_callback build"
:debug:build     (procedure "portbuild::build_main" line 8)
:debug:build     invoked from within
:debug:build "$procedure $targetname"
:error:build See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_sysutils_osquery/osquery/main.log for details.

I cded into /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_sysutils_osquery/osquery/work/build and ran the make as the macports user without the -j16 argument. This, I think, exposes the real problem:

make[2]: Entering directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_sysutils_osquery/osquery/work/build'
[ 48%] Downloading: https://s3.amazonaws.com/osquery-packages/third-party/pre-built/none/Jinja2-2.10-py2.py3-none-any.whl
cd /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_sysutils_osquery/osquery/work/build/libs/fb/jinja2 && /opt/local/bin/python3.9 /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_sysutils_osquery/osquery/work/osquery-4.3.0/tools/cmake/downloader.py https://s3.amazonaws.com/osquery-packages/third-party/pre-built/none/Jinja2-2.10-py2.py3-none-any.whl /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_sysutils_osquery/osquery/work/build/libs/fb/jinja2/Jinja2-2.10-py2.py3-none-any.whl 74c935a1b8bb9a3947c50a54766a969d4846290e1e788ea44c1392163723c3bd
Downloading...
Failed to retrieve the file from the given url
HTTP Error 404: Not Found
make[2]: *** [libs/fb/jinja2/Jinja2-2.10-py2.py3-none-any.whl] Error 1
make[2]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_sysutils_osquery/osquery/work/build'
make[1]: *** [libs/fb/jinja2/CMakeFiles/thirdparty_pythonmodule_jinja2_downloader.dir/all] Error 2
make[1]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_sysutils_osquery/osquery/work/build'
make: *** [all] Error 2
make: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_sysutils_osquery/osquery/work/build' 

I'm kind of new to Macports internals, but it looks like this is why the build is failing. On the other hand, maybe this is a red herring?

[ryandesign (Ryan Carsten Schmidt)]

It also looks like the port is out of date. Maybe updating will already fix this. There are also other open tickets that have yet to be addressed.

[PeteToscano (Pete Toscano)]

Replying to ryandesign:

It also looks like the port is out of date. Maybe updating will already fix this. There are also other open tickets that have yet to be addressed.

Osquery version 4.3.0 came out in April 2020. There have been several releases since then and version 4.9 is out now. I'd be happy to attempt to update it if you can point me to documentation on how to get started with this kind of thing.