clang-12 @12.0.1_1 +analyzer+debug+libstdcxx+tests: compiling consumes all available system memory; forces restart

[cooljeanius (Eric Gallager)]

I'm on Big Sur (11.6.8) with Xcode 13.2.1; I tried to get an Activity Monitor screenshot this time around before the system ran out of resources; let me try attaching that...

[cooljeanius (Eric Gallager)]

screenshot of a bunch of clang processes each taking up a bunch of memory while compiling clang-12

[cooljeanius (Eric Gallager)]

(as a side note, this bug has me wondering: instead of having just a binary yes/no for use_parallel_build, what about allowing Portfiles to set a maximum number of build jobs? 16 is probably too many here, but turning parallel build off completely here and limiting it to just a single job would probably also take too long... maybe just 4 jobs would work?)

[kencu (Ken)]

you can try easily -- set it in macports.conf

# Number of simultaneous make(1) jobs to use when building ports. If set
# to 0, the number of jobs will be the lesser of:
# - number of automatically-detected CPU cores
# - gigabytes of physical memory + 1
#buildmakejobs       	0

[jmroot (Joshua Root)]

You can certainly lower the number of build.jobs in a Portfile.

build.jobs    [expr {${build.jobs} > 4 ? 4 : ${build.jobs}}]

[ryandesign (Ryan Carsten Schmidt)]

If you are on arm64, then yes, I have also observed Big Sur crashing when too much memory is used. Since so many things changed under the hood in Big Sur, not to mention the use of an entirely new processor architecture for macOS, this must've been one of those things they didn't notice initially and didn't fix until Monterey.

We don't want to limit builds to a specific number of jobs. We want to limit them to an amount that will not exceed available memory or processing capacity.

You reported this against MacPorts 2.7.2. As of MacPorts 2.10.0 we now have the build.mem_per_job option. Your screenshot shows clang processes using up to 1.81 GB of memory. MacPorts defaults to assuming jobs will use no more than 1 GB of memory. This port should set build.mem_per_job to the number of megabytes that the largest job could consume. That's how both of the aforementioned issues were addressed.

If clang-12 uses this much memory per job, some earlier or later clangs might too and should also set that setting.