#65671 closed defect (fixed)

jemalloc @5.3.0_2: universal build fails for Monterey M1 arm64 when building x86_64 slice

Reported by: potterpg Owned by: kencu (Ken)
Priority: Normal Milestone:
Component: ports Version: 2.7.2
Keywords: monterey Cc: mascguy (Christopher Nielsen), barracuda156, Dave-Allured (Dave Allured)
Port: jemalloc

Description

I seem to have a build failure on this port.

Universal build on a mac M1 device.

Adding log file to assist in debug.

Attachments (1)

main.log (512.0 KB) - added by potterpg 20 months ago.
Log file of build…

Download all attachments as: .zip

Change History (20)

Changed 20 months ago by potterpg

Attachment: main.log added

Log file of build...

comment:1 Changed 19 months ago by mascguy (Christopher Nielsen)

Cc: mascguy barracuda156 added
Keywords: monterey added
Summary: build fails: jemalloc-5.3.0_2+universal.darwin_21.arm64-x86_64jemalloc @5.3.0_2: universal build fails for Monterey
Version: 2.7.2

Sergey, are you able to build this port Universal, for PPC/Rosetta?

comment:2 Changed 19 months ago by kencu (Ken)

(I thought ...) I fixed this two months ago, as per 65213.

[44412d7dbfb8102f854a0353ce2645893e83dc17/macports-ports]

and then moved that into muniversal and out of the jemalloc portfile.

[a05eaf3b301f7cdddfe8cbc9f4ff69a140c3e92c/macports-ports]

Is it not working again?

Version 0, edited 19 months ago by kencu (Ken) (next)

comment:3 Changed 19 months ago by kencu (Ken)

One thing -- I had no M1 machine to test on, so I did my work on a Monterey Intel system. It is conceivable that things work differently on an M1. I'll see if Monterey Intel is still fixed later on today.

comment:4 Changed 19 months ago by mascguy (Christopher Nielsen)

This appears to potentially be a run-of-the-mill compilation error, not necessarily universal-specific. But perhaps I'm missing something...?

include/jemalloc/internal/rtree.h:118:3:
error: constant expression evaluates to -12 which cannot be narrowed to type 'unsigned int' [-Wc++11-narrowing]
        {RTREE_NSB, RTREE_NHIB + RTREE_NSB}
         ^~~~~~~~~
Last edited 19 months ago by mascguy (Christopher Nielsen) (previous) (diff)

comment:5 Changed 19 months ago by kencu (Ken)

Yes, this error this time seems quite different. During the Intel build on an M1, it seems to be this:

4362	:info:build In file included from src/jemalloc_cpp.cpp:10:
4363	:info:build In file included from include/jemalloc/internal/jemalloc_internal_includes.h:52:
4364	:info:build In file included from include/jemalloc/internal/arena_structs.h:4:
4365	:info:build In file included from include/jemalloc/internal/arena_stats.h:8:
4366	:info:build In file included from include/jemalloc/internal/pa.h:6:
4367	:info:build In file included from include/jemalloc/internal/ecache.h:5:
4368	:info:build In file included from include/jemalloc/internal/san.h:5:
4369	:info:build In file included from include/jemalloc/internal/emap.h:5:
4370	:info:build include/jemalloc/internal/rtree.h:118:3: error: constant expression evaluates to -12 which cannot be narrowed to type 'unsigned int' [-Wc++11-narrowing]

comment:6 Changed 19 months ago by Dave-Allured (Dave Allured)

Cc: Dave-Allured added

comment:7 Changed 19 months ago by kencu (Ken)

builds fine as universal on Monterey Intel, didn't touch a thing in the Portfile:

 % port -v installed jemalloc
The following ports are currently installed:
  jemalloc @5.3.0_2+universal (active) requested_variants='+universal' platform='darwin 21' archs='arm64 x86_64' date='2022-08-22T17:31:05-0700'

so -- either clean and try again, or will have to find someone with an M1 to see if they can reproduce the build error and take it from there.

Last edited 19 months ago by kencu (Ken) (previous) (diff)

comment:8 Changed 19 months ago by kencu (Ken)

Summary: jemalloc @5.3.0_2: universal build fails for Montereyjemalloc @5.3.0_2: universal build fails for Monterey M1 arm64 when building x86_64 slice

comment:9 Changed 19 months ago by Dave-Allured (Dave Allured)

Configure did not correctly detect the LG_VADDR parameter in this case. Main.log, line 456:

:info:configure checking number of significant virtual address bits... 0

The bogus zero value corresponds with the indicated errors, "can not narrow expression to unsigned int". This should be a processor-specific value like 48 or 56. I'm not sure which.

There is a configure option to provide a fixed value, and skip auto detection. From jemalloc/INSTALL.md:

  • --with-lg-vaddr=<lg-vaddr>

Specify the number of significant virtual address bits. By default, the configure script attempts to detect virtual address size on those platforms where it knows how, and picks a default otherwise. This option may be useful when cross-compiling.

There is an upstream issue with some useful discussion: https://github.com/jemalloc/jemalloc/issues/1997

comment:10 Changed 19 months ago by kencu (Ken)

yep, that looks like it. There is a patch in your referenced issue that might prove useful.

comment:11 Changed 19 months ago by Dave-Allured (Dave Allured)

All right. Jemalloc is currently in good support, so I would like to try to fix this upstream. Bit layout of the virtual address is highly specialized knowledge, so I would prefer to keep that upstream, rather than in a port file.

I am not at all familiar with MP universal builds. Does anyone have advice for best practice to pass processor selection to upstream's configure process, when cross compiling?

comment:12 Changed 19 months ago by potterpg

OK, so add this somewhere:

--with-lg-page=48

where should I add it, so I can try it out on my M1 Mac? I'm guessing in the jemalloc-5.3.0-arm64 folder somewhere?

Tried manual

 ./configure --with-lg-page=48

complained with too small a hugepage so tried

./configure --with-lg-page=48 --with-lg-hugepage=48

which it seemed to accept. followed by a make, which just had many errors, so I guess missing many parameters.

Happy to try any suggestions to help, but clearly I need advice of what needs doing.

comment:13 Changed 19 months ago by kencu (Ken)

you would have to add the right args to the merger universal args for the arm64 universal build only, if you prefer not to use the patch.

see the libvpx Portfile, for an example, amongst many others

https://github.com/macports/macports-ports/blob/master/multimedia/libvpx/Portfile

I would do it, but no M1 to test it, so..l

Last edited 19 months ago by kencu (Ken) (previous) (diff)

comment:14 Changed 19 months ago by potterpg

not a preference, I don't know how to use 'the' patch!

So, line 73

configure.args-append  --enable-shared

add something similar ?

configure.args-append --with-lg-page=48 --with-lg-hugepage=48

Maybe seperate lines?

or do you mean something else?

Let me know, and I can test it for you...

comment:15 Changed 19 months ago by potterpg

In the Portfile I Tried:

platform darwin {
    if {${build_arch} eq "arm64"} {
        configure.args-append --with-lg-page=48
        configure.args-append --with-lg-hugepage=48
    }
}

The config.log file suggests it was used as planned:

It was created by configure, which was
generated by GNU Autoconf 2.69.  Invocation command line was

  $ ./configure --prefix=/opt/local --with-jemalloc-prefix= --with-lg-page=48 --with-lg-hugepage=48 --host=aarch64-apple-darwin21.6.0

With further errors! ie

:info:build include/jemalloc/internal/bitmap.h:90:4: error: "Unsupported bitmap size"
:info:build #  error "Unsupported bitmap size"

and

:info:build include/jemalloc/internal/slab_data.h:9:18: error: use of undeclared identifier 'BITMAP_GROUPS_MAX'
:info:build         bitmap_t bitmap[BITMAP_GROUPS_MAX];

Which seemed odd errors for a seemingly working port on other architectures...

:info:build fatal error: too many errors emitted, stopping now [-ferror-limit=]
:info:build 29 warnings and 20 errors generated.

comment:16 Changed 19 months ago by kencu (Ken)

look at the libvpx portfile I referenced above, and the muniversal Portgroup

you will need to manipulate

merger_configure_args(${my_arch})

to add the needed argument to the arm build.

or use the patch here instead if you prefer that, which also should work

https://github.com/jemalloc/jemalloc/issues/1997#issuecomment-1041589117

Last edited 19 months ago by kencu (Ken) (previous) (diff)

comment:17 Changed 19 months ago by potterpg

Thanks Ken,

Finally understood what that means, literally create a file with that as contents, and then install the port...

So, duplicated the patch-quantum.diff, changed the contents to the linked comment...

Still seem to have the reported errors above though...

comment:18 Changed 19 months ago by potterpg

uninstalled it completely, then installed again...

This time it seemed to be fine?

Yes, confirmed, the lib is universal...

So, don't know what the previous errors were about.

comment:19 Changed 15 months ago by kencu (Ken)

Owner: set to kencu
Resolution: fixed
Status: newclosed

In b96e6fc66897b29c45391b8cf70dd191f9d78167/macports-ports (master):

jemalloc: fix universal build on arm64

if rosetta is installed the test for address bits fails
when running the x86_64 configure.

closes: #65671
closes: #65748

Note: See TracTickets for help on using tickets.