Opened 4 months ago

Last modified 3 months ago

#65671 new defect

jemalloc @5.3.0_2: universal build fails for Monterey M1 arm64 when building x86_64 slice

Reported by: potterpg Owned by:
Priority: Normal Milestone:
Component: ports Version: 2.7.2
Keywords: monterey Cc: mascguy (Christopher Nielsen), barracuda156, Dave-Allured (Dave Allured)
Port: jemalloc

Description

I seem to have a build failure on this port.

Universal build on a mac M1 device.

Adding log file to assist in debug.

Attachments (1)

main.log (512.0 KB) - added by potterpg 4 months ago.
Log file of build…

Download all attachments as: .zip

Change History (19)

Changed 4 months ago by potterpg

Attachment: main.log added

Log file of build...

comment:1 Changed 3 months ago by mascguy (Christopher Nielsen)

Cc: mascguy barracuda156 added
Keywords: monterey added
Summary: build fails: jemalloc-5.3.0_2+universal.darwin_21.arm64-x86_64jemalloc @5.3.0_2: universal build fails for Monterey
Version: 2.7.2

Sergey, are you able to build this port Universal, for PPC/Rosetta?

comment:2 Changed 3 months ago by kencu (Ken)

(I thought ...) I fixed this two months ago, as per 65213.

[44412d7dbfb8102f854a0353ce2645893e83dc17/macports-ports]

and then moved that into muniversal and out of the jemalloc portfile.

[c9a6e03fc195486bbc91148a44393ae9f333f109/macports-ports]

[a05eaf3b301f7cdddfe8cbc9f4ff69a140c3e92c/macports-ports]

Is it not working again?

Last edited 3 months ago by kencu (Ken) (previous) (diff)

comment:3 Changed 3 months ago by kencu (Ken)

One thing -- I had no M1 machine to test on, so I did my work on a Monterey Intel system. It is conceivable that things work differently on an M1. I'll see if Monterey Intel is still fixed later on today.

comment:4 Changed 3 months ago by mascguy (Christopher Nielsen)

This appears to potentially be a run-of-the-mill compilation error, not necessarily universal-specific. But perhaps I'm missing something...?

include/jemalloc/internal/rtree.h:118:3: error: constant expression evaluates to -12 which cannot be narrowed to type 'unsigned int' [-Wc++11-narrowing]
        {RTREE_NSB, RTREE_NHIB + RTREE_NSB}
         ^~~~~~~~~
Version 0, edited 3 months ago by mascguy (Christopher Nielsen) (next)

comment:5 Changed 3 months ago by kencu (Ken)

Yes, this error this time seems quite different. During the Intel build on an M1, it seems to be this:

4362	:info:build In file included from src/jemalloc_cpp.cpp:10:
4363	:info:build In file included from include/jemalloc/internal/jemalloc_internal_includes.h:52:
4364	:info:build In file included from include/jemalloc/internal/arena_structs.h:4:
4365	:info:build In file included from include/jemalloc/internal/arena_stats.h:8:
4366	:info:build In file included from include/jemalloc/internal/pa.h:6:
4367	:info:build In file included from include/jemalloc/internal/ecache.h:5:
4368	:info:build In file included from include/jemalloc/internal/san.h:5:
4369	:info:build In file included from include/jemalloc/internal/emap.h:5:
4370	:info:build include/jemalloc/internal/rtree.h:118:3: error: constant expression evaluates to -12 which cannot be narrowed to type 'unsigned int' [-Wc++11-narrowing]

comment:6 Changed 3 months ago by Dave-Allured (Dave Allured)

Cc: Dave-Allured added

comment:7 Changed 3 months ago by kencu (Ken)

builds fine as universal on Monterey Intel, didn't touch a thing in the Portfile:

 % port -v installed jemalloc
The following ports are currently installed:
  jemalloc @5.3.0_2+universal (active) requested_variants='+universal' platform='darwin 21' archs='arm64 x86_64' date='2022-08-22T17:31:05-0700'

so -- either clean and try again, or will have to find someone with an M1 to see if they can reproduce the build error and take it from there.

Last edited 3 months ago by kencu (Ken) (previous) (diff)

comment:8 Changed 3 months ago by kencu (Ken)

Summary: jemalloc @5.3.0_2: universal build fails for Montereyjemalloc @5.3.0_2: universal build fails for Monterey M1 arm64 when building x86_64 slice

comment:9 Changed 3 months ago by Dave-Allured (Dave Allured)

Configure did not correctly detect the LG_VADDR parameter in this case. Main.log, line 456:

:info:configure checking number of significant virtual address bits... 0

The bogus zero value corresponds with the indicated errors, "can not narrow expression to unsigned int". This should be a processor-specific value like 48 or 56. I'm not sure which.

There is a configure option to provide a fixed value, and skip auto detection. From jemalloc/INSTALL.md:

  • --with-lg-vaddr=<lg-vaddr>

Specify the number of significant virtual address bits. By default, the configure script attempts to detect virtual address size on those platforms where it knows how, and picks a default otherwise. This option may be useful when cross-compiling.

There is an upstream issue with some useful discussion: https://github.com/jemalloc/jemalloc/issues/1997

comment:10 Changed 3 months ago by kencu (Ken)

yep, that looks like it. There is a patch in your referenced issue that might prove useful.

comment:11 Changed 3 months ago by Dave-Allured (Dave Allured)

All right. Jemalloc is currently in good support, so I would like to try to fix this upstream. Bit layout of the virtual address is highly specialized knowledge, so I would prefer to keep that upstream, rather than in a port file.

I am not at all familiar with MP universal builds. Does anyone have advice for best practice to pass processor selection to upstream's configure process, when cross compiling?

comment:12 Changed 3 months ago by potterpg

OK, so add this somewhere:

--with-lg-page=48

where should I add it, so I can try it out on my M1 Mac? I'm guessing in the jemalloc-5.3.0-arm64 folder somewhere?

Tried manual

 ./configure --with-lg-page=48

complained with too small a hugepage so tried

./configure --with-lg-page=48 --with-lg-hugepage=48

which it seemed to accept. followed by a make, which just had many errors, so I guess missing many parameters.

Happy to try any suggestions to help, but clearly I need advice of what needs doing.

comment:13 Changed 3 months ago by kencu (Ken)

you would have to add the right args to the merger universal args for the arm64 universal build only, if you prefer not to use the patch.

see the libvpx Portfile, for an example, amongst many others

https://github.com/macports/macports-ports/blob/master/multimedia/libvpx/Portfile

I would do it, but no M1 to test it, so..l

Last edited 3 months ago by kencu (Ken) (previous) (diff)

comment:14 Changed 3 months ago by potterpg

not a preference, I don't know how to use 'the' patch!

So, line 73

configure.args-append  --enable-shared

add something similar ?

configure.args-append --with-lg-page=48 --with-lg-hugepage=48

Maybe seperate lines?

or do you mean something else?

Let me know, and I can test it for you...

comment:15 Changed 3 months ago by potterpg

In the Portfile I Tried:

platform darwin {
    if {${build_arch} eq "arm64"} {
        configure.args-append --with-lg-page=48
        configure.args-append --with-lg-hugepage=48
    }
}

The config.log file suggests it was used as planned:

It was created by configure, which was
generated by GNU Autoconf 2.69.  Invocation command line was

  $ ./configure --prefix=/opt/local --with-jemalloc-prefix= --with-lg-page=48 --with-lg-hugepage=48 --host=aarch64-apple-darwin21.6.0

With further errors! ie

:info:build include/jemalloc/internal/bitmap.h:90:4: error: "Unsupported bitmap size"
:info:build #  error "Unsupported bitmap size"

and

:info:build include/jemalloc/internal/slab_data.h:9:18: error: use of undeclared identifier 'BITMAP_GROUPS_MAX'
:info:build         bitmap_t bitmap[BITMAP_GROUPS_MAX];

Which seemed odd errors for a seemingly working port on other architectures...

:info:build fatal error: too many errors emitted, stopping now [-ferror-limit=]
:info:build 29 warnings and 20 errors generated.

comment:16 Changed 3 months ago by kencu (Ken)

look at the libvpx portfile I referenced above, and the muniversal Portgroup

you will need to manipulate

merger_configure_args(${my_arch})

to add the needed argument to the arm build.

or use the patch here instead if you prefer that, which also should work

https://github.com/jemalloc/jemalloc/issues/1997#issuecomment-1041589117

Last edited 3 months ago by kencu (Ken) (previous) (diff)

comment:17 Changed 3 months ago by potterpg

Thanks Ken,

Finally understood what that means, literally create a file with that as contents, and then install the port...

So, duplicated the patch-quantum.diff, changed the contents to the linked comment...

Still seem to have the reported errors above though...

comment:18 Changed 3 months ago by potterpg

uninstalled it completely, then installed again...

This time it seemed to be fine?

Yes, confirmed, the lib is universal...

So, don't know what the previous errors were about.

Note: See TracTickets for help on using tickets.