Opened 3 months ago

Last modified 5 weeks ago

#61445 new defect

Java version not detected on macOS Big Sur 11.0.1

Reported by: breun (Nils Breunese) Owned by:
Priority: Normal Milestone:
Component: ports Version:
Keywords: Cc: ericmoret, cjones051073 (Chris Jones), catap (Kirill A. Korinsky), amake (Aaron Madlon-Kay)
Port:

Description

When trying to upgrade a port on macOS Big Sur 11.0.1 which requires Java (via 'PortGroup java 1.0'), I get the following error:

% sudo port upgrade maven3
--->  Fetching archive for maven3
--->  Attempting to fetch maven3-3.6.3_0.darwin_20.noarch.tbz2 from https://packages.macports.org/maven3
--->  Attempting to fetch maven3-3.6.3_0.darwin_20.noarch.tbz2 from https://nue.de.packages.macports.org/maven3
--->  Attempting to fetch maven3-3.6.3_0.darwin_20.noarch.tbz2 from https://mse.uk.packages.macports.org/maven3
--->  Computing dependencies for maven3
--->  Fetching distfiles for maven3
Error: maven3 requires Java 1.7+ but no such installation could be found.
Error: Failed to fetch maven3: missing required Java version
Error: See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_java_maven3/maven3/main.log for details.
Error: Follow https://guide.macports.org/#project.tickets to report a bug.

However, OpenJDK 8 is installed via MacPorts just fine:

% /usr/libexec/java_home -f -v 1.7
/Library/Java/JavaVirtualMachines/openjdk8/Contents/Home
% port installed openjdk8
The following ports are currently installed:
  openjdk8 @8u272_0 (active)
% echo $JAVA_HOME
/Library/Java/JavaVirtualMachines/openjdk8/Contents/Home

The full log file:

version:1
:debug:sysinfo macOS 11.0 (darwin/20.1.0) arch i386
:debug:sysinfo MacPorts 2.6.4
:debug:sysinfo Xcode 12.2
:debug:sysinfo SDK 11.0
:debug:sysinfo MACOSX_DEPLOYMENT_TARGET: 11.0
:debug:main Executing org.macports.main (maven3)
:debug:main dropping privileges: euid changed to 502, egid changed to 501.
:debug:archivefetch archivefetch phase started at Sat Nov 14 01:55:29 CET 2020
:msg:archivefetch --->  Fetching archive for maven3
:debug:archivefetch Executing org.macports.archivefetch (maven3)
:debug:archivefetch euid/egid changed to: 0/0
:debug:archivefetch chowned /opt/local/var/macports/incoming to macports
:debug:archivefetch euid/egid changed to: 502/501
:info:archivefetch --->  maven3-3.6.3_0.darwin_20.noarch.tbz2 doesn't seem to exist in /opt/local/var/macports/incoming/verified
:msg:archivefetch --->  Attempting to fetch maven3-3.6.3_0.darwin_20.noarch.tbz2 from https://packages.macports.org/maven3
:debug:archivefetch Fetching archive failed: The requested URL returned error: 404
:msg:archivefetch --->  Attempting to fetch maven3-3.6.3_0.darwin_20.noarch.tbz2 from https://nue.de.packages.macports.org/maven3
:debug:archivefetch Fetching archive failed: The requested URL returned error: 404 Not Found
:msg:archivefetch --->  Attempting to fetch maven3-3.6.3_0.darwin_20.noarch.tbz2 from https://mse.uk.packages.macports.org/maven3
:debug:archivefetch Fetching archive failed: The requested URL returned error: 404 Not Found
version:1
:debug:sysinfo macOS 11.0 (darwin/20.1.0) arch i386
:debug:sysinfo MacPorts 2.6.4
:debug:sysinfo Xcode 12.2
:debug:sysinfo SDK 11.0
:debug:sysinfo MACOSX_DEPLOYMENT_TARGET: 11.0
:msg:archivefetch --->  Computing dependencies for maven3:info:archivefetch .:debug:archivefetch Searching for dependency: openjdk11
:debug:archivefetch Found Dependency: receipt exists for openjdk11
:debug:archivefetch Searching for dependency: maven_select
:debug:archivefetch Found Dependency: receipt exists for maven_select
:debug:main Executing org.macports.main (maven3)
:debug:main dropping privileges: euid changed to 502, egid changed to 501.
:debug:fetch fetch phase started at Sat Nov 14 01:55:33 CET 2020
:notice:fetch --->  Fetching distfiles for maven3
:debug:fetch Executing proc-pre-org.macports.fetch-fetch-0
:debug:fetch Discovered JAVA_HOME via /usr/libexec/java_home: /Library/Java/JavaVirtualMachines/openjdk15-openj9/Contents/Home
:debug:fetch Adding dependency on JDK fallback openjdk11
:error:fetch maven3 requires Java 1.7+ but no such installation could be found.
:error:fetch Failed to fetch maven3: missing required Java version
:debug:fetch Error code: NONE
:debug:fetch Backtrace: missing required Java version
:debug:fetch     while executing
:debug:fetch "$pre $targetname"
:error:fetch See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_java_maven3/maven3/main.log for details.

Change History (27)

comment:1 Changed 3 months ago by jmroot (Joshua Root)

What if you run the java_home command with 1.7+ rather than 1.7?

/usr/libexec/java_home -f -v 1.7+
Last edited 3 months ago by jmroot (Joshua Root) (previous) (diff)

comment:2 Changed 3 months ago by stanft (Stephan Anft)

I have the same problem since Big Sur, i.e. with maven3 and gradle port.

This is the results of /usr/libexec/java_home on my system:

/usr/libexec/java_home -f -v 1.7: The operation couldn’t be completed. Unable to locate a Java Runtime that supports (null).
/usr/libexec/java_home -f -v 1.7+: The operation couldn’t be completed. Unable to locate a Java Runtime that supports (null).
/usr/libexec/java_home -f -v 1.8: /Library/Java/JavaVirtualMachines/openjdk8/Contents/Home

It seems that the fail fast option might be the culprit. Following the result for version string 1.7+ with no fail fast:

/usr/libexec/java_home -v 1.7+: /Library/Java/JavaVirtualMachines/jdk-15.jdk/Contents/Home

... which is the newest Java SDK on my machine.

Regards, Stephan

comment:3 Changed 3 months ago by breun (Nils Breunese)

That command works fine on my machine:

% /usr/libexec/java_home -f -v 1.7+
/Library/Java/JavaVirtualMachines/openjdk8/Contents/Home

comment:4 Changed 3 months ago by stanft (Stephan Anft)

Is this on Big Sur, too? And what JDKs are installed in total on your machine? I have too Big Sur machines here, and both have the same problem.

comment:5 Changed 3 months ago by breun (Nils Breunese)

I don't know who you're asking, but I created this ticket and I'm indeed on Big Sur.

I have these JDKs installed:

% port installed | grep openjdk
  openjdk8 @8u272_0 (active)
  openjdk8-graalvm @20.2.0_0 (active)
  openjdk8-openj9 @8u272_0 (active)
  openjdk11 @11.0.9_0 (active)
  openjdk11-graalvm @20.2.0_0 (active)
  openjdk11-openj9 @11.0.9_0 (active)
  openjdk15 @15.0.1_0 (active)
  openjdk15-openj9 @15.0.1_0 (active)

Or from the perspective of java_home:

% /usr/libexec/java_home -V
Matching Java Virtual Machines (8):
    15.0.1 (x86_64) "AdoptOpenJDK" - "OpenJDK 15.0.1" /Library/Java/JavaVirtualMachines/openjdk15-openj9/Contents/Home
    15.0.1 (x86_64) "AdoptOpenJDK" - "OpenJDK 15.0.1" /Library/Java/JavaVirtualMachines/openjdk15/Contents/Home
    11.0.9 (x86_64) "AdoptOpenJDK" - "OpenJDK 11.0.9" /Library/Java/JavaVirtualMachines/openjdk11-openj9/Contents/Home
    11.0.9 (x86_64) "AdoptOpenJDK" - "OpenJDK 11.0.9" /Library/Java/JavaVirtualMachines/openjdk11/Contents/Home
    11.0.8 (x86_64) "GraalVM Community" - "GraalVM CE 20.2.0" /Library/Java/JavaVirtualMachines/openjdk11-graalvm/Contents/Home
    1.8.0_272 (x86_64) "AdoptOpenJDK" - "OpenJDK 8" /Library/Java/JavaVirtualMachines/openjdk8/Contents/Home
    1.8.0_272 (x86_64) "AdoptOpenJDK" - "OpenJDK 8" /Library/Java/JavaVirtualMachines/openjdk8-openj9/Contents/Home
    1.8.0_262+10 (x86_64) "GraalVM Community" - "GraalVM CE 20.2.0" /Library/Java/JavaVirtualMachines/openjdk8-graalvm/Contents/Home
/Library/Java/JavaVirtualMachines/openjdk8/Contents/Home

comment:6 Changed 3 months ago by Tatsh (Andrew Udvare)

Cc: Tatsh added

comment:7 Changed 3 months ago by amake (Aaron Madlon-Kay)

I'm also having this problem.

The issue seems to be that using -f -v with a non-literal version (e.g. 1.8+) fails when run as the root user.

$ sudo /usr/libexec/java_home -f -v 1.8+
The operation couldn’t be completed. Unable to locate a Java Runtime that supports (null).
Please visit http://www.java.com for information on installing Java.

$ sudo /usr/libexec/java_home -f -v 1.8 
/Library/Java/JavaVirtualMachines/openjdk8/Contents/Home

$ /usr/libexec/java_home -f -v 1.8+    
/Library/Java/JavaVirtualMachines/openjdk8/Contents/Home

I tried wrapping find_java_home in the portgroup with exec_as_uid to run the checks as the (non-root) invoking user, but it still didn't work. It might have something to do with the env rather than the user itself.

Last edited 3 months ago by amake (Aaron Madlon-Kay) (previous) (diff)

comment:8 Changed 3 months ago by amake (Aaron Madlon-Kay)

I have a temporary workaround ready here: https://github.com/macports/macports-ports/pull/9123

comment:9 Changed 3 months ago by Aaron Madlon-Kay <aaron@…>

In a5935f1fd89e02a43832bd9914766729b76ca792/macports-ports (master):

java-1.0: temporarily skip version check on Big Sur

See #61445

comment:10 Changed 3 months ago by breun (Nils Breunese)

Hm, that command doesn't fail for me and gives the same result, whether run via sudo or not:

~ % sudo /usr/libexec/java_home -f -v 1.8+
/Library/Java/JavaVirtualMachines/openjdk15-openj9/Contents/Home
~ % /usr/libexec/java_home -f -v 1.8+
/Library/Java/JavaVirtualMachines/openjdk15-openj9/Contents/Home

comment:11 Changed 3 months ago by JBYoshi (Jonathan Browne)

On my system, I also get an error when running the command with sudo. However, manually setting the $JAVA_HOME environment variable to a valid installation will cause it to succeed:

jbyoshi@Yoshi ~ % sudo /usr/libexec/java_home -f -v 1.8+ 
The operation couldn’t be completed. Unable to locate a Java Runtime that supports (null).
Please visit http://www.java.com for information on installing Java.

jbyoshi@Yoshi ~ % sudo JAVA_HOME=/Library/Java/JavaVirtualMachines/openjdk8/Contents/Home /usr/libexec/java_home -f -v 1.8+
/Library/Java/JavaVirtualMachines/openjdk8/Contents/Home

In addition, unsetting the environment variable on a non-sudo environment reproduces the error:

jbyoshi@Yoshi ~ % unset JAVA_HOME
jbyoshi@Yoshi ~ % /usr/libexec/java_home -f -v 1.8+
The operation couldn’t be completed. Unable to locate a Java Runtime that supports (null).
Please visit http://www.java.com for information on installing Java.

Strangely, the command always outputs the value provided for $JAVA_HOME when it's provided, even if the versions aren't compatible.

jbyoshi@Yoshi ~ % sudo JAVA_HOME=/Library/Java/JavaVirtualMachines/openjdk8/Contents/Home /usr/libexec/java_home -f -v 15+ 
/Library/Java/JavaVirtualMachines/openjdk8/Contents/Home

comment:12 Changed 3 months ago by Tatsh (Andrew Udvare)

Cc: Tatsh removed

comment:13 Changed 3 months ago by ericmoret

Cc: ericmoret added

comment:14 Changed 3 months ago by cjones051073 (Chris Jones)

Cc: cjones051073 added

comment:15 Changed 3 months ago by catap (Kirill A. Korinsky)

I've discovered that -f is a cause for strange behaviour:

catap@Kirills-mini-m1 ~ % env | grep JAVA
catap@Kirills-mini-m1 ~ % /usr/libexec/java_home -V       
Matching Java Virtual Machines (2):
    16 (arm64) "Azul Systems, Inc." - "Zulu 16.0.65-ea" /Library/Java/JavaVirtualMachines/zulu-jdk16/Contents/Home
    11.0.9.1 (arm64) "Azul Systems, Inc." - "Zulu 11.43.1007" /Library/Java/JavaVirtualMachines/zulu-jdk11/Contents/Home
/Library/Java/JavaVirtualMachines/zulu-jdk16/Contents/Home
catap@Kirills-mini-m1 ~ % /usr/libexec/java_home -v 1.8+
/Library/Java/JavaVirtualMachines/zulu-jdk16/Contents/Home
catap@Kirills-mini-m1 ~ % /usr/libexec/java_home -f -v 1.8+
The operation couldn’t be completed. Unable to locate a Java Runtime that supports (null).
Please visit http://www.java.com for information on installing Java.

catap@Kirills-mini-m1 ~ % 

I have no idea that -f means because I haven't found it inside man page for java_home for mac OS Catalina and Big Sur.

=> I've opened PR: https://github.com/macports/macports-ports/pull/9264 that solved issue on two machines with Big Sur.

comment:16 Changed 3 months ago by cjones051073 (Chris Jones)

The fix you propose changes the behaviour of the command in ways we do not want. It means the command will, on failure to find a matching JDK return the default. e.g.

Oberon ~/Projects/MacPorts/ports > /usr/libexec/java_home -h
Usage: java_home [options...]
    Returns the path to a Java home directory from the current user's settings.

Options:
    [-v/--version   <version>]       Filter versions (as if JAVA_VERSION had been set in the environment).
    [-a/--arch      <architecture>]  Filter architecture (as if JAVA_ARCH had been set in the environment).
    [-F/--failfast]                  Fail when filters return no JVMs, do not continue with default.
    [   --exec      <command> ...]   Execute the $JAVA_HOME/bin/<command> with the remaining arguments.
    [-X/--xml]                       Print full JVM list and additional data as XML plist.
    [-V/--verbose]                   Print full JVM list with architectures.
    [-h/--help]                      This usage information.

Oberon ~/Projects/MacPorts/ports > /usr/libexec/java_home -v 16+    
/Library/Java/JavaVirtualMachines/openjdk15/Contents/Home

we do not want this, as it means an inappropriate JDK will be returned on macOS 11.

This is a I think clearly a bug on macOS 11, and until Apple fixes it the current workaround in the PG is the best option.

Last edited 3 months ago by cjones051073 (Chris Jones) (previous) (diff)

comment:17 Changed 3 months ago by breun (Nils Breunese)

MacPorts uses -f, but /usr/libexec/java_home -h only shows -F/--failfast (at least on 10.15). On which versions of macOS does /usr/libexec/java_home -h list -f as a valid option?

comment:18 Changed 3 months ago by catap (Kirill A. Korinsky)

Cc: catap added

comment:19 Changed 2 months ago by Aaron Madlon-Kay <amake@…>

In c08c2d0d7053492a0782203b939754f2233f5638/macports-ports (master):

java-1.0 portgroup: better workaround for Big Sur issue

See #61445

comment:20 in reply to:  17 Changed 2 months ago by amake (Aaron Madlon-Kay)

Replying to breun:

MacPorts uses -f, but /usr/libexec/java_home -h only shows -F/--failfast (at least on 10.15). On which versions of macOS does /usr/libexec/java_home -h list -f as a valid option?

It appears that we've been using -f since my commit here in October 2018: [4c0d76dc7a6bf8a319e3c23248abb15445f6dc16/macports-ports]

My OS X 10.8 VM also shows only -F/--failfast. I couldn't confirm 100% that -f and -F behave the same way because I don't have a JVM on the VM and it looks like a pain to get now (that Safari is too old for modern TLS, I guess).

We could change it to -F but that's clearly not the cause of this particular problem.

Last edited 2 months ago by ryandesign (Ryan Schmidt) (previous) (diff)

comment:21 Changed 2 months ago by amake (Aaron Madlon-Kay)

Cc: amake added

comment:22 Changed 2 months ago by Aaron Madlon-Kay <amake@…>

In 292365f0d2ddc7969e5e91c5201e5b3264924685/macports-ports (master):

java-1.0 portgroup: wildcards not supported on Big Sur

See #61445

comment:23 Changed 2 months ago by Aaron Madlon-Kay <amake@…>

In 79582c24624bdad52679a66a3872206962e319b9/macports-ports (master):

java-1.0 portgroup: restore skip if check fails on Big Sur

The last couple of changes reduced the chance of spurious warnings, but there
are still some failure cases: for example maven3 wants Java 1.7+, which is
handled as 1.7, but we don't have a port for Java 1.7.

See #61445

comment:24 Changed 2 months ago by breun (Nils Breunese)

I could add a subport for Java 1.7 (aka Java 7), but Java 7 reached end of life in 2011, so that doesn't sound like great idea. We could bump that maven3 dependency to Java 1.8+, but technically Maven 3 requires "JDK 1.7 or above to execute" according to https://maven.apache.org/download.cgi so we could create an issue for anyone that really wants to run Maven 3 on JDK 1.7 for some obscure reason (I can't think of any good ones though).

comment:25 Changed 7 weeks ago by Janosch (Janosch Peters)

As I am quiet annyoed by this bug I made an attempt to fix it. Here is the description from github:

Instead of using /usr/libexec/java_home -f -v this piece of code uses /usr/libexec/java_home -V and does the parsing of existing JVMs and the matching with ${java.version} on its own. The code only compares major versions (1.7, 1.8, 9, 15). Asterisk and plus notation is support, but only attached to the major version. 1.8+, 9* will work, 1.7.35+ won't. If the plus suffix is used and more than one candidate is a match, the highest JVM version wins. If there are multiple JVMs with the same major version, it just picks any one of those.

Please have a look at the PR https://github.com/macports/macports-ports/pull/9617

comment:26 in reply to:  25 ; Changed 5 weeks ago by telotortium (Robert Irelan)

Replying to Janosch:

I patched ​https://github.com/macports/macports-ports/pull/9617 into my installation, but I also needed to install openjdk8 before commons-lang3 (a dependency of pdftk-java) would build.

comment:27 in reply to:  26 Changed 5 weeks ago by Janosch (Janosch Peters)

Replying to telotortium:

Replying to Janosch:

I patched ​https://github.com/macports/macports-ports/pull/9617 into my installation, but I also needed to install openjdk8 before commons-lang3 (a dependency of pdftk-java) would build.

Hm, interesting. This does not happen in my environment for commons-lang3. Could you please post the output of /usr/libexec/java_home -V in the comment section of the PR? Thanks.

Note: See TracTickets for help on using tickets.