py-tensorflow-data-validation: build requires bazel, but latter not added to build dependencies

[mascguy (Christopher Nielsen)]

Failure seen when installing port from source:

:info:build running build
:info:build running bazel_build
:info:build Traceback (most recent call last):
:info:build   File "setup.py", line 148, in <module>
:info:build     setup(
:info:build   File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/setuptools/__init__.py", line 153, in setup
:info:build     return distutils.core.setup(**attrs)
:info:build   File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/distutils/core.py", line 148, in setup
:info:build     dist.run_commands()
:info:build   File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/distutils/dist.py", line 966, in run_commands
:info:build     self.run_command(cmd)
:info:build   File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/distutils/dist.py", line 985, in run_command
:info:build     cmd_obj.run()
:info:build   File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/distutils/command/build.py", line 135, in run
:info:build     self.run_command(cmd_name)
:info:build   File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/distutils/cmd.py", line 313, in run_command
:info:build     self.distribution.run_command(command)
:info:build   File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/distutils/dist.py", line 984, in run_command
:info:build     cmd_obj.ensure_finalized()
:info:build   File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/distutils/cmd.py", line 107, in ensure_finalized
:info:build     self.finalize_options()
:info:build   File "setup.py", line 70, in finalize_options
:info:build     raise RuntimeError(
:info:build RuntimeError: Could not find "bazel" binary. Please visit https://docs.bazel.build/versions/master/install.html for installation instruction.

Digging into the problem a bit: It looks like the Bazel portgroup doesn't add bazel as a build dependency, when bazel.build_cmd is set to an empty string. (And the portfile for py-tensorflow-data-validation sets it to empty.) The problem is, per the logs above, Bazel is needed.

The Bazel portgroup was changed on 4/13/2021, adding the aforementioned behavior:

​https://github.com/macports/macports-ports/commit/8c0bd91205487930883d7bc8c8fa866ec569964c

It's not clear whether this is a problem with the Bazel portgroup, or a combination of factors.

[mascguy (Christopher Nielsen)]

Assigning to the portgroup author, CJones, as a starting point.

[cjones051073 (Chris Jones)]

yeah, there's a slight issue here. I'll take a look/

[Chris Jones <jonesc@…>]

In 2b77ac6557b5631c90a47ca6de47138f67c5a748/macports-ports (master):

bazel PG: Fix issue with declaring bazel dep when build command is set to null
Closes: #62725

[mascguy (Christopher Nielsen)]

Chris, thanks for the quick fix. That solves the build dependency issue, and ensures Bazel is installed.

But now I'm seeing the following failure while building py-tensorflow-data-validation:

:info:build ERROR: /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-tensorflow-data-validation/py38-tensorflow-data-validation/work/bazel_build/8f919143399337036c8563e76d591ca2/external/zlib_archive/BUILD.bazel:5:11: undeclared inclusion(s) in rule '@zlib_archive//:zlib':
:info:build this rule is missing dependency declarations for the following files included by 'zlib_archive/adler32.c':
:info:build   '/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/limits.h'
:info:build   '/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h'
:info:build   '/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_symbol_aliasing.h'
[...]

(There were 125-ish headers listed from the MacOS SDK, but I truncated it for brevity.)

I did some quick Googling, but it wasn't immediately clear how to fix this.

Any thoughts?

[cjones051073 (Chris Jones)]

Yeah, I've seen this come and go. I have no good solution other than sometimes blacklist the system clang so macports compiler is used works...

[mascguy (Christopher Nielsen)]

From what I can see in the logs, MacPorts' clang is being used, rather than the system/Xcode version.

Any other ideas?

[cjones051073 (Chris Jones)]

not at this point I am afraid, I would have to experiment. All I can say is I have seen it before but never been able to work out exactly why...

[mascguy (Christopher Nielsen)]

I'm seeing similar errors on MacOS 10.15, too. (The ones in my previous comment were from MacOS 10.14.)

And the MacOS SDK is clearly being found, based on the paths:

:info:build ERROR: /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-tensorflow-data-validation/py38-tensorflow-data-validation/work/bazel_build/8f919143399337036c8563e76d591ca2/external/com_google_protobuf/BUILD:106:11: undeclared inclusion(s) in rule '@com_google_protobuf//:protobuf_lite':
:info:build this rule is missing dependency declarations for the following files included by 'com_google_protobuf/src/google/protobuf/arena.cc':
:info:build   '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/pthread.h'
:info:build   '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/string.h'
:info:build   '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/_types.h'
[...]

So I'm wondering if there's either a bug in Bazel 3.7.2, and/or some configuration needs to be changed...?

[cjones051073 (Chris Jones)]

I don't know... sorry, really not an expert on bazel...

you could try changing to use bazel 3.1 or 3.5 and see if that helps...

[mascguy (Christopher Nielsen)]

Okay. I'll create another ticket, specific to the build errors.

[mascguy (Christopher Nielsen)]

New ticket is: issue:62728