py-pyopencl @2023.1.4 does not work with Catalina

[lpagani91]

I installed pyopencl in all flavors from 3.7 to 3.11 but it does not work at all. When running it, I get a BUILD_PROGRAM_FAILURE

Traceback (most recent call last):
  File "test_cl.py", line 38, in <module>
    program =  cl.Program(context, source).build(args)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pyopencl/__init__.py", line 500, in build
    options_bytes=options_bytes)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pyopencl/__init__.py", line 554, in _build_and_catch_errors
    raise err
pyopencl._cl.RuntimeError: clBuildProgram failed: BUILD_PROGRAM_FAILURE - clBuildProgram failed: BUILD_PROGRAM_FAILURE - clBuildProgram failed: BUILD_PROGRAM_FAILURE

Build on <pyopencl.Device 'Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz' on 'Apple' at 0xffffffff>:


(options: -D Nx=128 -I /opt/local_2024/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pyopencl/cl)

I found suggestions on the Web like setting PYOPENCL_NO_CACHE to 1 but it did not help.

[lpagani91]

I have the same problem with my real program or with a simple program such as this one :

import pyopencl as cl
import numpy as np
import os
os.environ['PYOPENCL_NO_CACHE'] = '1'
os.environ['PYOPENCL_COMPILER_OUTPUT'] = '1'

def InitCL(GPU=0, platforms=[0,1,2,3]):
    platform, device, context, queue = None, None, None, None
    for iplatform in platforms:
        print("TRYING PLATFORM %d" % iplatform) 
        try:
            platform = cl.get_platforms()[iplatform]
            if (GPU>0):
                device   = [platform.get_devices(cl.device_type.GPU)[1]]
            else:
                device   = platform.get_devices(cl.device_type.CPU)
            context   =  cl.Context(device)
            queue     =  cl.CommandQueue(context)
            break
        except:
            pass
    print(platform, device)
    return platform, device, context, queue,  cl.mem_flags


platform, device, context, queue, mf = InitCL()

source = """
void __kernel test(__global float *A) {
     int id = get_global_id(0) ;
     if (id>=Nx) return ;
     A[id] = A[id]*A[id] ;
}
"""

N       =  128
args    =  "-D Nx=%d" % N
program =  cl.Program(context, source).build(args)
test    =  program.test
X       =  np.asarray(np.arange(N), np.float32)
X_buf   =  cl.Buffer(context, mf.READ_WRITE, 4*N)
cl.enqueue_copy(queue, X_buf, X)
test(queue, [N,], [1,], X_buf)
cl.enqueue_copy(queue, X, X_buf)
print(X)

[ryandesign (Ryan Carsten Schmidt)]

py-pyopencl @2023.1.4 in MacPorts is available for Python versions from 3.8 to 3.11 but your error message shows you are using Python 3.6 so either you are using a very old MacPorts-installed version of pyopencl from before the py36-pyopencl subport was deleted or you have installed pyopencl there manually. Either way, we don't support that. Use Python 3.8 or later. I installed py311-pyopencl, saved your test script as test.py and ran python3.11 test.py and got this output:

TRYING PLATFORM 0
<pyopencl.Platform 'Apple' at 0x7fff0000> [<pyopencl.Device 'Intel(R) Core(TM) i7-3720QM CPU @ 2.60GHz' on 'Apple' at 0xffffffff>]
[0.0000e+00 1.0000e+00 4.0000e+00 9.0000e+00 1.6000e+01 2.5000e+01
 3.6000e+01 4.9000e+01 6.4000e+01 8.1000e+01 1.0000e+02 1.2100e+02
 1.4400e+02 1.6900e+02 1.9600e+02 2.2500e+02 2.5600e+02 2.8900e+02
 3.2400e+02 3.6100e+02 4.0000e+02 4.4100e+02 4.8400e+02 5.2900e+02
 5.7600e+02 6.2500e+02 6.7600e+02 7.2900e+02 7.8400e+02 8.4100e+02
 9.0000e+02 9.6100e+02 1.0240e+03 1.0890e+03 1.1560e+03 1.2250e+03
 1.2960e+03 1.3690e+03 1.4440e+03 1.5210e+03 1.6000e+03 1.6810e+03
 1.7640e+03 1.8490e+03 1.9360e+03 2.0250e+03 2.1160e+03 2.2090e+03
 2.3040e+03 2.4010e+03 2.5000e+03 2.6010e+03 2.7040e+03 2.8090e+03
 2.9160e+03 3.0250e+03 3.1360e+03 3.2490e+03 3.3640e+03 3.4810e+03
 3.6000e+03 3.7210e+03 3.8440e+03 3.9690e+03 4.0960e+03 4.2250e+03
 4.3560e+03 4.4890e+03 4.6240e+03 4.7610e+03 4.9000e+03 5.0410e+03
 5.1840e+03 5.3290e+03 5.4760e+03 5.6250e+03 5.7760e+03 5.9290e+03
 6.0840e+03 6.2410e+03 6.4000e+03 6.5610e+03 6.7240e+03 6.8890e+03
 7.0560e+03 7.2250e+03 7.3960e+03 7.5690e+03 7.7440e+03 7.9210e+03
 8.1000e+03 8.2810e+03 8.4640e+03 8.6490e+03 8.8360e+03 9.0250e+03
 9.2160e+03 9.4090e+03 9.6040e+03 9.8010e+03 1.0000e+04 1.0201e+04
 1.0404e+04 1.0609e+04 1.0816e+04 1.1025e+04 1.1236e+04 1.1449e+04
 1.1664e+04 1.1881e+04 1.2100e+04 1.2321e+04 1.2544e+04 1.2769e+04
 1.2996e+04 1.3225e+04 1.3456e+04 1.3689e+04 1.3924e+04 1.4161e+04
 1.4400e+04 1.4641e+04 1.4884e+04 1.5129e+04 1.5376e+04 1.5625e+04
 1.5876e+04 1.6129e+04]

I'm confused that there are two prefixes mentioned in your errors: /opt/local and /opt/local_2024. What's the relationship between those two paths? Is one a symlink to the other or are they separate MacPorts installations?

[lpagani91]

Hi, sorry for the Python3.6, I also tested with 3.8 to 3.11 none is working.

Indeed /opt/local is a symlink to /local_2024 (the latest update of Macports and everything) or /local_xemacs a previous version (for Darwin 16 but seems to work with Darwin 19 as well) to run Xemacs (see my other ticket for this one)

[lpagani91]

python3.11 --version
Python 3.11.8

python3.11 test_cl.py
TRYING PLATFORM 0
<pyopencl.Platform 'Apple' at 0x7fff0000> [<pyopencl.Device 'Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz' on 'Apple' at 0xffffffff>]
Traceback (most recent call last):
  File "/Users/laurent/Transfert-Radiatif/LOC/test_cl.py", line 38, in <module>
    program =  cl.Program(context, source).build(args)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyopencl/__init__.py", line 527, in build
    self._build_and_catch_errors(
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyopencl/__init__.py", line 583, in _build_and_catch_errors
    raise err
pyopencl._cl.RuntimeError: clBuildProgram failed: BUILD_PROGRAM_FAILURE - clBuildProgram failed: BUILD_PROGRAM_FAILURE - clBuildProgram failed: BUILD_PROGRAM_FAILURE

Build on <pyopencl.Device 'Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz' on 'Apple' at 0xffffffff>:


(options: -D Nx=128 -I /opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyopencl/cl)

[lpagani91]

I am happy to know it works for you but how can I find the difference between your install and mine to explain the difference and cure it in the end ?