Apache2 / mod_wsgi / python 2.6 crash

[nerdrew@…]

The following mod_wsgi script fails with the MacPorts Apache but works with the Apple version of Apache.

The script:

import Carbon.File

def application(environ, start_response):
    status = '200 OK'
    output = "hello world"

    response_headers = [('Content-type', 'text/plain'), 
                        ('Content-Length', str(len(output)))]
    start_response(status, response_headers)

    return [output]

I have tested it with the Apple supplied Python 2.6 and the MacPorts Python 2.6, which makes me think the problem is somehow related to the MacPorts version of Apache.

The Apache error log has useless info. The error is:

[info] [client ::1] mod_wsgi (pid=52370, process='test', application=''): Loading WSGI script '/opt/local/apache2/htdocs/test/test.wsgi'.
[error] [client ::1] Premature end of script headers: test.wsgi
[notice] child pid 52370 exit signal Trace/BPT trap (5)
[info] mod_wsgi (pid=52370): Process 'test' has died, restarting.

[nerdrew@…]

One more thing, the

import Carbon.File

is the line that causes it to crash.

[blb@…]

Does the simple act of

$ /opt/local/bin/python2.6 -c 'import Carbon.File'

cause the crash? If so, can you attach the crash log, should be in ~/Library/Logs/CrashReporter.

[nerdrew@…]

No /opt/local/bin/python2.6 -c 'import Carbon.File' does not cause a crash. It is just with the Macports Apache and mod_wsgi that the script above crashes.

[blb@…]

Okay, so not python alone is crashing; when the crash happens, does httpd generate a crash log (most likely in /Library/Logs/CrashReporter)? If it does, does it look like the crash log in ticket #20286 (specifically at ​http://paste.pocoo.org/show/133699/ where the interesting thing is the frequent calls to PyEval_EvalCodeEx and PyEval_EvalFrameEx)?

[nerdrew@…]

Apache Crash Report

[nerdrew@…]

I've added the relevant crash report. I never remember to look at the crash reports. There are calls to PyEval_EvalCodeEx, and PyEval_EvalFrameEx, but not the recursive problem from the link above.

[nerdrew@…]

One thing to note is: using the Apple supplied httpd and the macports python, the above wsgi script works just fine. The crash report I attached shows that the Apple supplied python (2.6.1) is being used when httpd crashes.

[blb@…]

Hmm, that definitely should be using MacPorts' python, and may explain the crash; what is the output from the following?

$ otool -L /opt/local/apache2/modules/mod_wsgi.so

[nerdrew@…]

I have experimented with linking mod_wsgi to the built in Apple python (2.6.1) and to the MacPorts python (2.6.4). The first crash report I attached is for the mod_wsgi that is linked to the built in Apple python. I will attach the crash report for the one linked to the MacPorts python.

Here is the output of otool -L for all the mod_wsgi.so's I've tested:

 $ otool -L mod_wsgi-*
mod_wsgi-mpap-mppy.so:
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 124.1.1)
	/opt/local/Library/Frameworks/Python.framework/Versions/2.6/Python (compatibility version 2.6.0, current version 2.6.1)
mod_wsgi-mpap-stdpy.so:
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 124.1.1)
	/System/Library/Frameworks/Python.framework/Versions/2.6/Python (compatibility version 2.6.0, current version 2.6.1)
mod_wsgi-stdap-mppy.so:
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 124.1.1)
	/opt/local/Library/Frameworks/Python.framework/Versions/2.6/Python (compatibility version 2.6.0, current version 2.6.1)
mod_wsgi-stdap-stdpy.so:
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 124.1.1)
	/System/Library/Frameworks/Python.framework/Versions/2.6/Python (compatibility version 2.6.0, current version 2.6.1)

Note:

mpap = MacPorts Apache (2.2.14)
stdap = Apple Apache (2.2.11)
mppy = MacPorts Python (2.6.4)
stdpy = Apple Python (2.6.1)

I link the mod_wsgi I want to test to the module folder I want to test (I tried all of the combinations of Apple's Apache, MacPorts Apache, Apple's Python, and MacPorts Python) to narrow down the problem. Apple's Apache works with both Apple's Python and MacPorts Python. MacPorts Apache does NOT work with either Apple's Python or MacPorts Python.

[nerdrew@…]

Apache Crash Report - MacPorts Apache with MacPorts Python

[blb@…]

Hmm, mod_wsgi 2.5 crashes for me as well but in an entirely different way. I note 2.6 is current, so I updated mod_wsgi to that in r60651, which runs quite nicely for me. Give it a try, see if it is any better.

[nerdrew@…]

I have updated mod_wsgi to 2.6 or r60651, and Apache httpd still crashes for me.

[blb@…]

Is the crash log the same?

[nerdrew@…]

Yes it is. I will attach it anyway.

[nerdrew@…]

Apache Crash Report with mod_wsgi 2.6

[blb@…]

What other apache modules are you loading, maybe there's another which links to a different python?

[nerdrew@…]

I've tested it with most of the modules disabled. Here's the list of modules that are still active:

LoadModule authn_file_module modules/mod_authn_file.so
LoadModule authz_host_module modules/mod_authz_host.so
LoadModule authz_groupfile_module modules/mod_authz_groupfile.so
LoadModule authz_user_module modules/mod_authz_user.so
LoadModule authz_owner_module modules/mod_authz_owner.so
LoadModule authz_default_module modules/mod_authz_default.so
LoadModule auth_basic_module modules/mod_auth_basic.so
LoadModule auth_digest_module modules/mod_auth_digest.so
LoadModule include_module modules/mod_include.so
LoadModule log_config_module modules/mod_log_config.so
LoadModule env_module modules/mod_env.so
LoadModule mime_magic_module modules/mod_mime_magic.so
LoadModule mime_module modules/mod_mime.so
LoadModule autoindex_module modules/mod_autoindex.so
LoadModule vhost_alias_module modules/mod_vhost_alias.so
LoadModule dir_module modules/mod_dir.so
LoadModule alias_module modules/mod_alias.so
LoadModule rewrite_module modules/mod_rewrite.so
LoadModule wsgi_module modules/mod_wsgi.so

To the best of my knowledge, none of these modules load python.

[nerdrew@…]

The problem seems to be with the worker variant. Using the prefork mpm works (the Apple version of Apache uses the prefork mpm). I do not know why the worker mpm causes the house to fall down.

[ryandesign (Ryan Carsten Schmidt)]

Please test whether mod_wsgi 3.1 still has the problem.

[nerdrew@…]

So, I have done some more research into this. Here is what I have found. The problem should only exist on Snow Leopard, as it seems ​CFRuntime.c changed between 10.5 and 10.6. Looking at void __CFInitialize(void) in CFRuntime.c shows that "CoreFoundation must be initialized on the main thread." To work around this problem while continuing to use the worker mpm with mod_wsgi, you can include the following in your wsgi config file:

WSGIImportScript <your-wsgi-script> process-group=<your-process-group> application-group=%{GLOBAL}

I think this will only work for one script though, but I'm not sure.

[nerdrew@…]

There is a bug opened on openradar for the CoreFoundation problem (though it might not be a bug): ​http://openradar.appspot.com/7209349

[chrstphrchvz (Christopher Chavez)]

The ports involved here have been updated significantly (apache2, mod_wsgi) or are no longer depended on (python26). Is this ticket still relevant, or can it be closed?