Opened 2 years ago

Closed 12 months ago

Last modified 12 months ago

#49317 closed enhancement (fixed)

python*: Allow loading SQLite extensions

Reported by: m3drano@… Owned by: larryv (Lawrence Velázquez)
Priority: Normal Milestone:
Component: ports Version:
Keywords: haspatch Cc: petrrr, sonkm3 (sonkm3), daniel@…, jyrkiwahlstedt, dmarteau (David Marteau)
Port: python27 python32 python33 python34 python35 python36

Description

This patch adds the flag --enable-loadable-sqlite-extensions to the Python build, which allows to load extensions in SQLite databases natively. There is more information on the subject in the Python's documentation: https://docs.python.org/3.4/library/sqlite3.html#f1

This setting is enabled on other OSes as default too, such as Linux distributions (Debian, Arch) at least.

This patch applies to the port python34, but we can certainly extend it to other Python versions too if it is deemed appropriate. Other option would be to make this a port variant.

Attachments (1)

change-python34-maintainer.diff (548 bytes) - added by m3drano@… 2 years ago.

Download all attachments as: .zip

Change History (16)

Changed 2 years ago by m3drano@…

comment:1 Changed 2 years ago by ryandesign (Ryan Schmidt)

  • Cc jwa@… openmaintainer@… removed
  • Keywords haspatch added
  • Owner changed from macports-tickets@… to jwa@…

If there is no downside to enabling this, it should be on by default, not a variant. If there is no reason why it cannot be done to other python versions too, that should be done also; unnecessary differences between versions of python ports should be avoided. Since adding this configure argument changes the files the port installs, the port's revision must be increased.

The web page you mentioned above says:

The sqlite3 module is not built with loadable extension support by default, because some platforms (notably Mac OS X) have SQLite libraries which are compiled without this feature. To get loadable extension support, you must pass –enable-loadable-sqlite-extensions to configure

Perhaps they are referring to Apple's SQLite libraries in OS X, and this does not apply to the our SQLite libraries in MacPorts? Did you verify that adding this configure argument and rebuilding the port had the desired effect?

comment:2 Changed 2 years ago by petrrr

  • Cc petr@… added

Cc Me!

comment:3 Changed 2 years ago by m3drano@…

I agree it should be on by default. I have tested this with the SQLite version provided by MacPorts (3.8.11.2) and the system default in El Capitan (3.8.10.2). It works perfectly with the port, both with the sqlite3 tool and the python library. The default sqlite3 install has extension loading disabled (both on the sqlite3 tool and the Python 3 version provided).

$ /usr/bin/sqlite3 
SQLite version 3.8.10.2 2015-05-20 18:17:19
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
sqlite> .load liblibsqlitefunctions.so;
Error: unknown command or invalid arguments:  "load". Enter ".help" for help

$ sqlite3 
SQLite version 3.8.11.1 2015-07-29 20:00:57
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
sqlite> .load liblibsqlitefunctions.so

$ /usr/bin/python
Python 2.7.10 (default, Aug 22 2015, 20:33:39) 
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.59.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
>>> conn = sqlite3.connect('test.db')
>>> conn.enable_load_extensions(True)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'sqlite3.Connection' object has no attribute 'enable_load_extensions'
>>> 

$ python
Python 3.4.3 (default, Oct 16 2015, 19:24:20) 
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.72)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
>>> conn = sqlite3.connect('tmp.db')
>>> conn.enable_load_extension(True)
>>> conn.load_extension('liblibsqlitefunctions.so')
>>>

comment:4 Changed 20 months ago by m3drano@…

Do we have any update regarding this change proposal? I think we should enable it on all Python versions.

comment:5 Changed 19 months ago by sonkm3 (sonkm3)

  • Cc sonkm3@… added

Cc Me!

comment:6 Changed 14 months ago by daniel@…

  • Cc daniel@… added

Cc Me!

comment:7 Changed 14 months ago by daniel@…

I'm currently using the following super-trivial variant with the python 3.5 port in order to use loadable modules, so far without any issues, on OS X 10.11.6:

variant sqliteext description {Enable loadable sqlite extensions} {
    configure.args-append   --enable-loadable-sqlite-extensions
}

No other modifications were necessary. When using python3.5 from /opt/local, conn.enable_load_extensions(True) works and sqlite extensions can be loaded.

Patching portfiles locally is a major PITA, it would be great if this could be enabled by default in all python 3.x ports, or at least be an available variant in all python 3.x ports.

Last edited 14 months ago by daniel@… (previous) (diff)

comment:8 Changed 14 months ago by larryv (Lawrence Velázquez)

  • Cc larryv@… added

Cc Me!

comment:9 follow-up: Changed 13 months ago by dmarteau (David Marteau)

Please, is there currently a roadmap for this bug ?

Since pyspatialite is now totally obsolete (run only with spatialite 3), the only way to use spatialite with python is loading the extension explicitely. Do not been able to load sqlite extension in python on OSX is really painful.

Thx

comment:10 in reply to: ↑ 9 ; follow-up: Changed 13 months ago by larryv (Lawrence Velázquez)

  • Cc jwa@… added; larryv@… removed
  • Owner changed from jwa@… to larryv@…
  • Status changed from new to assigned

Sorry for the lack of action here. I think we should enable this on all Python ports and will do so this week, if Jyrki doesn’t object.

comment:11 in reply to: ↑ 10 Changed 13 months ago by jyrkiwahlstedt

Replying to larryv@…:

Sorry for the lack of action here. I think we should enable this on all Python ports and will do so this week, if Jyrki doesn’t object.

Go ahead, please!

comment:12 Changed 12 months ago by dmarteau (David Marteau)

  • Cc dhmarteau@… added

Cc Me!

comment:13 follow-up: Changed 12 months ago by m3drano@…

ping?

comment:14 in reply to: ↑ 13 Changed 12 months ago by larryv (Lawrence Velázquez)

  • Port python27 python32 python33 python35 python36 added
  • Resolution set to fixed
  • Status changed from assigned to closed
  • Summary changed from python34 @3.4.3 Allow loading SQLite extensions to python*: Allow loading SQLite extensions
  • Version 2.3.4 deleted

Sorry again for the delay. Enabled for all relevant Python ports in r153800. Thanks for your patience.

comment:15 Changed 12 months ago by m3drano@…

Thanks a lot for the change!

Note: See TracTickets for help on using tickets.