Failed to initialize MacPorts, sqlite error: near "COLUMN": syntax error (1) while executing query: ALTER TABLE registry.ports RENAME COLUMN negated_variants TO requested_variants

[peterruoff]

Hi,

after updating today Macports to ver 2.7.0 by using selfupdate the port commands do not longer work. I get the following message:

peter$ sudo port upgrade outdated
Password:
sqlite error: near "COLUMN": syntax error (1) while executing query: ALTER TABLE registry.ports RENAME COLUMN negated_variants TO requested_variants
    while executing
"registry::open $db_path"
    (procedure "mportinit" line 712)
    invoked from within
"mportinit ui_options global_options global_variations"
Error: /opt/local/bin/port: Failed to initialize MacPorts, sqlite error: near "COLUMN": syntax error (1) while executing query: ALTER TABLE registry.ports RENAME COLUMN negated_variants TO requested_variants
Peters-iMac:dropbox peter$ 

Any other port command shows the same error. I use macOS Mojave version 10.14.6, iMac (Retina 4K, 21.5-inch, 2017).

best wishes Peter Ruoff

[jmroot (Joshua Root)]

Looks like the sqlite3 headers base was built against don't match the library. Can you show the output of sudo port -df selfupdate? And also this:

grep "#define SQLITE_VERSION_NUMBER" /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/sqlite3.h

[raimue (Rainer Müller)]

Same problem for macOS 10.14 was also reported on macports-users: ​https://lists.macports.org/pipermail/macports-users/2021-May/049986.html

[ryandesign (Ryan Carsten Schmidt)]

If anyone experiences this on a system other than Mojave, let us know here. Letting us know your OS version, Xcode version, CLT version, and whether you updated by selfupdate or via the installer would help.

If you experience this problem and you upgraded to 2.7.0 via sudo port selfupdate, try downloading and running the installer from our web site. Does that fix it?

It looks like the 10.14 SDK has SQLite 3.24.0 headers while the 10.15 SDK has SQLite 3.28.0 headers. So maybe this problem could arise if you built base using the 10.15 SDK (which might happen if you have Xcode 11.x and/or the Xcode 11.x version of the CLT) and then run it on 10.14.

[ryandesign (Ryan Carsten Schmidt)]

Datapoint: We have not seen this problem on our macOS 10.14 buildbot worker, which uses Xcode 10 and thus the 10.14 SDK.

[peterruoff]

After installing MacPorts for Mojave again (after the above suggestion by Ryan; thanks!) and then doing the selfupdate and the upgrade, all is working fine now on my iMac.

I have another machine, a Mac Pro (Late 2013), running the same Mojave version as my iMac, but there no problems occurred at all; there I could upgrade MacPorts via selfupdate to the new version 2.7.0 without problems.

There was a difference, however, between the two machines: on the MacPro only 10.14 SDK was installed, while on the iMac there was an additional 10.15 SDK install (by some reason I no longer recall). Perhaps the additional 10.15 SDK caused the trouble during the upgrade by selfupdate? But both are still there after the successful reinstall of MacPorts on the iMac.

[jmroot (Joshua Root)]

Replying to peterruoff:

After installing MacPorts for Mojave again (after the above suggestion by Ryan; thanks!) and then doing the selfupdate and the upgrade, all is working fine now on my iMac.

You used the binary installer, I take it?

There was a difference, however, between the two machines: on the MacPro only 10.14 SDK was installed, while on the iMac there was an additional 10.15 SDK install (by some reason I no longer recall).

OK, so I guess the compiler is picking the later SDK unless told otherwise. The good news is that that situation should be handled correctly by [5a46743a6bd437d591985e514f471ed0441ea554/macports-base]. The bad news is that that doesn't help anyone selfupdating from 2.6.4.

Would you mind compiling and running the attached test program on both your systems to confirm the nature of the issue? (Instructions in the comment at the top of the file.)

[jmroot (Joshua Root)]

program to check sqlite versions

[jmroot (Joshua Root)]

And if others seeing the issue could also show the test program output and state which SDKs are installed on their system, that would be helpful data.

[jmroot (Joshua Root)]

In 0249b3cf7e2fcb721e5ecae38e632a23007e647d/macports-base (master):

Check SQLite runtime version

Check sqlite3_libversion_number() directly at runtime where it's not
too unwieldy to do so. In the other cases, still check at build time,
but using a new macro MP_SQLITE_VERSION, which is the lesser of
the SQLITE_VERSION_NUMBER and sqlite3_libversion_number() values found
at configure time. It's not binary portability, but it should be enough
for the case where selfupdate has to build against an SDK that doesn't
match the OS version it's building on.

Fixes: #62929

[jmroot (Joshua Root)]

Optimistically calling this fixed by the above commit, but please still provide the information requested earlier.

If you want to try out the fix, note that the error happens when upgrading the database. That means if you already reinstalled from the binary pkg, your database has been upgraded successfully, and you won't be able to reproduce the error even if you install a build that has the bug. (If you really want to try to reproduce in that situation, you have to first uninstall MacPorts completely, install 2.6.4 to get a registry database in the old format, and then install the new build. Probably want to do it in a separate throwaway prefix if you do it.)

[fiee (Henning Hraban Ramm)]

I reported the problem on the users list; sorry, couldn’t check earlier.

MacOS 10.14.6, upgrading MacPorts from 2.6.4

sqlitevers:
SQLite header version: 3028000,
SQLite runtime version: 3024000

[jmroot (Joshua Root)]

Replying to fiee:

MacOS 10.14.6, upgrading MacPorts from 2.6.4

sqlitevers:
SQLite header version: 3028000,
SQLite runtime version: 3024000

Thanks. And your SDK situation involves 10.15 I assume? Is 10.14 present as well?

[apjanke (Andrew Janke)]

I think I have a successful test here. I set up a fresh Mojave VM in VMware, with Xcode 11.3.1 which I *think* uses macOS 10.15 SDK by default, installed MacPorts 2.6.4 on it, upgraded it to MacPorts 2.7.0 from the release-2.7 branch in Git, and it seemed to work.

Host name: portia2 Fresh macOS 10.14.6 install, with all system updates applied.

Steps done:

• Install Xcode 11.3.1 from a downloaded .xzip file and run it to accept the EULA
• Install CLT with xcode-select --install
• System updates
• Install MacPorts 2.6.4 using the pkg file downloaded from the GitHub Releases page at ​https://github.com/macports/macports-base/releases/tag/v2.6.4
• Do NOT do sudo port selfupdate
• sudo port install wget to test port functionality
• Install MacPorts tip of release-2.7 branch on top of the 2.6.4 install
  • cd ~/local/repos
  • git clone https://github.com/macports/macports-base
  • cd macports-base
  • git checkout release-2.7
  • Install per ​https://guide.macports.org/#installing.macports.git
    • ./configure --enable-readline && make && sudo make install && make distclean
• Test port by installing sqlite
  • sudo port install sqlite3
  • → Success!

portia2:macports-base janke$ port -v
MacPorts 2.7.0
Entering shell mode... ("help" for help, "quit" to quit)
portia2:macports-base janke$ sudo port install sqlite3
--->  Computing dependencies for sqlite3
--->  Fetching archive for sqlite3
--->  Attempting to fetch sqlite3-3.35.5_0.darwin_18.x86_64.tbz2 from https://ywg.ca.packages.macports.org/mirror/macports/packages/sqlite3
--->  Attempting to fetch sqlite3-3.35.5_0.darwin_18.x86_64.tbz2.rmd160 from https://ywg.ca.packages.macports.org/mirror/macports/packages/sqlite3
--->  Installing sqlite3 @3.35.5_0
--->  Activating sqlite3 @3.35.5_0
--->  Cleaning sqlite3
--->  Scanning binaries for linking errors
--->  No broken files found.
--->  No broken ports found.
portia2:macports-base janke$ sudo port upgrade outdated
Nothing to upgrade.
--->  Scanning binaries for linking errors
--->  No broken files found.
--->  No broken ports found.
portia2:macports-base janke$ 

[jmroot (Joshua Root)]

Replying to apjanke:

Xcode 11.3.1 which I *think* uses macOS 10.15 SDK by default

You can check that by running plutil -p `xcrun --show-sdk-path`/SDKSettings.plist | fgrep CanonicalName and by compiling and running the C test program attached to this ticket. (xcrun --show-sdk-version should theoretically also work but seems to encounter errors sometimes.)

Host name: portia2 Fresh macOS 10.14.6 install, with all system updates applied.

Steps done:

• Install Xcode 11.3.1 from a downloaded .xzip file and run it to accept the EULA
• Install CLT with xcode-select --install
• System updates
• Install MacPorts 2.6.4 using the pkg file downloaded from the GitHub Releases page at ​https://github.com/macports/macports-base/releases/tag/v2.6.4
• Do NOT do sudo port selfupdate

Unfortunately (for our current purpose) the pkg will run selfupdate as part of its postflight script. So if that succeeded, you would now have 2.7.0. (Run port version to confirm.)

• sudo port install wget to test port functionality

… and if you have 2.7.0 and this worked, you are not affected by the issue.

The rest of the procedure looks fine. Thanks for your efforts.

[mashavecher]

10.14.6 SDK 10.15

MacPorts 2.6.4
SQLite header version: 3028000
SQLite runtime version: 3024000

git checkout release-2.7
./configure && make && sudo make install

MacBook-Pro:~ mashavecher$ port installed
Warning: port definitions are more than two weeks old, consider updating them by running 'port selfupdate'.
sqlite error: attempt to write a readonly database (8) while executing query: DROP TABLE registry.ports
    while executing
"registry::open $db_path"
    (procedure "mportinit" line 712)
    invoked from within
"mportinit ui_options global_options global_variations"
Error: /opt/local/bin/port: Failed to initialize MacPorts, sqlite error: attempt to write a readonly database (8) while executing query: DROP TABLE registry.ports

[jmroot (Joshua Root)]

@mashavecher: you may need to run any port command with sudo first to allow it to upgrade the database? Not sure why that wouldn't have happened during make install though.

[macportsraf]

I installed MacPorts-2.7.0-10.14-Mojave.pkg and everything is fine again, so thanks for that suggestion. And thanks for fixing the real problem as well.