libedit: unable to enter non-ASCII characters

[raimue (Rainer Müller)]

The sqlite3 shell does not allow typing non-ASCII characters. To reproduce this, open sqlite3 and try to copy and paste "Müller" into a query. The umlaut will be skipped and you end up with "Mller".

This works correctly with /usr/bin/sqlite3 on macOS 10.12.1, so it is probably not an upstream problem, but in the packaging.

$ otool -L /opt/local/bin/sqlite3
/opt/local/bin/sqlite3:
	/opt/local/lib/libedit.0.dylib (compatibility version 1.0.0, current version 1.54.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1238.0.0)
$ otool -L /usr/bin/sqlite3
/usr/bin/sqlite3:
	/usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current version 5.4.0)
	/usr/lib/libsqlite3.dylib (compatibility version 9.0.0, current version 252.0.0)
	/usr/lib/libedit.3.dylib (compatibility version 2.0.0, current version 3.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1238.0.0)

Maybe it is the missing ncurses, which is in depends_lib, but not actually linked. Could also be a problem with --enable-readline vs. --enable-editline. The configure script states that the supported library is auto-detected, so one of them should be passed as --disable-*line.

[Schamschula (Marius Schamschula)]

The sqlite3 configure script insists on picking up /usr/include/readline/readline.h, which of course is for BSD editline, even when I specify --disable-editline.

Is there a standard way to force ./configure just to look in ${prefix}/include instead?

[raimue (Rainer Müller)]

Hm, it seems to be using -I/opt/local/include with the compile command, which would contain readline/readline.h and thus should take precedence?

That you cannot select readline is caused by the patch to the configure script that forces -ledit to be tried before -lreadline. And in fact, if I leave out the patch, --enable-readline --disable-editline produces a sqlite3 that accepts wide characters.

Turns out this is in fact a issue with our libedit. The readline() compatiblity interface does not seem to accept these characters. Note that this works with the libedit provided by macOS, so I would assume it is a bug in libedit itself.

Here is a small test program with a shell sequence to test it:

$ cat > readline-test.c
#include <editline/readline.h>

int main() {
    char *line = readline("test: ");
    printf("%s\n", line);
    return 0;
}
$ clang -o readline-test-apple readline-test.c -ledit
$ clang -I/opt/local/include -L/opt/local/lib -o readline-test-macports readline-test.c -ledit
$ ./readline-test-apple
test: Müller
Müller
$ ./readline-test-macports
test: Mller
Mller

[Schamschula (Marius Schamschula)]

In bdceba87/macports-ports:

sqlite3: use readline rather than editline to permit non-ASCII characters

Closes: #52866

[raimue (Rainer Müller)]

Technically the problem is solved for sqlite3 with the commit, but the bug in libedit still exists and might affect other applications.

[Schamschula (Marius Schamschula)]

Unfortunately, the recent update to libedit @20160903-3.1_0 does not fix this issue.

[MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)]

In 760795dd/macports-ports:

libedit: allow typing non-ASCII characters

fixes #52866

[MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)]

In 760795dd/macports-ports:

libedit: allow typing non-ASCII characters

fixes #52866

[Schamschula (Marius Schamschula)]

In b510cce/macports-ports:

sqlite3: revert to editline

See: #52866
See: 760795dd90b00862e47b4bf40a88da99e36688bd/macports-ports

[larryv (Lawrence Velázquez)]

In ae7c13cb/macports-ports:

sqlite3: Remove readline dependency

Completes b510ccebf141f1f4e0e9275a2cab0baa28f94628.
See #52866.