dblatex @0.3.11_1 can be built with Python 3

[ballapete (Peter "Pete" Dyballa)]

Because I would like to live in Big Sur (and outside) without an extra version of Python 2.7 I translated the dblatex Python 2 files to Python 3 files – mostly by means of 2to3. One exception is lib/dbtexmf/core/dbtex.py which uses quite often the file() function which I had to translate manually to open(). Since I am not a Python programmer I presume that the internet is right here.

By means of a cumulative patch file of diffs from 2to3 it worked to build with Python 3.8. Since I do not know how to 'make test' in Python I am leaving this to others. And I also have to leave actual test of this Python 3.8 variant dblatex to others since I have no use case for it. And also do not know how to create one or better two.

[ballapete (Peter "Pete" Dyballa)]

Patches to translate Python 2 code to Python 3 code

[ballapete (Peter "Pete" Dyballa)]

Build also succeeds on High Sierra.

[ballapete (Peter "Pete" Dyballa)]

One more patch seems to be needed for install step:

unning install
Error: invalid syntax (which.py, line 283)
+checking XSLT xsltconf... yes
Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_textproc_dblatex/dblatex/work/dblatex-0.3.11" && /opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/pyt\
hon3.8 setup.py install --root=/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_textproc_dblatex/dblatex/work/destroot --catalogs=/opt/local/var/macports/build/_opt_local_var_macports_so\
urces_rsync.macports.org_macports_release_tarballs_ports_textproc_dblatex/dblatex/work/destroot/opt/local/etc/xml/catalog
Exit code: 1

[ballapete (Peter "Pete" Dyballa)]

The next version of the patch file solves the error. Nevertheless a new error is introduced that I do not seem to understand (yet):

SDKROOT='/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk'
Executing:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_textproc_dblatex/dblatex/work/dblatex-0.3.11" && /opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8 setup.py install --root=/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_textproc_dblatex/dblatex/work/destroot --catalogs=/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_textproc_dblatex/dblatex/work/destroot/opt/local/etc/xml/catalog
DEBUG: system:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_textproc_dblatex/dblatex/work/dblatex-0.3.11" && /opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8 setup.py install --root=/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_textproc_dblatex/dblatex/work/destroot --catalogs=/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_textproc_dblatex/dblatex/work/destroot/opt/local/etc/xml/catalog
running install
Error: 'utf-8' codec can't decode byte 0xf4 in position 1187: invalid continuation byte
+checking XSLT xsltconf... yes
+checking XSLT saxon... no (missing saxon)
warning: XSLT not found: saxon
+checking epstopdf... yes
+checking convert... yes
+checking fig2dev... no
warning: not found: fig2dev
+checking latex... yes
+checking makeindex... yes
+checking pdflatex... yes
+checking kpsewhich... yes
Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_textproc_dblatex/dblatex/work/dblatex-0.3.11" && /opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8 setup.py install --root=/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_textproc_dblatex/dblatex/work/destroot --catalogs=/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_textproc_dblatex/dblatex/work/destroot/opt/local/etc/xml/catalog
Exit code: 1

The file /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_textproc_dblatex/dblatex/work/destroot/opt/local/etc/xml/catalog does not yet exist and neither the xml subdirectory in /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_textproc_dblatex/dblatex/work/destroot/opt/local/etc.

[ballapete (Peter "Pete" Dyballa)]

New version of patches to translate Python 2 code to Python 3 code

[ballapete (Peter "Pete" Dyballa)]

The problem is with some LaTeX STY files that come with dblatex: Python 3 assumes they come in UTF-8 encoding, but the contain in comments names with accented characters that are coded in ISO8859-15 or such. Either these files need to be re-encoded or these accented chars need to be translated into LaTeX 7-bit primitives, for example é -> \´e.

[ballapete (Peter "Pete" Dyballa)]

Python2-3_patches_too.diff patches other Python files, those that will be installed, STY-ISO8859-UTF8.diff patches the LaTeX STYle files with accented characters.

I am stuck here:

  221	def kpsewhich(tex_file):
  222	    if os.name == "nt":
  223	        close_fds = False
  224	    else:
  225	        close_fds = True
  226	        p = Popen("kpsewhich %s" % tex_file, shell=True,
  227	              stdin=PIPE, stdout=PIPE, close_fds=close_fds)
  228	    out = "".join(p.stdout.readlines()).strip()
  229	    return out

kpsewhich runs once, returning "amsmath.sty" into the else branch. Tracing the execution of setup.py seems to indicate the statement on line #228 is faulty:

setup.py(228):     out = "".join(p.stdout.read()).strip()
setup.py(384):             except Exception as e:
setup.py(384):                 print("Error: %s" % e, file=sys.stderr)
Error: sequence item 0: expected str instance, int found

[ballapete (Peter "Pete" Dyballa)]

Patches Python files to be installed from version 2 to version 3

[ballapete (Peter "Pete" Dyballa)]

Patches LaTeX STY files with accented characters from ISO8859-1 to UTF-8

[ballapete (Peter "Pete" Dyballa)]

​https://sourceforge.net/p/dblatex/dblatex-py3/ci/default/tree/ a version 3.12 that claims to be Python 3 compliant…

[ballapete (Peter "Pete" Dyballa)]

  dblatex @0.3.12pre1_0 (active) platform='darwin 20' archs='noarch' date='2020-11-22T15:10:48+0100'

built, plus its variant +mactex. The proof that this version of dblatex works maybe is that this installed:

  gtk-doc @1.32_1+pdf+python38 (active) platform='darwin 20' archs='noarch' date='2020-11-22T15:27:04+0100'

At the end of the build step a small Python3 utility reports:

running build_scripts
creating build/scripts-3.8
running build_doc
/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/setuptools/dist.py:452: UserWarning: Normalizing '0.3.12pre1' to '0.3.12rc1'
  warnings.warn(tmpl.format(**locals()))
DEBUG: Privilege de-escalation not attempted as not running as root.
DEBUG: destroot phase started at Sun Nov 22 15:10:09 CET 2020
--->  Staging dblatex into destroot

I am attaching my variant of Portfile. It may not be perfect, the line

master_sites        sourceforge:project/dblatex/dblatex/${name}3-${version}

or

master_sites        sourceforge:project/dblatex/dblatex/${distname}

might be incorrect. I also did not fetch the archive, I had downloaded it manually and put into /opt/local/var/macports/distfiles/dblatex.

[ballapete (Peter "Pete" Dyballa)]

Possible Portfile to build dblatex @0.3.12pre1

[ballapete (Peter "Pete" Dyballa)]

On High Sierra these are installed:

  dblatex @0.3.12pre1_0 (active) platform='darwin 17' archs='noarch' date='2020-11-22T20:28:43+0100'
  gtk-doc @1.32_1+pdf+python38 (active) platform='darwin 17' archs='noarch' date='2020-11-22T20:32:48+0100'

[ballapete (Peter "Pete" Dyballa)]

On PPC Leopard these could be installed:

  dblatex @0.3.12pre1_0 (active) platform='darwin 9' archs='noarch' date='2020-11-22T21:11:09+0100'
  gtk-doc @1.32_1+pdf+python38 (active) platform='darwin 9' archs='noarch' date='2020-11-22T21:20:43+0100'

[ballapete (Peter "Pete" Dyballa)]

On Big Sur a bug exists when dblatex is built. A make check in gtk-docs source directory shows this failure:

Traceback (most recent call last):
  File "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gnome_gtk-doc/gtk-doc/work/gtk-doc-1.32/gtkdoc-mkpdf", line 48, in <module>
    sys.exit(mkpdf.run(options))
  File "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gnome_gtk-doc/gtk-doc/work/gtk-doc-1.32/gtkdoc/mkpdf.py", line 75, in run
    pc = subprocess.Popen([config.dblatex, '--help'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/subprocess.py", line 854, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/subprocess.py", line 1702, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/opt/local/bin/dblatex'

The reason is not that /opt/local/bin/dblatex does not exist but that the script starts with

#!/opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.83.8

The correct interpreter is of course /opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8!

[ballapete (Peter "Pete" Dyballa)]

Correcting the path to Python 3.8 manually lets make check for gtk-doc run successfully on all mentioned platforms. I sent an eMail to the maintainer of dblatex3 informing him about the wrong path.

[ballapete (Peter "Pete" Dyballa)]

It's actually this line from the original version of dblatex's Portfile that causes the faulty path to Python 3:

    reinplace "s|#!/usr/bin/env python|#!${python.bin}|" ${destroot}${python.prefix}/bin/dblatex

The built file scripts/dblatex starts with

#! /usr/bin/env python3

so Portfile should better contain

    reinplace "s|#!/usr/bin/env python3|#!${python.bin}|" ${destroot}${python.prefix}/bin/dblatex

which leads to a different fault in the installed dblatex executable:

#!/opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8.8

I don't know where the cause for this is. I think you could release a dblatex-devel Portfile for those who want to get rid of Python 2.7.

[neverpanic (Clemens Lang)]

In d41567761ddcc6fc94d2e022a50be66cbd638855/macports-ports (master):

dblatex: Update to 0.3.12, use Python 3.10

Closes: #61530