dblatex: incorrect UTF-8 handling and escaped characters in listing environment

[oksmith77]

Hi,

I am using dblatex 0.3.12 built with MacPorts.

The handling of Unicode characters does not seem correct to me.

The dblatex option

latex.encoding=utf8

should result in a UTF-8 .tex file with Unicode characters as is except within a listing environment. In that case, the Unicode characters need to be properly escaped. As far as I am concerned, they need no additional translation/mapping.

TEST #1 =======

The following produces a .tex file as expected

% dblatex --type=tex -o mytest.tex mytest.xml

The following does *NOT* produce a .tex file as expected.

% dblatex -P latex.encoding=utf8 --type=tex -o mytest.tex mytest.xml

In the above, the escape characters in the listing are incorrect.

TEST #2 =======

The following does *NOT* produce a .tex file as *FAILS* expected

% dblatex --type=tex -o mytest2.tex mytest2.xml

I recommend a more meaningful error message that includes the suggestion to use UTF-8 and xelatex.

The following should produce the expected .tex file but does *NOT*.

% dblatex -P latex.encoding=utf8 --type=tex -o mytest2.tex mytest2.xml

Again, the escape characters in the listing are incorrect.

TEST #3 =======

The following does *NOT* produce a .tex file as *FAILS* expected.

% dblatex --type=tex -o mytest3.tex mytest3.xml

The missing characters could be optionally added to the mapping table and then the test would work.

The following does work correctly.

% dblatex -P latex.encoding=utf8 --type=tex -o mytest3.tex mytest3.xml

Please let me know if you need additional information.

Thanks.

[oksmith77]

Test files

[neverpanic (Clemens Lang)]

I believe this needs a patch. This problem does not seem to be specific to macports, though, so this should be addressed upstream.

lib/dbtexmf/dblatex/rawverb.py

@@ -36 +36 @@
         n = tex_handler_counter[self._errors]
         for c in ntext:
             if ord(c) > 255:
-                c = str(self.pre) + c + str(self.post)
+                c = self.pre.decode(self.output_encoding) + c + self.post.decode(self.output_encoding)
                 n += 1
             text += c
         tex_handler_counter[self._errors] = n

[neverpanic (Clemens Lang)]

In 1e94a17651ea6d0c9b45e7e7b09859c469698c2f/macports-ports (master):

dblatex: Fix python3 encoding problem

Closes: #64720

[oksmith77]

I submitted a bug report to the project Sourceforge page. You may want to upload your patch there.

Thanks of taking care of this so quickly!