#58323 closed defect (fixed)
gettext @0.19.8.1: Running msgfmt to build Java2 resources crashes with specific string lengths of msgctxt and msgid
Reported by: | p3k (Tobi Schäfer) | Owned by: | ryandesign (Ryan Schmidt) |
---|---|---|---|
Priority: | Normal | Milestone: | |
Component: | ports | Version: | |
Keywords: | haspatch | Cc: | jmroot (Joshua Root) |
Port: | gettext |
Description
This is kind of a weird bug which so far I only could observe using the msgfmt
binary (version 0.19.8.1) on macos.
The following de.po
file causes msgfmt to crash with ”Abort trap: 6“ when running msgfmt --java2 --source -d/tmp --resource=foo.bar.Messages --locale=de de.po
# SOME DESCRIPTIVE TITLE. # Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER # This file is distributed under the same license as the PACKAGE package. # FIRST AUTHOR <EMAIL@ADDRESS>, YEAR. # msgid "" msgstr "" "MIME-Version: 1.0\n" "Content-Type: text/plain; charset=UTF-8\n" "Content-Transfer-Encoding: 8bit\n" "Plural-Forms: nplurals=2; plural=(n != 1);\n" msgctxt "123456789" msgid "abcdef" msgstr "whatever"
The expected output should be this Java file (and in fact, this is what I get with a Linux binary of msgfmt, same version):
/* Automatically generated by GNU msgfmt. Do not modify! */ package foo.bar; public class Messages_de extends java.util.ResourceBundle { private static final java.lang.String[] table; static { java.lang.String[] t = new java.lang.String[4]; t[0] = ""; t[1] = "MIME-Version: 1.0\nContent-Type: text/plain; charset=UTF-8\nContent-Transfer-Encoding: 8bit\nPlural-Forms: nplurals=2; plural=(n != 1);\n"; t[2] = "1a\u000423456789012345"; t[3] = "whatever"; table = t; } public java.lang.Object handleGetObject (java.lang.String msgid) throws java.util.MissingResourceException { int hash_val = msgid.hashCode() & 0x7fffffff; int idx = (hash_val % 2) << 1; java.lang.Object found = table[idx]; if (found != null && msgid.equals(found)) return table[idx + 1]; return null; } public java.util.Enumeration getKeys () { return new java.util.Enumeration() { private int idx = 0; { while (idx < 4 && table[idx] == null) idx += 2; } public boolean hasMoreElements () { return (idx < 4); } public java.lang.Object nextElement () { java.lang.Object key = table[idx]; do idx += 2; while (idx < 4 && table[idx] == null); return key; } }; } public java.util.ResourceBundle getParent () { return parent; } }
The problem seems to be related to the combined string lenghts of the msgctxt
and msgid
values.
For example, the following values crash msgfmt, too (msgstr
can be arbitrary):
msgctxt "four" msgid "sixplusfive" msgctxt "context" msgid "can hurt"
It looks like if the sum of both values’ lengths equals 15, msgfmt is going to crash – slightly modified values prevent the crash and create the Java message file accordingly:
msgctxt "123456789x" msgid "abcdef" msgctxt "123456789" msgid "abcdef_" msgctxt "four" msgid "six-plus-five" msgctxt "context" msgid "is hard"
What could cause this behaviour? And how could it sneak into the macos binary only – not in the Linux one (assuming the same source code is used for both platforms)?
The issue could be verified on two different macos installations, one of them with the following specs:
# sw_vers ProductName: Mac OS X ProductVersion: 10.14.3 BuildVersion: 18D109 # msgfmt -V msgfmt (GNU gettext-tools) 0.19.8.1 Copyright (C) 1995-1998, 2000-2016 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Written by Ulrich Drepper. # port version Version: 2.5.4
Please let me know if additional information is required.
Attachments (2)
Change History (14)
comment:1 Changed 23 months ago by p3k (Tobi Schäfer)
comment:2 follow-up: 3 Changed 23 months ago by jmroot (Joshua Root)
Keywords: | msgfmt removed |
---|---|
Owner: | set to ryandesign |
Status: | new → assigned |
Summary: | Running msgfmt to build Java2 resources crashes with specific string lengths of msgctxt and msgid → gettext @0.19.8.1: Running msgfmt to build Java2 resources crashes with specific string lengths of msgctxt and msgid |
Could you attach the crash log? It should be in either /Library/Logs/DiagnosticReports
or ~/Library/Logs/DiagnosticReports
.
Changed 23 months ago by p3k (Tobi Schäfer)
Attachment: | msgfmt_2019-04-10-120308_macke.crash added |
---|
comment:3 Changed 23 months ago by p3k (Tobi Schäfer)
Replying to jmroot:
Could you attach the crash log? It should be in either
/Library/Logs/DiagnosticReports
or~/Library/Logs/DiagnosticReports
.
sure, did so. thanks for the quick reply.
comment:4 Changed 23 months ago by jmroot (Joshua Root)
There's an off-by-one error in msgid_hashcode which results in a string not being null-terminated correctly, which leads to a stack overflow. I guess malloc is allocating in 16-byte chunks and the memory is usually starting zeroed, so unless your string ends exactly on the boundary you don't see the issue.
comment:6 Changed 23 months ago by jmroot (Joshua Root)
Cc: | jmroot added |
---|---|
Keywords: | haspatch added |
comment:7 Changed 23 months ago by p3k (Tobi Schäfer)
Thanks for investigating and finding the culprit so fast.
Do you have any idea, though, why the error does not occur on Linux?
comment:8 Changed 23 months ago by jmroot (Joshua Root)
Could be many things; malloc allocating in different sized chunks, something else being in the adjacent memory, etc. The problem is masked even by building with -O0 -g instead of -Os.
comment:9 Changed 23 months ago by ryandesign (Ryan Schmidt)
Replying to jmroot:
OK to commit this, Ryan?
Sure. I filed a bug report with the developers: https://savannah.gnu.org/bugs/?56112
comment:10 Changed 23 months ago by jmroot (Joshua Root)
Resolution: | → fixed |
---|---|
Status: | assigned → closed |
comment:11 Changed 23 months ago by jmroot (Joshua Root)
Looks like the same mistake is made in write_java_msgid as well, though that one happens not to cause a crash.
Changed 23 months ago by jmroot (Joshua Root)
Attachment: | write-java.c.patch added |
---|
comment:12 Changed 23 months ago by ryandesign (Ryan Schmidt)
Upstream has committed your fixes, plus another similar fix in write-csharp.c. Looks like they're gearing up for a 0.20 release.
Also see issue for Brew binary: https://github.com/Homebrew/homebrew-core/issues/38808