Opened 5 years ago

Closed 5 years ago

Last modified 5 years ago

#58323 closed defect (fixed)

gettext @0.19.8.1: Running msgfmt to build Java2 resources crashes with specific string lengths of msgctxt and msgid

Reported by: p3k (Tobi Schäfer) Owned by: ryandesign (Ryan Carsten Schmidt)
Priority: Normal Milestone:
Component: ports Version:
Keywords: haspatch Cc: jmroot (Joshua Root)
Port: gettext

Description

This is kind of a weird bug which so far I only could observe using the msgfmt binary (version 0.19.8.1) on macos.

The following de.po file causes msgfmt to crash with ”Abort trap: 6“ when running msgfmt --java2 --source -d/tmp --resource=foo.bar.Messages --locale=de de.po

# SOME DESCRIPTIVE TITLE.
# Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER
# This file is distributed under the same license as the PACKAGE package.
# FIRST AUTHOR <EMAIL@ADDRESS>, YEAR.
#
msgid ""
msgstr ""
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Plural-Forms: nplurals=2; plural=(n != 1);\n"

msgctxt "123456789"
msgid "abcdef"
msgstr "whatever"

The expected output should be this Java file (and in fact, this is what I get with a Linux binary of msgfmt, same version):

/* Automatically generated by GNU msgfmt.  Do not modify!  */
package foo.bar;
public class Messages_de extends java.util.ResourceBundle {
  private static final java.lang.String[] table;
  static {
    java.lang.String[] t = new java.lang.String[4];
    t[0] = "";
    t[1] = "MIME-Version: 1.0\nContent-Type: text/plain; charset=UTF-8\nContent-Transfer-Encoding: 8bit\nPlural-Forms: nplurals=2; plural=(n != 1);\n";
    t[2] = "1a\u000423456789012345";
    t[3] = "whatever";
    table = t;
  }
  public java.lang.Object handleGetObject (java.lang.String msgid) throws java.util.MissingResourceException {
    int hash_val = msgid.hashCode() & 0x7fffffff;
    int idx = (hash_val % 2) << 1;
    java.lang.Object found = table[idx];
    if (found != null && msgid.equals(found))
      return table[idx + 1];
    return null;
  }
  public java.util.Enumeration getKeys () {
    return
      new java.util.Enumeration() {
        private int idx = 0;
        { while (idx < 4 && table[idx] == null) idx += 2; }
        public boolean hasMoreElements () {
          return (idx < 4);
        }
        public java.lang.Object nextElement () {
          java.lang.Object key = table[idx];
          do idx += 2; while (idx < 4 && table[idx] == null);
          return key;
        }
      };
  }
  public java.util.ResourceBundle getParent () {
    return parent;
  }
}

The problem seems to be related to the combined string lenghts of the msgctxt and msgid values.

For example, the following values crash msgfmt, too (msgstr can be arbitrary):

msgctxt "four"
msgid "sixplusfive"

msgctxt "context"
msgid "can hurt"

It looks like if the sum of both values’ lengths equals 15, msgfmt is going to crash – slightly modified values prevent the crash and create the Java message file accordingly:

msgctxt "123456789x"
msgid "abcdef"

msgctxt "123456789"
msgid "abcdef_"

msgctxt "four"
msgid "six-plus-five"

msgctxt "context"
msgid "is hard"

What could cause this behaviour? And how could it sneak into the macos binary only – not in the Linux one (assuming the same source code is used for both platforms)?


The issue could be verified on two different macos installations, one of them with the following specs:

# sw_vers
ProductName:    Mac OS X
ProductVersion: 10.14.3
BuildVersion:   18D109

# msgfmt -V
msgfmt (GNU gettext-tools) 0.19.8.1
Copyright (C) 1995-1998, 2000-2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Written by Ulrich Drepper.

# port version
Version: 2.5.4

Please let me know if additional information is required.

Attachments (2)

msgfmt_2019-04-10-120308_macke.crash (10.5 KB) - added by p3k (Tobi Schäfer) 5 years ago.
write-java.c.patch (949 bytes) - added by jmroot (Joshua Root) 5 years ago.

Download all attachments as: .zip

Change History (14)

comment:1 Changed 5 years ago by p3k (Tobi Schäfer)

Version 0, edited 5 years ago by p3k (Tobi Schäfer) (next)

comment:2 Changed 5 years ago by jmroot (Joshua Root)

Keywords: msgfmt removed
Owner: set to ryandesign
Status: newassigned
Summary: Running msgfmt to build Java2 resources crashes with specific string lengths of msgctxt and msgidgettext @0.19.8.1: Running msgfmt to build Java2 resources crashes with specific string lengths of msgctxt and msgid

Could you attach the crash log? It should be in either /Library/Logs/DiagnosticReports or ~/Library/Logs/DiagnosticReports.

Changed 5 years ago by p3k (Tobi Schäfer)

comment:3 in reply to:  2 Changed 5 years ago by p3k (Tobi Schäfer)

Replying to jmroot:

Could you attach the crash log? It should be in either /Library/Logs/DiagnosticReports or ~/Library/Logs/DiagnosticReports.

sure, did so. thanks for the quick reply.

comment:4 Changed 5 years ago by jmroot (Joshua Root)

There's an off-by-one error in msgid_hashcode which results in a string not being null-terminated correctly, which leads to a stack overflow. I guess malloc is allocating in 16-byte chunks and the memory is usually starting zeroed, so unless your string ends exactly on the boundary you don't see the issue.

comment:5 Changed 5 years ago by jmroot (Joshua Root)

OK to commit this, Ryan?

comment:6 Changed 5 years ago by jmroot (Joshua Root)

Cc: jmroot added
Keywords: haspatch added

comment:7 Changed 5 years ago by p3k (Tobi Schäfer)

Thanks for investigating and finding the culprit so fast.

Do you have any idea, though, why the error does not occur on Linux?

comment:8 Changed 5 years ago by jmroot (Joshua Root)

Could be many things; malloc allocating in different sized chunks, something else being in the adjacent memory, etc. The problem is masked even by building with -O0 -g instead of -Os.

Last edited 5 years ago by jmroot (Joshua Root) (previous) (diff)

comment:9 in reply to:  5 Changed 5 years ago by ryandesign (Ryan Carsten Schmidt)

Replying to jmroot:

OK to commit this, Ryan?

Sure. I filed a bug report with the developers: https://savannah.gnu.org/bugs/?56112

comment:10 Changed 5 years ago by jmroot (Joshua Root)

Resolution: fixed
Status: assignedclosed

In a4c79a0b578a7c230518cb8408e5653b5bef430d/macports-ports (master):

gettext: fix crash due to buffer overflow

Fixes: #58323

comment:11 Changed 5 years ago by jmroot (Joshua Root)

Looks like the same mistake is made in write_java_msgid as well, though that one happens not to cause a crash.

Changed 5 years ago by jmroot (Joshua Root)

Attachment: write-java.c.patch added

comment:12 Changed 5 years ago by ryandesign (Ryan Carsten Schmidt)

Upstream has committed your fixes, plus another similar fix in write-csharp.c. Looks like they're gearing up for a 0.20 release.

Note: See TracTickets for help on using tickets.