imake @1.0.5_1 Proposed fix for imake / clang inoperability

[qbarnes (Quentin Barnes)]

There's been a number of bugs submitted against various ports packages which break build because of xmkmf/imake not working with clang. (#41951, #40862, #41293, #41434, et. al.). There's been various hacks and workarounds to the packages, but nothing I've found that addresses the imake problem directly.

I have a patch set attached to fix imake for this problem. This can not only fix it for various dependent ports, but will fix it for people wanting to use xmkmf and imake on non-macports packages (like me).

I'm not an active Macports maintainer, so please scrutinize my proposed changes carefully.

If this patchset is accepted as a changeset, I would also recommend deleting the two IMAKECPP environment appends from src/port1.0/portconfigure.tcl. Once imake is fixed, this workaround in portconfigure.tcl is unnecessary and actually causes problems and additional work for port maintainers. When building software that uses xmkmf, there is no need to blacklist clang, but these IMAKECPP environment changes require maintainers to always blacklist clang or manually set IMAKECPP themselves to override.

In my suggested changes to imake, I do blacklist clang, but there is no reason I had to. It just saves some amount of hacking on imakemdep.h. The main change to the Portfile is to change imakemdep.h to set the DEFAULT_CC macro to ${configure.cc}, so I set that via blacklisting clang. I had considered making the imake package dependent on gcc42 or gcc48 and using "gcc-mp-4.2" (or -4.8), instead of ${configure.cc}, but thought the blacklist/${configure.cc} approach was better.

In imakemdep.h there is no need to define both DEFAULT_CPP and DEFAULT_CC as had been previously done. DEFAULT_CC takes precedence.

The patch file has three changes. One is for the DEFAULT_CC string that gets replaced with ${configure.cc}. The second one was in the last patch file. There is no need for it now since we're not building with clang, but I kept it anyways. I also made the equivalent change later down in the file has well. If the imake maintainer desires, just leave off the last two deltas. I left them in though because with the right changes to imakemdep.h it should be possible to build imake with clang, just not use it for imake's preprocessor.

I tried for some time to see if I could get clang to work with imake, but I concluded that it is impossible, at least for now with the way clang is currently implemented. If someone wants to prove me wrong, I'd be happy to hear it. I'll summarize my findings and conclusions here if anyone wants to pick up the torch and cares to try.

With clang, its preprocessor has two modes of operation, "traditional" and "standards compliant". Though in traditional mode, it does not follow many other K&R C preprocessors do when filtering out comments. With many K&R C preprocessors a comment (e.g. /**/) will be completely removed. However, clang inserts a whitespace, which is allowed (thus creating the 'X11 .rules' file not found error). In standard mode, we have ## operators, but they won't work for what the compiler considers invalid preprocessing tokens. Even if that were worked around somehow, the clang preprocessor also mangles tab characters replacing them with a single space character -- a fatal flaw for a makefile.

From my reading of ISO C99, Annex A.1.1, I don't see how a character sequence such as '<TopLevelProject' is declared an invalid preprocessing token given the last entry of the definition of a "preprocessing-token". If someone knows why, I'm curious. If you would, please send me an email of what I'm not reading correctly.

The general problem with imake though is that it should never have been designed to use a C preprocessor on a non-C files. For upstream imake developers, the only real fix is to abandon using a C preprocessor and provide a compatible preprocessor as part of the package.

I also tried using "mcpp" in -@kr mode as the cpp for imake. It failed with a memory fault.

[qbarnes (Quentin Barnes)]

Patchfile for imake's Portfile

[qbarnes (Quentin Barnes)]

Replacement version for imake's patch-imakemdep.h.diff patch file

[qbarnes (Quentin Barnes)]

Cc Me!

[mf2k (Frank Schima)]

In the future, please fill in the Port field and Cc the port maintainers (port info --maintainers imake). As reporter, you do not need to Cc yourself.

[qbarnes (Quentin Barnes)]

Replying to macsforever2000@…:

In the future, please fill in the Port field and Cc the port maintainers (port info --maintainers imake). As reporter, you do not need to Cc yourself.

Sure thing! I admit I was in a hurry to finish the initial filing thinking I would just finish up what I missed later. When I went back later to see what I might have missed (port and keywords) and fill in what got overlooked, I tried to fill them in but didn't see a way for me to do it. Is there a way I can edit the field after filing?

As for the Cc:, yeah, that was an experiment. I wanted to see what would happen if I clicked on the "CcMe!" button while I was the reporter. I was curious if it would just blindly add me or if the button's code would have a check and just say I was already copied on the ticket. :-) I didn't delete myself because I was curious if I would get two copies or still just one.

[diekhans (Mark Diekhans)]

Cc Me!

[cooljeanius (Eric Gallager)]

Cc Me!

[WolfgangFahl (Wolfgang Fahl)]

Cc Me!

[WolfgangFahl (Wolfgang Fahl)]

Sounds wonderful. Please bare my ignorance - how do i get this patch working? I could patch the Portfile successfully. But how do I patch patch-imakemdep.h.diff given the patch that is a diff of imakemdep.h?

Here is a script I started - it even attempts a local port repository - but i wouldn't know where to copy the files and what to do ...

#!/bin/bash
# WF 2014-02-27

# configure where you want to have your local port repository
# see https://guide.macports.org/chunked/development.local-repositories.html
ports=$HOME/ports

# check where the port base directory is
portdir=`which port`
# e.g. /opt/local/bin/port
portdir=`dirname $portdir`
# e.g. /opt/local/bin
portdir=`dirname $portdir`
# e.g. /opt/local

#
#  autoinstall a cmd 
#   param1: the command
#   param2: the port
# 
#     e.g. autoinstall rcs rcs
#
autoinstall() {
  local l_cmd=$1
  local l_port=$2
  which $l_cmd
  if [ $? -ne 0 ]
  then
     sudo port install $l_port
  fi
}


#
#  patch and diff
# 
patchAndDiff() {
  local l_org=$1
  local l_patch=$2
  autoinstall rcs rcs
  if [ ! -d RCS ]
  then
    mkdir RCS
  fi
  ci -l $l_org
  patch  $l_org $l_patch
  rcsdiff $l_org
}

# show result
echo "patching imake using port directory $portdir"
if [ ! -d $ports ]
then
  mkdir -p $ports 
fi

# check that the local port repository has been configured
sources=$portdir/etc/macports/sources.conf
portrep="file://${ports}"
grep "$portrep" $sources
if [ $? -ne 0 ]
then
  echo "missing $portrep in $sources" 1>&2
	echo "please edit $sources and rerun $0"
	exit 1
fi

cd $ports
ticket=https://trac.macports.org/raw-attachment/ticket/42547
for patch in patch-imakemdep.h.diff imake-Portfile.patch
do
  if [ ! -f $patch ]
  then
    wget --no-check-certificate $ticket/$patch
  fi
done

# patch imake Portfile locally
cp -p $portdir/var/macports/sources/rsync.macports.org/release/tarballs/ports/x11/imake/Portfile .
patchAndDiff Portfile imake-Portfile.patch

mv patch-imakemdep.h.diff p1
cp -p $portdir/var/macports/sources/rsync.macports.org/release/tarballs/ports/x11/imake/files/patch-imakemdep.h.diff .
#patchAndDiff 

sudo portindex

[diekhans (Mark Diekhans)]

Cc Me!

[diekhans (Mark Diekhans)]

Cc Me!

[qbarnes (Quentin Barnes)]

Replying to wf@…:

Sounds wonderful. Please bare my ignorance - how do i get this patch working? I could patch the Portfile successfully. But how do I patch patch-imakemdep.h.diff given the patch that is a diff of imakemdep.h?

The directions for submitting patches said not to submit a patch of a patch file, but a whole file (​http://guide.macports.org/#project.contributing.updates). You replace (overwrite) the existing patch file with the new one.

[MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)]

I believe r126419 fixed this.
If not, please feel free to reopen.

[team@…]

build log

[team@…]

I just tried to build tightvnc and got :

././/config/cf/Imake.tmpl:1320:10: fatal error: ' X11 .rules' file not found

MacPorts 2.3.4

[MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)]

Replying to team@…:

I just tried to build tightvnc and got :

././/config/cf/Imake.tmpl:1320:10: fatal error: ' X11 .rules' file not found

MacPorts 2.3.4

This does not seem to be a problem with the port imake.
tightvnc uses its own version of imake that must be fixed.
Please see #40862.