bugs in vtk-devel Portfile on Tiger

[jjstickel@…]

For the variant 'data', a "tar --strip-components ..." is used. The tar with Tiger predates the --strip-components option, and hence this fails. One option is to force the use of gnutar as provided by macports.

For the variant 'doc', a "ln -Fhs ..." is used. The ln with Tiger does not have the 'F' option, and hence this fails. Although the 'gln' in coreutils provides the 'F' option, it does not provide the 'h' option.

The "file copy ..." commands fail if a file already exists in the destination (as happens during debugging of a failed install). Is there an appropriate option that can be used to allow an overwrite?

Has "use_parallel_build yes" been tested with this Portfile? It would help speed up the compile time for this large package.

Thanks, Jonathan

[dweber@…]

Hi Jonathan,

I have no access to a Tiger platform to test any of these changes. Please advise on your solutions, if you find them to work on Tiger, so that I can then test those solutions for Leopard.

There may be simple alternatives to the tar and ln bugs, but I'm not going to resolve this unless I have access to a Tiger platform.

When debugging the install process, I often clean out the destroot entirely because of this error. My debug process looks something like:

sudo port destroot vtk-devel +huge # examine build and destroot, revise Portfile as required sudo rm -rf ${workpath}/destroot sudo vim ${workpath}/.macports*.state # to remove destroot state sudo port -o -k -d destroot vtk-devel +huge

The -o option avoids an entire rebuild just because the Portfile changed. Of course, if you change build settings or something, then you may want to rebuild. For the most part, the debug process has to tweak the destroot (vtk usually compiles OK).

The -k option is just there to 'keep' the build. I leave it in the command line just in case I change the destroot to an install command (my macports config does a clean after an install).

The parallel build option was explored. It does work sometimes, but it can fail for no reason sometimes. For the sake of reliability over speed, the option was omitted. Note that the macports config file must be modified from default settings for this take any effect anyway.

Take care, Darren

PS, Leaving this ticket open, pending further advice on a possible fix for Tiger.

[dweber@…]

Oops, let's fix that formatting!

When debugging the install process, I often clean out the destroot entirely because of the file copy error. My debug process looks something like:

$ sudo port destroot vtk-devel +huge
$ # examine build and destroot, revise Portfile as required
$ sudo rm -rf ${workpath}/destroot
$ sudo vim ${workpath}/.macports*.state # to remove destroot state
$ sudo port -o -k -d destroot vtk-devel +huge

The -o option avoids an entire rebuild just because the Portfile changed. Of course, if you change build settings or something, then you may want to rebuild. For the most part, the debug process has to tweak the destroot (vtk usually compiles OK).

The -k option is just there to 'keep' the build. I leave it in the command line just in case I change the destroot to an install command (my macports config does a clean after an install).

[jjstickel@…]

OK, I can try to provide a patch that works for Tiger. But I a question first. For the link command mentioned above, I wonder if the '-F' (uppercase) option is in error (allow the superuser to attempt to hard link directories). Shouldn't it be '-f' (lowercase)? An option related to hard links doesn't make much sense with the '-s' option.

[dweber@…]

I think the line in question is in the doc variant, where we have:

system "ln -Fhs ${vtkDocPath} ${destroot}/${vtkDocPathLink}"

So this is a system ln command (not a tcl ln command, as in some other variants).

On my leopard system, man ln contains the following:

 1 LN(1)             BSD General Commands Manual            LN(1)
...
20      -F    If the target file already exists and is a directory, then remove it so that the link may occur.
21        The -F option should be used with either -f or -i options.  If none is specified, -f is implied.
22        The -F option is a no-op unless -s option is specified.
...
24      -h    If the target_file or target_dir is a symbolic link, do not follow it.  This is most useful with
25        the -f option, to replace a symlink which may point to a directory.
...
37      -s    Create a symbolic link.

As I read this, it will create a symbolic link for a directory, replacing the symlink if it exists already.

Are we reading different man pages?

Suggestion. Take a look at the other variants where a tcl ln command is used. I don't recall exactly why this doc variant uses a system ln command, but I do recall that the symlinks created in ${prefix}/share/doc are not exactly what I want.

What I really want is this:

/opt/local/share/doc/vtk-5.4 -> /opt/local/share/vtk-5.4/doc

This is what I get:

/opt/local/share/doc/vtk-5.4/doc -> /opt/local/share/vtk-5.4/doc/

This is how it is defined, in a nutshell (after consolidating a few things from the Portfile):

set vtkDocPath     ${prefix}/share/${distname}/doc
set vtkDocPathLink ${prefix}/share/doc/${distname}
system "ln -Fhs ${vtkDocPath} ${destroot}/${vtkDocPathLink}"

I have no idea why the vtkDocPathLink actually becomes ${prefix}/share/doc/${distname}/doc

Best, Darren

[jjstickel@…]

I think I have resolved the problems I reported on Tiger. See attached patch to the portfile. I even fixed the doc linking according to what you suggest above.

The patch includes "use_parallel_build yes". I have not experienced any problems with parallel builds. If you are clearly seeing problems, then you should keep the line explicitly in the portfile, but flagged 'no'. This way others know it has been considered and tested.

[dweber@…]

I'm concerned that we need to directly address the version of tar used. Can you try this solution on Tiger?

I'm considering these lines for the post-destroot of the data variant:

        # tar-1.14 uses --strip-path, tar-1.14.90+ uses --strip-components
        set tarArg "--strip-components"
        set tarVersion [lrange [split [exec tar --version]] 3 3]
        set tarMaj [lrange [split $tarVersion .] 0 0]
        set tarMin [lrange [split $tarVersion .] 1 1]
        set tarRev [lrange [split $tarVersion .] 2 2]
        if (expr $tarMaj == 1 && $tarMin == 14 && $tarRev < 90) {
                  set tarArg "--strip-path"
        }
        system "tar $tarArg 1 -C ${destroot}/${vtkDataPath} -zxvf ${distpath}/${vtkDataFile}"

[dweber@…]

Please try out this initial revision to test whether it fixes the problems on Tiger:  http://trac.macports.org/browser/users/dweber/graphics/vtk-devel/Portfile

[jjstickel@…]

I'm sorry, but I can't do any more testing right now. I have no reason to use vtk-devel (vtk-5.4) over vtk5 (vtk-5.2).

To be honest, I don't understand why so much complexity is needed in the vtk-devel Portfile. The vtk-devel Portfile is 27 kb whereas the vtk5 Portfile is only 6 kb. Another thing I just noticed is that the installed libraries with vtk-devel are an order of magnitude larger in size compared to vtk5. For example, libvtkCommon.5.4.dylib is 27.1 MB compared to just 3.8 MB for libvtkCommon.5.2.1.dylib. Must be related to the compile options. Are debug symbols being included? Is this intentional?

[dweber@…]

Replying to jjstickel@…:

I'm sorry, but I can't do any more testing right now. I have no reason to use vtk-devel (vtk-5.4) over vtk5 (vtk-5.2).

OK, thanks for the feedback so far. It has taken me many days to develop vtk-devel, too many days to learn both MacPorts, cmake, and vtk quirks, but there it is. In case you find a minute to run it overnight on Tiger, just run the port destroot vtk-devel so it will not try to install. It would be great if there were no conflicts in the installation between vtk5 and vtk-devel, but there might be some binaries or symlinks that conflict.

To be honest, I don't understand why so much complexity is needed in the vtk-devel Portfile. The vtk-devel Portfile is 27 kb whereas the vtk5 Portfile is only 6 kb. Another thing I just noticed is that the installed libraries with vtk-devel are an order of magnitude larger in size compared to vtk5. For example, libvtkCommon.5.4.dylib is 27.1 MB compared to just 3.8 MB for libvtkCommon.5.2.1.dylib. Must be related to the compile options. Are debug symbols being included? Is this intentional?

a) There are more comments in vtk-devel (to explain my understanding of how things work; it's not gospel, so my understanding may change and hence vtk-devel may change, but it's in there for any 'openmaintainer' to read in my absence).

b) There are more variants in vtk-devel (wrappers, database, mpi, boost).

c) vtk5 handles the RPATH settings differently to vtk-devel. The vtk5 port uses -DCMAKE_BUILD_WITH_INSTALL_RPATH:BOOL=ON and it has to set build.env-append DYLD_LIBRARY_PATH=${worksrcpath}/bin. I've seen a post from the fink package maintainer that he does likewise. Perhaps that is the way to go, but after careful study of cmake settings and how they should work, I opted not to use any DYLD_LIBRARY_PATH or similar settings in vtk-devel. The point is that cmake is supposed to reset all the rpath strings during the installation (destroot), which it does for the majority of the build, but it fails to do so for the examples, testing, and wrapping. This is a bug that should be fixed up-stream in cmake or vtk. So, for now, there are a lot of extra bits in vtk-devel to call install_name_tool to change the rpath settings on various things (especially the examples and testing variants). I was able to abstract and consolidate a lot of that work for the InsightToolkit port into a tcl proc. Perhaps vtk5 handles this in a more elegant way in the post-configure phase of the port (I should look into how that works).

d) vtk-devel has an explicit cmake build type setting, RelWithDebInfo, so I hope there are debug symbols in there. I've adopted the general GNU convention for releases that contain debug symbols for libraries (gcc/g++ options -g and -O2 are provided by cmake RelWithDebInfo). Is that is a problem for a system with limited disk space, the Portfile setting to change is -DCMAKE_BUILD_TYPE:STRING=RelWithDebInfo to -DCMAKE_BUILD_TYPE:STRING=Release. I'm adding to vtk-devel a couple of variants for the build type, so it's easy to select a pure 'release' or 'debug' build. See the update for revision 4 in my user svn:  http://trac.macports.org/browser/users/dweber/graphics/vtk-devel/Portfile

Anyhow, sounds like your on the way with vtk5. BTW, you might like to look into whether vtk5 can be configured for mpi (perhaps borrow the vtk-devel variant and see if it works in vtk5). If you need cocoa, vtk-devel adds the compiler setting for garbage collection on 10.5+.

Good luck! Darren

[dweber@…]

No way of testing the current vtk-devel, at rev 5, on Tiger. Closing this ticket.