file @5.15_0 gives errors instead of a result

[ralph@…]

Having just installed "file" on 10.9, I get errors on trying to use it with many files, e.g.

file /opt/local/var/db/texmf/fonts/map/dvipdfm/updmap/dvipdfm_dl14.map 
/opt/local/var/db/texmf/fonts/map/dvipdfm/updmap/dvipdfm_dl14.map: ERROR: line 163: regex error 17, (illegal byte sequence)

file ought to tell me what type of file this is. Instead, it gives some error message which is nothing to do with the file being examined. This happens with a wide range of files, giving the same error message in each case,

However, the file type of some files is correctly determined, e.g.

file /Users/someone/Papers/Comment.pdf 
/Users/someone/Papers/Comment.pdf: PDF document, version 1.3

I don't think this is 10.9 specific, and believe I saw it before on 10.8

[ralph@…]

Just to verify this is the macports version of file here

$ which file
/opt/local/bin/file

$ file -v
file-5.15
magic file from /opt/local/share/misc/magic

[ryandesign (Ryan Carsten Schmidt)]

I see this too. You should report this problem to the developers of file so that they can fix it.

[ralph@…]

But it does not happen with the built in file, so I suspect it may have something to do with the way macports builds it (although these are different versions, I admit):

$ /opt/local/bin/file /Development/chmod-g-as-u /Development/chmod-g-as-u: ERROR: line 163: regex error 17, (illegal byte sequence) $ /usr/bin/file /Development/chmod-g-as-u /Development/chmod-g-as-u: a /usr/bin/perl script text executable

[ralph@…]

OK, it seems this has already been reported to the developers of file:

​http://bugs.gw.com/view.php?id=292

[ryandesign (Ryan Carsten Schmidt)]

MacPorts /opt/local/bin/file and OS X /usr/bin/file are completely different software packages that happen to perform similar functions. So the functioning or not functioning of one has no bearing on the other.

[cooljeanius (Eric Gallager)]

Cc Me!

[ryandesign (Ryan Carsten Schmidt)]

Has duplicate #41602.

[neverpanic (Clemens Lang)]

Not a Mavericks bug. This happened on older OS releases, too.

[suominen (Kimmo Suominen)]

fix regex errors on OS X; for file 5.17

[suominen (Kimmo Suominen)]

Cc Me!

[ryandesign (Ryan Carsten Schmidt)]

@kimmo, thank you for the patch. In future, if you add an attachment to a ticket, please also write a comment in the ticket so that an email notification is sent. Email notifications are sent only for added comments, not for added attachments.

The patch does not fix the issue. Please re-open the upstream ticket. I do not appear to have permission to do so myself.

[suominen (Kimmo Suominen)]

@ryandesign, sorry, didn't know about the need for a comment. Thanks for pointing that out.

Can you please provide more information about how the issue is not fixed and how to reproduce the problem?

For me the patch fixes the issue. I even created a local port today to verify it within MacPorts: ​https://github.com/suominen/macports-gw/tree/master/sysutils/file

I didn't include the existing patches in the port as they seemed unnecessary or undesirable. However, I don't think including them would make a difference when it comes to the issue in this ticket.

It would be helpful if each patch file explained the reason for the patch. This way it would be easier for others to assess them and consider for inclusion upstream. It is a standard practice now in pkgsrc: ​http://cvsweb.netbsd.org/bsdweb.cgi/pkgsrc/sysutils/file/patches/patch-aa?rev=1.6&content-type=text/plain

Here are my thoughts on the patches, without having an easy way of knowing why they really were added originally.

• patch-getline.diff: this seems unnecessary; the binary is linked correctly and runs without the change

• patch-magic-Magdir-msdos.diff: having file produce different results depending on which packaging system was used seems very undesirable to me. However, a change like this would definitely be appropriate to bring up on the file mailing list or bug tracker.

• patch-magic-Makefile.am: an uncompiled collection of the magic entries is not needed, but I could see how it might be interesting as a reference. Was that the reason for the patch? I think proposing this upstream with the reasoning behind it would also be good.

If I can help with proposing any of these (or future issues) upstream, please let me know.

[mojca (Mojca Miklavec)]

Cc Me!

[jmroot (Joshua Root)]

Replying to kimmo@…:

Here are my thoughts on the patches, without having an easy way of knowing why they really were added originally.

You can view the commit logs for the patchfiles in Trac.

• patch-getline.diff: this seems unnecessary; the binary is linked correctly and runs without the change

r102948 "Fix build on Snow Leopard"

• patch-magic-Magdir-msdos.diff: having file produce different results depending on which packaging system was used seems very undesirable to me. However, a change like this would definitely be appropriate to bring up on the file mailing list or bug tracker.

r41083 “patch the magic information to fix a bogus entry"

• patch-magic-Makefile.am: an uncompiled collection of the magic entries is not needed, but I could see how it might be interesting as a reference. Was that the reason for the patch? I think proposing this upstream with the reasoning behind it would also be good.

r41083 “add a variant that installs the plain-text magic(5) file”

No real insight from that last one, but apparently at least one person uses it since they filed #23710.

[ryandesign (Ryan Carsten Schmidt)]

Replying to kimmo@…:

Can you please provide more information about how the issue is not fixed and how to reproduce the problem?

I simply try to use the file command, and it fails in the way described in the summary of this ticket:

$ sw_vers
ProductName:	Mac OS X
ProductVersion:	10.9.2
BuildVersion:	13C64
$ file -version
file-5.17
magic file from /opt/local/share/misc/magic
$ echo hello > test
$ file test
test: ERROR: line 163: regex error 17, (illegal byte sequence)
$ LANG=C file test
test: ASCII text
$

[suominen (Kimmo Suominen)]

Replying to jmr@…:

Replying to kimmo@…:

Here are my thoughts on the patches, without having an easy way of knowing why they really were added originally.

You can view the commit logs for the patchfiles in Trac.

• patch-getline.diff: this seems unnecessary; the binary is linked correctly and runs without the change

r102948 "Fix build on Snow Leopard"

Well, I don't expect the patch to hurt anything, but without additional info it won't get merged upstream. It might be useful to note that in the patch, though, so that anyone evaluating upstream merging in the future can just move along.

• patch-magic-Magdir-msdos.diff: having file produce different results depending on which packaging system was used seems very undesirable to me. However, a change like this would definitely be appropriate to bring up on the file mailing list or bug tracker.

r41083 “patch the magic information to fix a bogus entry"

Yes, I can see the reasoning. The magic for office files is non-trivial and has been discussed quite a bit on the developers list. (And I'm not expert in it.)

I just feel that this kind of a modification to file is undesirable, as it means that file from MacPorts then gives different results from all other installations of file. If this results in someone reporting an issue to the file developers, it is annoying to realize it is different just due to the packaging system used.

I would encourage getting it fixed upstream, but I feel I don't have enough understanding of office file formats to pursue it myself. (Sorry.)

That said, I'll leave it to the owner of the package to decide how to maintain it. :)

• patch-magic-Makefile.am: an uncompiled collection of the magic entries is not needed, but I could see how it might be interesting as a reference. Was that the reason for the patch? I think proposing this upstream with the reasoning behind it would also be good.

r41083 “add a variant that installs the plain-text magic(5) file”

No real insight from that last one, but apparently at least one person uses it since they filed #23710.

And when it was made default (variant removed) in r102987, the comment only says "always include the plain-text magic" -- no reason given.

I'll see if there is a reason not to include it. I could imagine one being that editing the source doesn't result in the compiled version getting updated (similar to "newaliases" and sendmail).

[suominen (Kimmo Suominen)]

Replying to ryandesign@…:

Replying to kimmo@…:

Can you please provide more information about how the issue is not fixed and how to reproduce the problem?

I simply try to use the file command, and it fails in the way described in the summary of this ticket:

Are you sure the patch is included in your Portfile?

figaro:~> sw_vers
ProductName:	Mac OS X
ProductVersion:	10.9.2
BuildVersion:	13C64
figaro:~> file -version
file-5.17
magic file from /opt/local/share/misc/magic
figaro:~> port info file | head -1
file @5.17_1 (sysutils)
figaro:~> env | grep -E '(LANG|LC_)'
LC_CTYPE=en_US.UTF-8
figaro:~> echo hello > test
figaro:~> file test
test: ASCII text

The "5.17_1" version is my local port from ​https://github.com/suominen/macports-gw

[suominen (Kimmo Suominen)]

Replying to kimmo@…:

For a moment I thought maybe it is LANG somehow overriding LC_CTYPE even though it is supposed to be the other way around. But fortunately not.

figaro:~> env LANG=en_US.UTF-8 file test
test: ASCII text

[petrrr]

Cc Me!

[ryandesign (Ryan Carsten Schmidt)]

Please let me know if file 5.18 fixes the problem.

[suominen (Kimmo Suominen)]

It does.

Unfortunately it introduces a new problem, which affects the output of many cases, for example PNG files. You may want to add this patch until 5.19 comes out: ​https://raw.githubusercontent.com/suominen/macports-gw/master/sysutils/file/files/patch-post-518.diff

[ralph@…]

Replying to ryandesign@…:

Please let me know if file 5.18 fixes the problem.

Looks good here.

[ryandesign (Ryan Carsten Schmidt)]

Replying to kimmo@…:

Unfortunately it introduces a new problem, which affects the output of many cases, for example PNG files. You may want to add this patch until 5.19 comes out: ​https://raw.githubusercontent.com/suominen/macports-gw/master/sysutils/file/files/patch-post-518.diff

Thanks, I've committed that in r118485.

Replying to kimmo@…:

• patch-magic-Magdir-msdos.diff: having file produce different results depending on which packaging system was used seems very undesirable to me. However, a change like this would definitely be appropriate to bring up on the file mailing list or bug tracker.

r41083 “patch the magic information to fix a bogus entry"

Yes, I can see the reasoning. The magic for office files is non-trivial and has been discussed quite a bit on the developers list. (And I'm not expert in it.)

I just feel that this kind of a modification to file is undesirable, as it means that file from MacPorts then gives different results from all other installations of file. If this results in someone reporting an issue to the file developers, it is annoying to realize it is different just due to the packaging system used.

I would encourage getting it fixed upstream, but I feel I don't have enough understanding of office file formats to pursue it myself. (Sorry.)

That said, I'll leave it to the owner of the package to decide how to maintain it. :)

I've also removed this patch. Anyone interested in having this change back can convince the developers to add it to their sources.

[suominen (Kimmo Suominen)]

Thank you, Ryan.