Opened 14 years ago

Closed 14 years ago

Last modified 11 years ago

#25128 closed defect (fixed)

Point out when checksum errors are due to broken DNS or web servers

Reported by: ryandesign (Ryan Carsten Schmidt) Owned by: macports-tickets@…
Priority: Normal Milestone: MacPorts 1.9.2
Component: base Version: 1.8.2
Keywords: haspatch Cc: cooljeanius (Eric Gallager)
Port:

Description

We've had quite a lot of tickets filed lately about checksum errors, and I have a feeling a fair number of them are because of the "helpful" DNS servers ISPs seem to be so fond of providing these days. When asking such DNS servers about a hostname that does not exist, they do not respond that the hostname does not exist; instead, they respond with the IP address of the ISP's search page, and if you are trying to connect to it over HTTP, they do a web search for you and send you the results. OpenDNS does this by default, so does Time Warner's Roadrunner service and probably others. MacPorts can't currently recognize this, so it saves that search results page as the distfile, which then of course doesn't match the expected checksums.

The attached patch fixes this by using the "file" command to check if the downloaded file is an HTML page, and if so, discarding it and trying another mirror. I have tested this on Snow Leopard only. Before committing, I would like to verify that the "file" command on Leopard and preferably Tiger works the same way (i.e. accepts the same flags).

There is already special-case code in this procedure for dealing with SourceForge mirrors. Possibly the new code I propose could replace the old SourceForge code; I was unable to locate a test case that triggered the SourceForge special-case code.

Attachments (3)

helpful-dns.diff (1.1 KB) - added by ryandesign (Ryan Carsten Schmidt) 14 years ago.
proposed patch
helpful-dns-2.diff (1.3 KB) - added by ryandesign (Ryan Carsten Schmidt) 14 years ago.
revised patch, tested on Tiger, Leopard, & Snow Leopard
helpful-dns-3.diff (1.3 KB) - added by ryandesign (Ryan Carsten Schmidt) 14 years ago.
final version

Download all attachments as: .zip

Change History (12)

Changed 14 years ago by ryandesign (Ryan Carsten Schmidt)

Attachment: helpful-dns.diff added

proposed patch

comment:1 Changed 14 years ago by ryandesign (Ryan Carsten Schmidt)

The --mime-type flag appears to only be valid for the version of the file command on Snow Leopard (and the version in MacPorts); it doesn't exist in the version of file on Tiger or Leopard. But the --mime flag does, and it outputs the same information I need (and a little more that I don't need, which I can just discard).

Changed 14 years ago by ryandesign (Ryan Carsten Schmidt)

Attachment: helpful-dns-2.diff added

revised patch, tested on Tiger, Leopard, & Snow Leopard

comment:2 Changed 14 years ago by ryandesign (Ryan Carsten Schmidt)

Keywords: haspatch added

comment:3 Changed 14 years ago by jmroot (Joshua Root)

Summary: Avoid checksum errors due to helpful DNS serversAvoid checksum errors due to broken DNS servers

I really don't like the idea of legitimising the completely broken behaviour of these DNS servers (and web servers, since they must not even be serving the "helpful" page with a 404 code). The checksum failure reports do indicate a genuine issue with the ports (even if it would be an extremely minor one given working DNS). Giving additional information about the likely cause of a checksum failure would be OK.

comment:4 in reply to:  3 Changed 14 years ago by ryandesign (Ryan Carsten Schmidt)

Replying to jmr@…:

I really don't like the idea of legitimising the completely broken behaviour of these DNS servers (and web servers, since they must not even be serving the "helpful" page with a 404 code).

Well, we could go on a campaign of lobbying the ISPs to fix this. I think this would be a lot of work and I don't know that it would be successful. Meanwhile the attached patch was a couple lines of code and is completely successful as far as I can tell in my testing.

The checksum failure reports do indicate a genuine issue with the ports (even if it would be an extremely minor one given working DNS). Giving additional information about the likely cause of a checksum failure would be OK.

The genuine issue is that the port is referring to a server that doesn't exist? True, but in the case of a properly-functioning DNS server, we simply skip it and move on; the patch in this ticket simply affords that same convenience to users with these increasingly-popular DNS servers. And the patch does print a detailed error message explaining that the distfile file was unexpectedly an HTML page, so if a port maintainer sees this output (either experienced themselves or submitted in a ticket) they can investigate then, just like they can if they see a message that a server could not be contacted.

For me, without the patch, I encounter the issue extremely often, because I often work at friends' houses and coffee shops, each of which generally has such a DNS server now. My mirror_sites.tcl on each of my local machines is modified to include each of the other machines (so they can fetch distfiles from each other quickly); when using my laptop elsewhere, these local machine names don't exist so I run into the issue for pretty much every port. So I would like to commit this patch.

Changed 14 years ago by ryandesign (Ryan Carsten Schmidt)

Attachment: helpful-dns-3.diff added

final version

comment:5 Changed 14 years ago by jmroot (Joshua Root)

Milestone: MacPorts 1.9.1

comment:6 Changed 14 years ago by jmroot (Joshua Root)

Milestone: MacPorts 1.9.2
Resolution: fixed
Status: newclosed
Summary: Avoid checksum errors due to broken DNS serversPoint out when checksum errors are due to broken DNS or web servers

Let's be helpful without sweeping the problem under the rug. r70143

comment:7 Changed 14 years ago by ryandesign (Ryan Carsten Schmidt)

Joshua also created wiki page MisbehavingServers describing the problem.

comment:8 Changed 12 years ago by ryandesign (Ryan Carsten Schmidt)

See #33930 for the continuation.

comment:9 Changed 11 years ago by cooljeanius (Eric Gallager)

Cc: egall@… added

Cc Me!

Note: See TracTickets for help on using tickets.