Opened 4 years ago

Last modified 11 days ago

#48899 assigned update

ghc & haskell-platform update?

Reported by: J.Gilbey@… Owned by: neverpanic (Clemens Lang)
Priority: Normal Milestone:
Component: ports Version:
Keywords: Cc: elventear (Pepe Barbe), marcel@…, claus.atzenbeck@…, stromnov (Andrew Stromnov), andre.david@…, arthur.michaut@…, john@…, jowens@…, grimreaper (Eitan Adler), gorticus (Jason Mitchell), kprussing (Keith), akimd (Akim Demaille), dsedivec, agm650, mouse07410 (Mouse), mojca (Mojca Miklavec), l2dy (Zero King)
Port: haskell-platform ghc

Description

Is it feasible to update the haskell-platform port at some point? Upstream is now at 7.10.2, which introduces some significant changes from what I can tell.

Thanks!

Change History (48)

comment:1 Changed 4 years ago by mf2k (Frank Schima)

Cc: cal@… removed
Owner: changed from macports-tickets@… to cal@…
Version: 2.3.3

comment:2 Changed 3 years ago by neverpanic (Clemens Lang)

Status: newassigned

Yes, I'm aware of it and I plan to update it.

comment:3 Changed 3 years ago by jason.johnson.081@…

Hello, any update on this?

comment:4 Changed 3 years ago by jason.johnson.081@…

Hello,

Is this still maintained? This issue is getting close to a year old.

Thanks

comment:5 Changed 3 years ago by neverpanic (Clemens Lang)

I'd still like to do this, but time is limiting. If you're willing to help, I can give you some pointers.

comment:6 Changed 3 years ago by neverpanic (Clemens Lang)

Status update: I've had a first successful build of the new GHC and am currently testing Debian's patch to fix an issue where the build path of the compiler leaks into the ABI hashes (see https://ghc.haskell.org/trac/ghc/ticket/10424), since that would affect us with every binary package we ship from the buildbots.

comment:7 Changed 3 years ago by larryv (Lawrence Velázquez)

Cc: larryv@… added

Cc Me!

comment:8 Changed 3 years ago by elventear (Pepe Barbe)

Cc: elventear@… added

Cc Me!

comment:9 Changed 2 years ago by larryv (Lawrence Velázquez)

Cc: marcel@… claus.atzenbeck@… added

Adding reporters of #52221 and #52317.

comment:10 Changed 2 years ago by stromnov (Andrew Stromnov)

Cc: stromnov@… added

Cc Me!

comment:11 Changed 2 years ago by andre.david@…

Cc: andre.david@… added

Cc Me!

comment:12 Changed 2 years ago by arthur.michaut@…

Cc: arthur.michaut@… added

Cc Me!

comment:13 Changed 2 years ago by andre.david@…

According to #52317, this is blocking ghc availability in OSX sierra (and consequently of pandoc).

comment:14 Changed 2 years ago by john@…

Cc: john@… added

Cc Me!

comment:15 Changed 2 years ago by machielbos@…

Here's another user who needs ghc under macOS Sierra to install pandoc!

comment:16 Changed 2 years ago by larryv (Lawrence Velázquez)

Cc: jowens@… added

Has duplicate #52491.

comment:17 Changed 2 years ago by larryv (Lawrence Velázquez)

Cc: larryv@… removed

Cc Me!

comment:18 Changed 2 years ago by grimreaper (Eitan Adler)

Cc: lists@… added

Cc Me!

comment:19 Changed 2 years ago by neverpanic (Clemens Lang)

comment:20 Changed 2 years ago by gorticus (Jason Mitchell)

Cc: jason-macports@… added

Cc Me!

comment:21 Changed 2 years ago by koyeung (King-On Yeung)

Cc: koyeung added

comment:22 Changed 2 years ago by kprussing (Keith)

Is a major hold up still the need to create all the Portfiles to build stack? I started on a few to get pandoc updated, but then I saw how many I would need to update. So, is there a communal place where the edits are being worked on (per the first comment in the linked message)?

comment:23 Changed 2 years ago by kprussing (Keith)

Cc: kprussing added

comment:24 Changed 2 years ago by akimd (Akim Demaille)

Cc: akimd added

comment:25 Changed 2 years ago by informatimago (Pascal J. Bourguignon)

It looks like by just adding:

#ifdef darwin_HOST_OS
#include <mach/mach_time.h>
#endif

in rts/posix/GetTime.c , the following errors can be eliminated.

:info:build "inplace/bin/ghc-stage1" -optc-m64 -optc-fno-stack-protector -optc-Wall -optc-Wextra -optc-Wstrict-prototypes -optc-Wmissing-prototypes -optc-Wmissing-declarations -optc-Winline -optc-Waggregate-return -optc-Wpointer-arith -optc-Wmissing-noreturn -optc-Wnested-externs -optc-Wredundant-decls -optc-Iincludes -optc-Iincludes/dist -optc-Iincludes/dist-derivedconstants/header -optc-Iincludes/dist-ghcconstants/header -optc-Irts -optc-Irts/dist/build -optc-DCOMPILING_RTS -optc-fno-strict-aliasing -optc-fno-common -optc-DDTRACE -optc-O2 -optc-fomit-frame-pointer -optc-DRtsWay=\"rts_v\" -static  -H32m -O -Iincludes -Iincludes/dist -Iincludes/dist-derivedconstants/header -Iincludes/dist-ghcconstants/header -Irts -Irts/dist/build -DCOMPILING_RTS -package-name rts -dcmm-lint  -DDTRACE     -i -irts -irts/dist/build -irts/dist/build/autogen -Irts/dist/build -Irts/dist/build/autogen           -O2    -c rts/posix/GetTime.c -o rts/dist/build/posix/GetTime.o
:info:build 
:info:build rts/posix/GetTime.c:44:5:
:info:build      error: use of undeclared identifier 'mach_timebase_info_data_t'
:info:build         mach_timebase_info_data_t info;
:info:build         ^
:info:build 
:info:build rts/posix/GetTime.c:45:12:
:info:build      warning: implicit declaration of function 'mach_timebase_info' is invalid in C99 [-Wimplicit-function-declaration]
:info:build         (void) mach_timebase_info(&info);
:info:build                ^
:info:build 
:info:build rts/posix/GetTime.c:45:32:
:info:build      error: use of undeclared identifier 'info'; did you mean 'sinf'?
:info:build         (void) mach_timebase_info(&info);
:info:build                                    ^~~~
:info:build                                    sinf
:info:build 
:info:build /usr/include/math.h:342:14:  note: 'sinf' declared here
:info:build extern float sinf(float);
:info:build              ^
:info:build 
:info:build rts/posix/GetTime.c:46:44:
:info:build      error: use of undeclared identifier 'info'; did you mean 'int'?
:info:build         timer_scaling_factor_numer = (uint64_t)info.numer;
:info:build                                                ^~~~
:info:build                                                int
:info:build 
:info:build rts/posix/GetTime.c:46:44:  error: expected expression
:info:build 
:info:build rts/posix/GetTime.c:47:44:
:info:build      error: use of undeclared identifier 'info'; did you mean 'int'?
:info:build         timer_scaling_factor_denom = (uint64_t)info.denom;
:info:build                                                ^~~~
:info:build                                                int
:info:build 
:info:build rts/posix/GetTime.c:47:44:  error: expected expression
:info:build 1 warning and 6 errors generated.
:info:build make[1]: *** [rts/dist/build/posix/GetTime.o] Error 1

comment:27 Changed 22 months ago by dsedivec

Cc: dsedivec added

comment:28 Changed 12 months ago by raimue (Rainer Müller)

Port: ghc added

comment:29 Changed 8 months ago by markemer (Mark Anderson)

I'd be willing to help, but I'm running in circles just trying to get ghc 8.4.3 to build. Are you still offering up those pointers? I think I brought this up a while back while trying to install pandoc.

comment:30 Changed 8 months ago by neverpanic (Clemens Lang)

Absolutely. Here's my current WIP state that updates to 8.0.3: https://github.com/neverpanic/macports-ports/tree/haskell-platform-update. You may have to get a newer ghc-bootstrap compiler to compile 8.4.3, though.

comment:31 Changed 4 months ago by agm650

Cc: agm650 added

comment:32 Changed 4 months ago by agm650

Hello,

I'm willing to help on this topic. I've made the ghc-bootstrap to use the 8.6.2 version.

So after I guess I have to go on with the whole haskell-platform update (a bit scaring)?

comment:33 Changed 3 months ago by ryandesign (Ryan Schmidt)

Cc: mouse07410 added
Summary: haskell-platform update?ghc & haskell-platform update?

Has duplicate #57825.

comment:34 Changed 3 months ago by mouse07410 (Mouse)

Guys, I know little about Haskell, but something about C and such. Can I help?

comment:35 Changed 3 months ago by koyeung (King-On Yeung)

Cc: koyeung removed

comment:36 Changed 3 months ago by kencu (Ken)

Currently we are trying to manually replace the work of cabal to integrate it into the Macports way of doing things, and this is proving to be a huge, perhaps unsustainable, project.

Just calling cabal to install the packages is easy. But doesn't let Macports know what is going on.

We could find a way to drive and query cabal....(much like we might do with pip and cpan, perhaps.)

Or, we could parse the cabal package information or output to automate portfile generation and updates.

But doing it all manually is collapsing...

comment:37 Changed 3 months ago by jdswinbank (John Swinbank)

Cc: jdswinbank added

comment:38 Changed 3 months ago by jdswinbank (John Swinbank)

Cc: jdswinbank removed

comment:39 Changed 3 months ago by mouse07410 (Mouse)

Just calling cabal to install the packages is easy. But doesn't let Macports know what is going on

In Python, there are far more packages that are installable (and the user is likely to want to install) via pip than are supported by Macports. This already is causing me problems, because some packages are available in both Macports and pip, others - only in pip. And there are frequent collisions between the two.

Based on the above Python experience, I strongly suggest that we do as you said above:

Just calling cabal to install the packages is easy. But doesn't let Macports know what is going on

and keep Macports away from Hackage package management. You aren't going to be able to replicate in Macports even 10% of what's on Hackage - like the Python maintainers cannot replicate a significant part of PYPI. And that's fine. Just provide a working Python and a working pip - and the user will take care of the rest. I suggest using the same approach with Haskell and cabal.

comment:40 Changed 3 months ago by kencu (Ken)

what have the other package management systems decided to do about this issue? I assume all of them are in exactly the same boat.

comment:41 Changed 3 months ago by mouse07410 (Mouse)

what have the other package management systems decided to do about this issue? I assume all of them are in exactly the same boat.

Some of them probably do something reasonable, and some - don't (in my opinion). For example, IMHO what Macports Python does is crazy (IMHO), and causes unnecessary conflicts because, as I explained, there's no way for Macports to replicate he entire PYPI repo, so some packages get overridden when the user needs to install something, e.g., from PYPI and that thing pulls its own dependencies - again directly from PYPI. Leaving the packages to the corresponding infrastructure, and providing just the "main" binaries - ghc, cabal, and stack - would be best. And simplest too.

Then, BTW, you won't need to bother with the unnecessary dependency on LLVM. I just rebuilt GHC from the sources, and it was smooth as castor oil. Why Macport GHC port can't do the same? Don't overcomplicate things - KISS rules.

If the maintainers feel married to the current approach - let's leave the current (ancient) port of ghc-7.8.3 alone, with all the goodies it offers, and add another "minimalistic" port of ghc-8.6.3 that provides only ghc (not dependent on LLVM!), cabal, and stack. Everything else users can install themselves just fine.

Last edited 3 months ago by mouse07410 (Mouse) (previous) (diff)

comment:42 Changed 3 months ago by mouse07410 (Mouse)

Last edited 3 months ago by mouse07410 (Mouse) (previous) (diff)

comment:43 in reply to:  41 Changed 3 months ago by neverpanic (Clemens Lang)

Replying to mouse07410:

Some of them probably do something reasonable, and some - don't (in my opinion). For example, IMHO what Macports Python does is crazy (IMHO), and causes unnecessary conflicts because, as I explained, there's no way for Macports to replicate he entire PYPI repo, so some packages get overridden when the user needs to install something, e.g., from PYPI and that thing pulls its own dependencies - again directly from PYPI.

I think the current approach is supposed to be "package everything that needs native dependencies". The question of native dependencies is where the whole approach of "don't package any of the libraries" falls apart, because users use MacPorts precisely to avoid having to compile their own copy of $library_used_by_python_module or to figure out how to get the Python build system to find MacPorts' copy of said library.

Leaving the packages to the corresponding infrastructure, and providing just the "main" binaries - ghc, cabal, and stack - would be best. And simplest too.

Back when we originally packaged GHC, we were told by the Haskell community that we should really be providing GHC "batteries included" by packaging the haskell platform. Maybe it is time to revisit this decision.

However, keep in mind that many users will expect packages such as pandoc and shellcheck to be available from MacPorts. Approaching this as "if you want pandoc, install cabal/stack and run cabal/stack install pandoc" is not the best alternative from a UX perspective – especially since we don't have a good way to communicate this exception to the common usage of MacPorts.

Then, BTW, you won't need to bother with the unnecessary dependency on LLVM. I just rebuilt GHC from the sources, and it was smooth as castor oil. Why Macport GHC port can't do the same? Don't overcomplicate things - KISS rules.

The LLVM dependency isn't the issue here, I think. Building a newer version of GHC isn't very complicated. It's the platform vs. no platform and libraries vs. not packaging libraries discussion that makes things complicated.

Btw, stack has hundreds of dependencies – it's what kept me from finishing the packaging of a current version of haskell-platform. Even just building stack with the current approach probably needs some kind of automation.

If the maintainers feel married to the current approach - let's leave the current (ancient) port of ghc-7.8.3 alone, with all the goodies it offers, and add another "minimalistic" port of ghc-8.6.3 that provides only ghc (not dependent on LLVM!), cabal, and stack. Everything else users can install themselves just fine.

I'm not married to that approach. If you want to submit patches to update the GHC port and drop haskell-platform as well as the other Haskell ports, I'm fine with that. I don't have a strong opinion on LLVM or not (although I don't believe it's a big problem), however, I would expect commonly used ports such as pandoc and shellcheck to be available from MacPorts. If you can provide that plus a copy of stack, that'd be great.

Maybe a good approach would also be to build all tools (such as pandoc) using stack from a Portfile, so that they all ship their dependencies in a private folder. We've recently adopted a similar approach for ports using rust's cargo. See the cargo 1.0 PortGroup, for example.

comment:44 Changed 3 months ago by mouse07410 (Mouse)

I think the current approach is supposed to be "package everything that needs native dependencies". The question of native dependencies is where the whole approach of "don't package any of the libraries" falls apart

Respectfully disagree. One reason - it proved to because infeasible to provide Macports clones of all the packages maintained by other env-specific ecosystems. Python is the best example of that.

For example, I use Macports not because I can't do pip install pycryptodome, but because it's convenient for me to, e.g., install a reasonable version of OpenSSL via sudo port install openssl (actually, a bad example - as one of the contributors, I do keep source tree of OpenSSL, and regularly rebuild master and stable branches ;).

I see absolutely nothing wrong for the user to do sudo port install gcc8 && pip-3.7 install numpy. It does not seem much harder than sudo port install py37-numpy.

Approaching this as "if you want pandoc, install cabal/stack and...

Why would a Haskell user want to install pandoc via Macports??? Any more than installing Haskell-libsodium via Macports? Or <put your favorite Haskell package here>?

Macports cannot and shouldn't become a substitute for ecosystem-specific package managers.

Even just building stack with the current approach probably needs some kind of automation...

Yes, non-trivial - which is why I'd be perfectly content packaging pre-built stack binary and keeping it. For example, my system uses pre-built stack - I did not rebuilt it, not do I want to. It works. I don't need Macports to provide it to me, nor do I feel like I'm obligated to rebuild it from sources.

It's the platform vs. no platform and libraries vs. not packaging libraries discussion that makes things complicated

There already is Haskell Platform. The main reason I don't use it in its entirety is because it's GHC conflicts with Macports libiconv. So I had to rebuild GHC myself. But stack from Haskell Platform is perfectly usable as is. Whatever Haskell libraries I need - stack or cabal will install for me, and would do that the right way. No reason for Macports to try emulating that.

Back when we originally packaged GHC, we were told by the Haskell community that we should really be providing GHC "batteries included" by packaging the haskell platform. Maybe it is time to revisit this decision.

Yes!! I strongly urge to revisit. Haskell ecosystem is very strong. It's neither needed nor possible to replicate it. The only reason I'm suggesting refreshing the port is because the default GHC provided by Haskel ecosystem conflicts with Macports. Everything else from Haskell eco coexists with Macports just fine, so no need to re-do it.

Maybe a good approach would also be to build all tools (such as pandoc) using stack from a Portfile, so that they all ship their dependencies in a private folder. We've recently adopted a similar approach for ports using rust's cargo

Now, that is an idea. But I'm getting concerned when I hear "to build *all tools" from you. My question is - since those tools are trivially installed by stack, what benefit is there in offering what essentially is a Macports wrapper for it? And is it worth the cost (time and efforts)? Is sudo port install pandoc really that much simpler than stack install pandoc, especially for somebody who presumably got GHC (with cabal and stack) because that somebody is developing within the Haskell ecosystem?

comment:45 in reply to:  44 Changed 3 months ago by neverpanic (Clemens Lang)

Replying to mouse07410:

For example, I use Macports not because I can't do pip install pycryptodome, but because it's convenient for me to, e.g., install a reasonable version of OpenSSL via sudo port install openssl (actually, a bad example - as one of the contributors, I do keep source tree of OpenSSL, and regularly rebuild master and stable branches ;).

I see absolutely nothing wrong for the user to do sudo port install gcc8 && pip-3.7 install numpy. It does not seem much harder than sudo port install py37-numpy.

That's fine if (and only if) the pip-3.7 install numpy will work out of the box. It usually does with stuff in /usr/local, but making your pip find stuff in /opt/local commonly requires flags. Additionally, using MacPorts pip will put files in MacPorts prefix that MacPorts doesn't know about, which has previously led to problems. We want to avoid that.

Approaching this as "if you want pandoc, install cabal/stack and...

Why would a Haskell user want to install pandoc via Macports??? Any more than installing Haskell-libsodium via Macports? Or <put your favorite Haskell package here>?

Because people don't (and shouldn't) care that pandoc or shellcheck are written in Haskell. They just want the tool.

Yes, non-trivial - which is why I'd be perfectly content packaging pre-built stack binary and keeping it. For example, my system uses pre-built stack - I did not rebuilt it, not do I want to. It works. I don't need Macports to provide it to me, nor do I feel like I'm obligated to rebuild it from sources.

Sure, but given this argument, we might as well not package Haskell at all in MacPorts. We could just remove it and tell people to use the upstream binaries provided by the GHC developers.

There already is Haskell Platform. The main reason I don't use it in its entirety is because it's GHC conflicts with Macports libiconv. So I had to rebuild GHC myself.

Hm, that's weird. Binaries on macOS reference their libraries using absolute paths, and the upstream GHC binary shouldn't link against /opt/local/lib/libiconv.2.dylib, so unless you have DYLD_LIBRARY_PATH in your environment, the upstream Haskell Platform should just work. Have you looked into why that's not the case?

Maybe a good approach would also be to build all tools (such as pandoc) using stack from a Portfile, so that they all ship their dependencies in a private folder. We've recently adopted a similar approach for ports using rust's cargo

Now, that is an idea. But I'm getting concerned when I hear "to build *all tools" from you. My question is - since those tools are trivially installed by stack, what benefit is there in offering what essentially is a Macports wrapper for it? And is it worth the cost (time and efforts)? Is sudo port install pandoc really that much simpler than stack install pandoc, especially for somebody who presumably got GHC (with cabal and stack) because that somebody is developing within the Haskell ecosystem?

You're making a few assumptions here that I don't think are correct:

  • I'm not trying to put an emphasis on all tools, but an emphasis on tools, i.e. it wouldn't make sense to package libraries in this way.
  • The benefit of having those wrapped is that you have a single way to update them and a single interface to manage them. I don't want to have to care about five different language-specific package managers when all I want is a single end-user tool.
  • Most of the Haskell stuff is distributable, so our build servers will provide precompiled binaries. This significantly speeds up installation.
  • Given sufficient automation (and none of the dependency hell), the time required to do this would probably not be very high, i.e. the cost would be low. In my eyes, that would make it worth the effort.

I agree there's not a big difference for somebody who develops in the Haskell ecosystem, but those people really don't need our help in figuring this out anyway.

comment:46 Changed 3 months ago by mouse07410 (Mouse)

Sure, but given this argument, we might as well not package Haskell at all in MacPorts. We could just remove it and tell people to use the upstream binaries provided by the GHC developers

That's what I wanted to do. It failed - see below.

There already is Haskell Platform. The main reason I don't use it in its entirety is because it's GHC conflicts with Macports libiconv. So I had to rebuild GHC myself.

Hm, that's weird. Binaries on macOS reference their libraries using absolute paths, and the upstream GHC binary shouldn't link against /opt/local/lib/libiconv.2.dylib, so unless you have DYLD_LIBRARY_PATH in your environment, the upstream Haskell Platform should just work. Have you looked into why that's not the case?

Of course I did (see #57821). There are two reasons for this problem:

  • libHSbase exists only as libHSbase.a, so forget about absolute paths (the offending/offended library is static);
  • libiconv.dylib from Macports mangles function names, but preserves the same library name.

The benefit of having those [(presumably executable) tools] wrapped is that you have a single way to update them and a single interface to manage them. I don't want to have to care about five different language-specific package managers when all I want is a single end-user tool

I don't see many Haskell-based tools that would qualify, and thus make what you describe worthwhile. But then, I cannot count myself as a significant contributor to Macports - so I defer to you here. Do as you see fit.

I agree there's not a big difference for somebody who develops in the Haskell ecosystem, but those people really don't need our help in figuring this out anyway

Ideally, a developer would download Haskell Platform and go from there. Because that platform conflicts with Macports (see above), the options for the developer are to

  • replace Macports with Brew, which does not suffer from this problem;
  • rebuild GHC from sources to make it Macports-compatible (that's what I finally did, exhausting other options);
  • make Macports libiconv.dylib compatible - make a variant that doesn't mangle names (this request was shot down);
  • get Macports GHC (considering how old it is, and the likelihood that Macports-installed Haskell packages would conflict with the newer ones makes this completely unattractive).

P.S. Factor in the fact that the current Haskell build tools (cabal and stack) are moving towards local snapshot-based approach - no global database of packages, registries at best per-user, or more likely - per-project. I don't see all the consequences of this, but it doesn't seem to play well with the Macports approach to global (per-machine) package repository.

comment:47 Changed 4 weeks ago by mojca (Mojca Miklavec)

Cc: mojca added

comment:48 Changed 11 days ago by l2dy (Zero King)

Cc: l2dy added
Note: See TracTickets for help on using tickets.