tag:blogger.com,1999:blog-927289640963145319.post1726803774680770691..comments2023-12-07T00:46:00.649-08:00Comments on Johan Tibell: The Cabal of my dreamsJohan Tibellhttp://www.blogger.com/profile/06875432206357419172noreply@blogger.comBlogger24125tag:blogger.com,1999:blog-927289640963145319.post-36671207993280872252015-07-22T02:30:51.929-07:002015-07-22T02:30:51.929-07:00There is in fact a leiningen plugin to do what you...There is in fact a leiningen plugin to do what you describe: lein-try https://github.com/rkneufeld/lein-tryjjlhttps://github.com/jjl/noreply@blogger.comtag:blogger.com,1999:blog-927289640963145319.post-21859092256826646382013-09-06T01:43:31.009-07:002013-09-06T01:43:31.009-07:00(3) is the major reason I cabal(-dev) install manu...(3) is the major reason I cabal(-dev) install manually things.<br />Leiningen (Clojure build tool) works the way you describe: you need to have a configured project and run 'lein repl' in it so that you may dowload and toy with a new lib in the repl. So you end up creating blank projects with just one dependency _just to bring it into repl scope_.<br /><br />Okay, that is not so horrible, but still it's annoying when you want to just try some stuff without needing it immediately for a project.Anonymoushttps://www.blogger.com/profile/11382268504770604242noreply@blogger.comtag:blogger.com,1999:blog-927289640963145319.post-62453378240948446522012-03-19T01:40:44.541-07:002012-03-19T01:40:44.541-07:00> Package databases are a mechanism, not a goal...> Package databases are a mechanism, not a goal. The goals are the use cases we want to support.<br /><br />Sure! My point is to bring up a use case which appears to be a bit different than the ones you were thinking of. Namely, a large project consisting of many packages, where there is a need for convenient fine-grained control over which combination of package versions get built together, not necessary as an executable, and without the hack of artificially narrowing the version ranges in many cabal files.<br /><br />I admit that I am biased, because this is the use case I deal with on a daily basis in my work. But I believe that support for this kind of use case is part of what makes the difference between Haskell being a serious industrial tool and not just a research topic and curious toy.Yitzhttp://claimid.com/yitzgalenoreply@blogger.comtag:blogger.com,1999:blog-927289640963145319.post-42040069897624335222012-03-18T19:31:25.423-07:002012-03-18T19:31:25.423-07:00cabal-dev install is very close to the cabal build...cabal-dev install is very close to the cabal build I describe above. Shortcomings include cabal-dev add-source taking a snapshot of another package's source directory instead of creating a live "link" to it.<br /><br />Lets try to separate mechanism from goal. We're engineers so we tend to get them mixed up. Package databases are a mechanism, not a goal. The goals are the use cases we want to support (build the package in the current source tree, launch ghci with the current package and its dependencies in scope, etc.)Johan Tibellhttps://www.blogger.com/profile/06875432206357419172noreply@blogger.comtag:blogger.com,1999:blog-927289640963145319.post-48859277088576342792012-03-18T03:46:16.398-07:002012-03-18T03:46:16.398-07:00Well, for step (3), when my project consists of ma...Well, for step (3), when my project consists of many packages, it can be very complex and fiddly to get all of the cabal files exactly right. During development, that may not be what I want to spend my time on at that stage. I just want to run the tests. And as Tillman points out, I may want to bring up some specific combination of package versions in ghci.<br /><br />It's so simple conceptually just to create a fresh directory and `cabal-dev install` the exact packages I need at exactly the right versions. To me, at least, the meaning of "install" is absolutely clear and is exactly what I need. It means: make these specific package versions available as compiled libraries in the current sandbox. I can group together large groups of unpublished package versions in separate lightweight yackage servers, so that I can load them easily into any sandbox.<br /><br />If it's somehow possible to get that kind of clarity, simplicity, and control without sandboxes, that's fine.<br /><br />The only problems I see with the current cabal - assuming that everything is done within sandboxes - are that you need to rebuild a lot of artifacts in each sandbox (not so much of a problem for me in practice usually, but nice if it could be improved), and the lack of parallel builds. Your good ideas could definitely help with those.<br /><br />All of that is for the case where a team is working on a large number of their own packages as part of a large project. For the simple case where I am working on only one package of my own, but with a large number of dependencies on other people's packages, then the system you are describing sounds great. There I don't want to have to think about package versions any more than absolutely necessary.Yitzhttp://claimid.com/yitzgalenoreply@blogger.comtag:blogger.com,1999:blog-927289640963145319.post-40417629878312657182012-03-15T16:13:26.131-07:002012-03-15T16:13:26.131-07:00cabal with explicit setup.hs dependencies would he...cabal with explicit setup.hs dependencies would help a lot here.<br /><br />We can already easily incorporate autotools builds into a cabal build; but I don't think that functionality has much business being an internal component to cabal/cabal-install. Extensions like that are *very* handy, but I think they should be extensions to a lean core tool.<br /><br />Cabal just dosen't make it easy to create complex new features for a specific build process (or class of build processes...).Roganhttps://www.blogger.com/profile/03431222160798159339noreply@blogger.comtag:blogger.com,1999:blog-927289640963145319.post-30223063717138000942012-03-15T10:05:57.355-07:002012-03-15T10:05:57.355-07:00In my use case (2), I actually want to see whether...In my use case (2), I actually want to see whether it would install, of course. Usually, if it builds, it also installs, but maybe some packages do extra steps in the install phase, what do I know. On the other hand, since there's no way to undo these extra steps, maybe I don't want to run them when I don't know yet whether I really want the package in the end.<br /><br />Maybe I actually just want `cabal test` to accept the name of a package on hackage. The semantics of `cabal test foo` would be something like the following:<br /><br />cabal unpack foo<br />cd foo-1.2.3<br />cabal configure --activate-testsuites-if-possible<br />cabal build<br />cabal test --do-not-complain-if-there-are-no-testsuites<br />cd ..<br />rm -r foo-1.2.3Tillmann Rendelnoreply@blogger.comtag:blogger.com,1999:blog-927289640963145319.post-82393545875665646642012-03-15T08:15:48.183-07:002012-03-15T08:15:48.183-07:00Tillmann,
I think (1) is a good use of cabal inst...Tillmann,<br /><br />I think (1) is a good use of cabal install. It combines cabal build and 'cp dist/build/exe $PATH'. I tried to say as much in the post.<br /><br />Why would (2) installing a library be required to see if it compiles? Unpack the source code and run cabal build. If it builds, it builds! Install only adds a file copy (and ghc-register) call on top of that.<br /><br />(3) is useful, although I expect it to be largely done through cabal repl (built during last GSoC), which would let you use ghci with a project under development and even if invoking ghci requires running some preprocessors first (like hsc2hs.)Johan Tibellhttps://www.blogger.com/profile/06875432206357419172noreply@blogger.comtag:blogger.com,1999:blog-927289640963145319.post-34588659699158846272012-03-15T08:10:44.434-07:002012-03-15T08:10:44.434-07:00Yitz, I don't quite follow your example in the...Yitz, I don't quite follow your example in the last paragraph. If you're developing a library, wouldn't you follow these steps:<br /><br />1. Edit<br />2. Compile (cabal build)<br />3. Run unit/integration/system tests (cabal test, manual tests)<br />4. Commit<br />5. Goto 1<br /><br />?<br /><br />Why does the library be installed to test it, unless your testing the packaging mechanism itself?Johan Tibellhttps://www.blogger.com/profile/06875432206357419172noreply@blogger.comtag:blogger.com,1999:blog-927289640963145319.post-80211969412826468682012-03-15T08:07:41.923-07:002012-03-15T08:07:41.923-07:00I have three use cases for `cabal install` which c...I have three use cases for `cabal install` which cannot be supported by `cabal build` because they are not in the context of a cabal project.<br /><br /> (1) Installing an executable.<br /> (2) Testing whether a library compiles on my machine.<br /> (3) Installing a library to explore it in ghci.<br /><br />These use cases should continue to be supported by cabal.Tillmann Rendelnoreply@blogger.comtag:blogger.com,1999:blog-927289640963145319.post-44612386630846771152012-03-15T02:08:46.424-07:002012-03-15T02:08:46.424-07:00I completely agree with what you say about "h...I completely agree with what you say about "hermetic builds". That is not a problem with the entire cabal system; it is a problem with the cabal command, which always targets a "global" package database. (Even the user package database is "global" because it affects all projects.)<br /><br />So I use `cabal install` exactly once after each new GHC install: `cabal install cabal-dev`. After that, I only use cabal-dev.<br /><br />You are suggesting something intermediate between cabal and cabal-dev: conceptually separate package databases per project, like with cabal-dev, but smart enough to share artifacts between projects when it can be proven that it makes sense. That would be really nice.<br /><br />`cabal install [lib]` (or `cabal-dev install [lib]`) makes perfect sense; that's what I do all day long. A large system is composed of many libraries, and only a small proportion of them create executables. I don't want to be forced to link my library into some other irrelevant package, just because it happens to create an executable, every time I compile the library I am working on. In order to test my library after I compile it, it usually does need to be "installed" in some package database every time I compile it, so `cabal build [lib]` would hardly be useful.Yitzhttp://claimid.com/yitzgalenoreply@blogger.comtag:blogger.com,1999:blog-927289640963145319.post-67168772459501685892012-03-15T01:46:08.353-07:002012-03-15T01:46:08.353-07:00For the build-bot project I'm working on, we&#...For the build-bot project I'm working on, we're currently writing some kind of "local hackage" service, that may at some point be relevant for a few of your ideas. <br /><br />Also, we've been discussing with Duncan about, at some point, factoring out many of the features cabal-install has in a separate library to be able to reuse them in the build-bot project and for the new shiny hackage-server. Maybe that would be the best time for making some changes so that cabal-install can support some of the ideas described in the post or in the comments.Alp Mestanhttps://www.blogger.com/profile/07949237471833747550noreply@blogger.comtag:blogger.com,1999:blog-927289640963145319.post-64632931315077409282012-03-14T22:07:05.378-07:002012-03-14T22:07:05.378-07:00dagitj,
I think I agree with more or less everyth...dagitj,<br /><br />I think I agree with more or less everything you say and I hope we'll get to those more tricky issues soon. My proposals are a bit more modest, but should build up some of the infrastructure we will need whether we support multi-language projects, different build systems, etc.Johan Tibellhttps://www.blogger.com/profile/06875432206357419172noreply@blogger.comtag:blogger.com,1999:blog-927289640963145319.post-51135576300999622222012-03-14T21:53:06.950-07:002012-03-14T21:53:06.950-07:00Johan,
I think part of the problem we have now ha...Johan,<br /><br />I think part of the problem we have now has to do with Cabal's design. It tries to be in charge of everything. On top of that we have issues like Evan points out, which boils down to: Haskell's FFI support means that you want to build Haskell plus the transitive closure of whatever the FFI supports.<br /><br />Since we've designed Cabal to be in charge of this, we need Cabal to be able to build the transitive closure of whatever the FFI supports!<br /><br />One way to change this is to add a mode of operation to cabal so that other build systems can ask cabal for information or to do things. For example, Evan could use a makefile to orchestrate the build, even the build of the haskell bits, if cabal could read in the .cabal file, calculate which packages to use, and then return the ghc invocation lines. Those lines could then be issued from make in the context of the full build process.<br /><br />Another issue with cabal is that it only knows about what the cabal team lets it know about. Thinking in terms of preprocessors: I've run into build issues where hsc2hs had a feature to overcome the issue but cabal didn't support that hsc2hs feature simply because no one bothered to add it yet. Unfortunately, even if I added that feature and got it accepted it would be quite some time before I could expect anyone else to have that feature in their cabal.<br /><br />There are other cases other than preprocessors where I really want to define my own behaviors for cabal and be able to share them with other projects and developers. I really wish I could add support for this or that as a 'cabal plugin' and upload it to hackage.<br /><br />Imagine the way git works, 'git foo' invokes 'foo' which is allowed to be a separate command/script/program. We could have the same convention for cabal. Imagine if you could create cabal plugins (and we could teach cabal how to build them and install them as needed) this way. Then all the crazy gtk2hs build tools could be cabal plugins.<br /><br />Other examples, cabal-dev and cab could both be plugins to cabal instead of a separate tools. Same goes for the other cabal tools that are on hackage now.<br /><br />As functional programmers, the control that cabal has should feel inverted (and unnatural?) to us.dagitjhttps://www.blogger.com/profile/10574172070227922360noreply@blogger.comtag:blogger.com,1999:blog-927289640963145319.post-5202265094379443462012-03-14T19:38:12.676-07:002012-03-14T19:38:12.676-07:00+1
In addition, cabal should support multiple rem...+1<br /><br />In addition, cabal should support multiple remote-repo. <br />Not only, we can mirror the official hackage. If the hackage.org is down, the whole haskell community *freeze*. But also, anyone can do overlay private remote-repo with official hackage.Leon Reloadhttps://www.blogger.com/profile/16582991860063096339noreply@blogger.comtag:blogger.com,1999:blog-927289640963145319.post-4791649660845546322012-03-14T14:24:07.224-07:002012-03-14T14:24:07.224-07:00That's great to hear Mikhail.That's great to hear Mikhail.Johan Tibellhttps://www.blogger.com/profile/06875432206357419172noreply@blogger.comtag:blogger.com,1999:blog-927289640963145319.post-16142464248679851012012-03-14T14:18:12.735-07:002012-03-14T14:18:12.735-07:00I've been really busy during the last two mont...I've been really busy during the last two months and wasn't able to work on the parallel patches, but I'm still committed to merging them in.Mikhail Glushenkovhttps://www.blogger.com/profile/16766775468165268210noreply@blogger.comtag:blogger.com,1999:blog-927289640963145319.post-65942667312580432282012-03-14T14:13:19.078-07:002012-03-14T14:13:19.078-07:00Evan,
In that case make or some other language ag...Evan,<br /><br />In that case make or some other language agnostic build system is likely a better fit for you. The Cabal build system is still quite far away from being language agnostic. Perhaps if we merge in some ideas from Shake.Johan Tibellhttps://www.blogger.com/profile/06875432206357419172noreply@blogger.comtag:blogger.com,1999:blog-927289640963145319.post-41811515822422003722012-03-14T14:11:51.551-07:002012-03-14T14:11:51.551-07:00Actually cabal won't build my library, because...Actually cabal won't build my library, because it involves both haskell and C++. The dependencies go across languages, e.g. hsc2hs needs to be rerun when the depended upon headers change. And of course hs files that depend on cc files have to rebuild the cc files when necessary.<br /><br />I don't have a .a file, and I don't want to copy it somewhere, the .o files get linked into the binary at compile time, and again dynamically at runtime for plugins.Evan Laforgehttps://www.blogger.com/profile/08282226608319565537noreply@blogger.comtag:blogger.com,1999:blog-927289640963145319.post-55068671976119452312012-03-14T12:51:21.054-07:002012-03-14T12:51:21.054-07:00Chris,
we'll definitely need a way to specify...Chris,<br /><br />we'll definitely need a way to specify package flags for dependencies as well e.g. --flags=gloss-web="--ghc-options=-XTrustWorthy". Long term I'd like to move away from per package flags as much as possible.Johan Tibellhttps://www.blogger.com/profile/06875432206357419172noreply@blogger.comtag:blogger.com,1999:blog-927289640963145319.post-30545991151189337352012-03-14T12:49:25.571-07:002012-03-14T12:49:25.571-07:00Evan,
Note that cabal build will build your libra...Evan,<br /><br />Note that cabal build will build your library just fine, leaving you with a .a file you can link into your bigger project. Are you sure you really want to install that .a somewhere using Cabal?<br /><br />Tacking a step back: in my mind Cabal will never replace the distro's package manager and thus multi-language projects need to be built using the distro package manager's tools or some other tool suited for such a task.Johan Tibellhttps://www.blogger.com/profile/06875432206357419172noreply@blogger.comtag:blogger.com,1999:blog-927289640963145319.post-55355298075334121842012-03-14T12:08:02.770-07:002012-03-14T12:08:02.770-07:00Sounds great... but another problem to address is ...Sounds great... but another problem to address is the need to build some packages in specific ways. For example, for gloss-web, I need to build gloss with --with-ghc-options=-XTrustworthy. Currently, I just do that... but if you eliminate 'cabal install' for libraries and/or do the hermetic build thing, then I need a new mechanism to indicate that Cabal should make sure gloss is built with that option. Doing this right would be better than the status quo, but doing it wrong would mean actually making things worse.Chris Smithhttps://www.blogger.com/profile/10635500913510539345noreply@blogger.comtag:blogger.com,1999:blog-927289640963145319.post-63575263606799735512012-03-14T12:03:06.895-07:002012-03-14T12:03:06.895-07:00WRT I don't know what cabal install means:
An...WRT I don't know what cabal install means:<br /><br />Any multi-language program is not going to be buildable with cabal. Also, in the case of mine, the whole point is to build a library *and* an executable. So as far as I can see, I'll always need 'cabal install lib' to pull in the necessary haskell dependencies.Evan Laforgehttps://www.blogger.com/profile/08282226608319565537noreply@blogger.comtag:blogger.com,1999:blog-927289640963145319.post-13883374032860420912012-03-14T11:55:09.419-07:002012-03-14T11:55:09.419-07:00Upvote!
I'd be up for helping out this project...Upvote!<br />I'd be up for helping out this project, though I don't know if I have enough slack in my work stack to actually be able to follow through at the moment. But I absolutely agree that cabal should evolve to being more of a full "proper" build tool / package managercarter t schonwaldhttps://www.blogger.com/profile/16546056304179288828noreply@blogger.com