Wednesday, August 25, 2010

Haskell in the Real World discussion at CUFP

Bryan O'Sullivan and I are organizing an informal discussion about Haskell in the Real World at this year's CUFP. The discussion will take place in the evening on Thursday 30 September.

Topic: where we as a community should be focusing our efforts over the coming year to continue to improve Haskell as a practical language for day-to-day hacking.

Subjects we expect to cover would include at least the following:

  • post-mortem crash analysis and debugging
  • improvements to the Cabal build system (e.g. parallel builds)
  • performance and documentation improvements for core libraries
  • plugging missing functionality holes in the family of packages on hackage

Tuesday, August 10, 2010

New I/O manager merged into the main GHC tree

Today Simon Marlow merged Bryan O'Sullivan and my I/O manager patches into the main GHC tree. All the tests pass, and it’s performing well.

The new I/O manager is primarily useful in high-performance server applications, where the server maintains many open connections simultaneously and only a small number of those connections are active at any given time. This is the common case for most web applications. As a bonus, timeouts are also handled more efficiently: you can have many more threads calling e.g. threadDelay simultaneously than before.

You don’t need to do change any code to see benefit from the new I/O manager: any code that uses forkIO and runs on a system that supports epoll or kqueue should scale better in terms of the number of open file descriptors.

The limit on number of open file descriptors is also gone: you can now have more than 1024 open file descriptors.

If you find any bugs, please use the GHC bug tracker to report them to me, “tibbe”.

Monday, August 9, 2010

Results from the State of Haskell, 2010 Survey

One week ago, I sent out a survey where I asked Haskell developers a few quick questions about their background and Haskell usage. Like the State of Clojure survey, I was trying to figure out

  • where Haskell programmers "come from",
  • in which domains Haskell is being used,
  • to what extent Haskell is used commercially, and
  • what the major weakness in the Haskell ecosystem are.

I got 804 responses, which is more than I expected. Since the survey was sent out to the main Haskell mailing lists, posted on my blog, and announced on Twitter there's most likely some selection bias in the results. However, given the number of replies, I still think the results are indicative of the community as a whole.

Without further ado, here are the results, together with some comments by me. You can find a link to the raw data at the end of this post.

How long have you been using Haskell?

The Haskell community continues to grow. Almost 75% joined the last three years.

How would you characterize your use of Haskell today?

Note that respondents could choose more than one option here, so results add up to more than 100%.

  • Lots of people use Haskell in their spare time for open source work.
  • Lots of people, like me, first discover Haskell in school.
  • I'm glad to see that as many as 25% are able to use Haskell at work, to some extent.
  • Many people are also currently tinkering with Haskell, perhaps looking for some "real" project to try it out on.

What is the status of Haskell in your workplace?

Note that only people who work at the moment responded to this question so the results differ a bit from the previous question. It would be interesting to know why little more than 50% of Haskell programmers don't use Haskell at work. Is it because of a company policy or isn't Haskell a good fit for the problem domain?

In which domain(s) are you using Haskell?

Note that respondents could choose more than one option here, so results add up to more than 100%.

There's been quite some buzz around Haskell web development on the mailing lists and on reddit lately, with several web frameworks being announced in a short time frame. I'd really like to see Haskell get better adoption in this area. Web development is still very immature: programmers create HTML documents by concatenating strings (using templating languages or not), leading to cut-n-paste programming and security problems. I think a web framework that integrates formlets could simplify much of the tedious and error prone work in web development today.

Compilers and interpreters is another big item, probably due to Haskell's use in academia but also due to Haskell being a very good language for manipulating algebraic data types, which are useful in compiler development. Since language attracts quite "hardcore" language geeks (in the positive sense), this is not much of a surprise.

I had a quick glance at the results categorized as "Others". Lots of different things in there but I don't think I've missed any major domains.

I would like to see more good libraries for (distributed) "Big Data" processing. The functional model works really well for such tasks, as shown by Google's MapReduce framework.

Which environment(s) do you use to work with Haskell?

Note that respondents could choose more than one option here, so results add up to more than 100%.

The typical Haskell programmers use a combination of a powerful text editor and a command line REPL (e.g. GHCi). Leksah is the biggest IDE at 6%. Haskell could probably gain increased adoption if it had e.g. a very good Eclipse plugin, as many programmers use that kind of environment today.

What language did you use just prior to adopting Haskell – or, if Haskell is not your primary language now, what is that primary language?

200

I'm only showing options that garnered more than 1% of responses; this was not a multiple-choice question. I might change it to be a multi-choice question next time as many people said they couldn't pick one language. The reason I tried to stick with one language is that I wanted people to try to pick one instead of listing all the languages they knew.

Lots of people come from the C, C++, and Java which is not that surprising given the sheer number of programmers that use those languages. If you look the Clojure survey very few people come to Clojure from C and C++. Perhaps more C and C++ programmers come to Haskell instead of Clojure because Haskell is a statically typed, compiled language with pretty good performance.

A large number of people have come over from Python. Perhaps due to that community having a large number of early adopters who are willing to try new languages.

If Haskell disappeared tomorrow, what language(s) might you use as a "replacement"?

Note that respondents could choose more than one option here, so results add up to more than 100%.

Again, I'm only showing options that garnered more than 1% of responses.

  • Perhaps unsurprisingly, the top 4 languages are all functional.
  • 29% of people in the Clojure community would move to Haskell if Clojure disappeared, according to the State of Clojure survey. 25% of Haskell programmers would move to Clojure of Haskell disappeared.

What do you think is Haskell's most glaring weakness / blind spot / problem?

There are a lot of responses to this question. I suggest you read the raw data for the full picture. Here's a summary:

  • Libraries, libraries, libraries: There are lots of libraries to choose from, but many users want better quality, better documentation and better cohesion. The Haskell Platform will hopefully address the third point eventually, but right now it mostly contains libraries adopted from the GHC extra libraries bundle and it doesn't feel like a tight knit set of core libraries. Perhaps the Haskell Platform should have its own documentation page with API docs and tutorials?
  • Reasoning about performance: Many programmers find it hard to reason about the performance of their programs, primarily due to laziness. We need more tutorials, books, and tools to help Haskell programmers reason about program performance. I'm giving a tutorial on high-performance Haskell at CUFP this year that I hope will be videotaped. Perhaps it's time for a high-performance Haskell book?
  • Development environments: Several people requested integration into widely used IDEs such as XCode, Visual Studio, and Eclipse.
  • A number of people have problem with package breakages. Better quality control and automatic continuous builds could help here.
  • A few people mentioned the learning curve, both in a positive and negative sense.

General Comments?

Lots of positive, encouraging comments. I've listed some of them below. Again, read the raw data for a complete picture.

  • "Excited and optimistic about the language, its future, and what appears to be a great community."
  • "I love Haskell. It's an endless source of knowledge and inspiration."
  • "Haskell is an amazing language. I don't know what I would do without it. Despite the list of problems above it remains tremendously useful. I write terse, readable programs that are frequently correct on the first try. It's unparalleled. Every advance I have made in understanding it thus far has had a substantial payoff."
  • "The Haskell community is very friendly and welcoming, and that makes a BIG difference."
  • "I am optimistic about Haskell, it is moving quickly in the right direction."
  • "I simply assume Haskell to be an important language (technology, really) in the future. However proud I am of being a Haskell'er, I don't see a time close where I'll be mentioning this in job interviews."
  • "main = mapM_ putStrLn $ words $ map Char.chr [73,32,108,111,118,101,32,72,97,115,107,101,108,108]"
  • "Haskell rocks my world!"

Lots of people thanked me for putting together the survey, so let me end with thanking all of you for taking your time to take it!

Raw data

All the source data is available in a spreadsheet in Google Docs or as an HTML table export from that spreadsheet.

I'll try to do a follow-up next year.

Monday, August 2, 2010

State of Haskell, 2010 survey

Inspired by the State of Clojure survey by Chas Emerick I’ve created a quick, 9-question State of Haskell survey, using Google Spreadsheets. The survey will hopefully give us some insight into how people use Haskell and perhaps also some ideas on how Haskell tools and libraries could be improved.

The results from this survey are now available.