Planet Bazaar

January 31, 2017

Jonathan Lange

Announcing haskell-cli-template

Last October, I announced servant-template, a cookiecutter template for creating production-ready Haskell web services.

Almost immediately after making it, I wished I had something for building command-line tools quickly. I know stack comes with a heap of them, but:

  • it’s hard to predict what they’ll do
  • adding a new template requires submitting a PR
  • cookiecutter has existed for ages and is pretty much better in every way

So I made haskell-cli-template. It’s very simple, it just makes a Haskell command-line project with some tests, command-line parsing, and a CircleCI build.

I wanted to integrate logging-effect, but after a few months away from it my tired little brain wasn’t able to figure it out. I like command-line tools with logging controls, so I suspect I’ll add it again in the future.

Let me know if you use haskell-cli-template to make anything cool, and please feel free to fork and extend.

by Jonathan Lange at January 31, 2017 12:00 AM

January 30, 2017

Jonathan Lange

Announcing graphql-api: Haskell library for GraphQL

Late last year, my friend Tom tried to convince me that writing REST APIs was boring and repetitive and that I should give this thing called GraphQL a try.

I was initially sceptical. servant, the REST library that I’m most familiar with, is lovely. Its clever use of Haskell’s type system means that all the boring boilerplate I’d have to write in other languages just goes away.

However, after watching Lee Byron’s Strange Loop talk on GraphQL I began to see his point. Being able to get many resources with the same request is very useful, and as someone who writes & runs servers, I very much want clients to ask only for the data that they need.

The only problem is that there isn’t really a way to write GraphQL servers in Haskell—until now.

Introducing graphql-api

Tom and I put together a proof-of-concept GraphQL server implementation called graphql-api, which we released to Hackage today.

It lets you take a GraphQL schema and translate it into a Haskell type that represents the schema. You can then write handlers that accept and return native Haskell types. graphql-api will take care of parsing and validating your user queries, and Haskell’s type system will make sure that your handlers handle the right thing.

Using graphql-api

Say you have a simple GraphQL schema, like this:

type Hello {
  greeting(who: String!): String!
}

which defines a single top-level type Hello that contains a single field, greeting, that takes a single, required argument who.

A user would query it with something like this:

{
  greeting("World")
}

And expect to see an answer like:

{
  "data": {
    "greeting": "Hello World!"
  }
}

To do this in Haskell with GraphQL, first we’d define the type:

type Hello = Object "Hello" '[]
  '[ Argument "who" Text :> Field "greeting" Text ]

And then a handler for that type:

hello :: Handler IO Hello
hello = pure greeting
 where
   greeting who = pure ("Hello " <> who <> "!")

We can then run a query like so:

queryHello :: IO Response
queryHello = interpretAnonymousQuery @Hello hello "{ greeting(who: \"World\") }"

And get the output we expect.

There’s a lot going on in this example, so I encourage you to check out our tutorial to get the full story.

graphql-api’s future

Tom and I put graphql-api together over a couple of months in our spare time because we wanted to actually use it. However, as we dug deeper into the implementation, we found we really enjoyed it and want to make a library that’s genuinely good and helps other people do cool stuff.

The only way to do that, however, is to release it and get feedback from our users, and that’s what we’ve done. So please use graphql-api and tell us what you think. If you build something cool with it, let us know.

For our part, we want to improve the error messages, make sure our handling for recursive data types is spot on, and smooth down a few rough edges.

Thanks

Tom and I want to thank J. Daniel Navarro for his great GraphQL parser and encoder, which forms the basis for what we built here.

About the implementation

graphql-api is more-or-less a GraphQL compiler hooked up to type-based executing (aka “resolving”) engine that’s heavily inspired by Servant and uses various arcane type tricks from GHC 8.

We tried to stick to implementing the GraphQL specification. The spec is very well written, but implementing it has taught us that GraphQL is not at all as simple as it looks at first.

I can’t speak very well to the type chicanery, except to point you at the code and at the Servant paper.

The compiler mostly lives in the GraphQL.Internal.Validation module. The main idea is that it takes the AST and translates it into a value that cannot possibly be wrong.

All the syntax stuff is from the original graphql-haskell, with a few fixes, and a tweak to guarantee that Name values are guaranteed to be correct.

by Jonathan Lange at January 30, 2017 12:00 AM

January 12, 2017

Jonathan Lange

Announcing grafanalib

Late last year, as part of my work at Weaveworks, I published grafanalib, a Python DSL for building Grafana dashboards.

We use it a lot, and it’s made our dashboards much nicer to maintain. I’ve written a blog post about it that you can find it on the Weaveworks blog.

by Jonathan Lange at January 12, 2017 12:00 AM

December 15, 2016

Elliot Murphy

Dec 15, 2016

Another morning run, and more dolphin. Wish there was a way to identify them. Got a picture of the sailboat named floating point. Can't remember the name of this typeface.

by Elliot Murphy at December 15, 2016 05:47 PM

Dec 14, 2016

There was some extra time before Darcy had to leave for the day, so we jumped in the boat a few minutes after sunrise. Best day yet for dolphin viewing, several dolphins were chasing fish toward us in the intersection of several canals. Darcy got some great photos of the dolphins playing, and observed how great it is to be able to enjoy your food.

by Elliot Murphy at December 15, 2016 05:46 PM

December 14, 2016

Elliot Murphy

Dec 13, 2016

Dec 13, 2016

Notified a client about an impeding outage that I noticed during an audit today, and did some studying on the Elm language. After the rainstorm finished I took a break from studying and hopped in the boat. Full moon, and it was bright enough to read. The river was a black mirror. Dolphins were out hunting fish with barely a ripple. Saw the largest shooting star I've ever seen, and the clouds over the barrier island in the Atlantic were full of heat lightning.

by Elliot Murphy at December 14, 2016 08:49 PM

December 12, 2016

Elliot Murphy

Dec 12, 2106

Headed out shortly after sunup today. Water was absolutely glassy, the winds from the weekend were gone. Dodged half a dozen large coconuts as we headed out.

Went past Sampson island into the main river because it was so calm. After clearing the mangroves could see straight to the bottom.

On the way back in the water was so smooth you could see a distinct V ahead on the surface as if some invisible boat. I guessed more dolphins, and sure enough there were two more herding fish back into the docks, lazily circling and diving.

by Elliot Murphy at December 12, 2016 02:42 PM

Dec 11, 2016

Another late night run after spending a fun afternoon at the zoo. Headed to the end of the canal and was delighted by a couple of Atlantic bottlenose dolphins chasing fish.

by Elliot Murphy at December 12, 2016 02:37 PM

Dec 8, 2016

Tonight I headed through the west canal,and it turned out to run the length of the next island. Saw a couple of amusing boat names. A ketch named "floating point" with a legacy computer typeface, and another sailboat named "satellite office".

by Elliot Murphy at December 12, 2016 02:36 PM

Dec 10, 2016

Woke up with energy today, and dressed quickly for the day. Even though I had to wear a suit for today, decided to risk a quick inspection of the island during the early morning light to assess whether we could bring anyone over. Heading back I saw Darcy sitting outside enjoying her morning coffee substitute, and I can't think of a nicer sight.

In the afternoon most of the family took naps, and then I pleaded with everyone to go for a quick island visit before dusk. We all piled on, including the two dogs. Some dogs on the canal bank started barking at some ducks, and daisy hopped in the water to join in. Cut the motor, the boat went over the dog, everyone stayed calm and we got turned around. Daisy was paddling in circles and we made it over and got her back in the boat.

Arriving at the island everyone enjoyed walking around on the trails and seeing the mangroves and pines and campsites. Daisy dried off by rolling in the white sand.

by Elliot Murphy at December 12, 2016 02:28 PM

Dec 9, 2016

Quite windy today, 12 to 15 mph gusts on the river. Small craft advisory and whitecaps. Headed out to the main channel around midnight to see how the Boston whaler skiff handled compared to our old proline with higher freeboard. Waves were 2-3 feet and the skiff definitely couldn't take anything bigger. Little boat on a following swell is always a thrill as it goosenecks and swerves sliding into the trough and you look directly into the side of the waves. Wish I could see some flying fish.

by Elliot Murphy at December 12, 2016 02:18 PM

Dec 7, 2016

Late at night before I made it out in the boat. Quiet, half moon and cloudy. The river was absolutely slippery with fish. Headed south, and saw the bioluminescence was stronger. Killed the lights and watched streaks of light through the water all around the boat as the schools darted and churned.

by Elliot Murphy at December 12, 2016 02:13 PM

December 07, 2016

Elliot Murphy

Dec 6, 2016

Intermittent heavy rains today. Vey glad I bought the hand operated bilge pump, handy to empty the bilge twice during the day. After we returned in the evening I finished provisioning the ubiqity network gear for our mobile network and went out in the boat just before midnight. Then moon was just setting, and the mangroves were still dripping from rain. Headed past the nature preserve into the river, shut off the stern light and just looked at the stars. Stirring a stick in the water and still captivated by the bioluminescence.

by Elliot Murphy at December 07, 2016 05:26 AM

December 06, 2016

Elliot Murphy

Dec 5, 2016

5 day streak. Tonight after getting home everyone agreed to go with me, and we set out on the boat again north on the great canal. Incredibly calm night with a bright moon but many clouds. All the boat lights are so pretty and we saw several docks with underwater lights. One even had a heron standing near the light fishing.

by Elliot Murphy at December 06, 2016 06:23 AM

Dec 4, 2016

Woke up at 6AM and recorded a time lapse of the sunrise at the beach. Then rushed back and headed out in the boat. Rushed too much and realized I left the keys behind after casting off. Tried to lasso a cleat and missed as the wind caught the bimini top. Happily there was an emergency paddle in the anchor locker, so 15 minutes of paddling later I made it back to the dock and headed out to our Sunday appointments. After lunch tried another boat run with rhaiza. Brought the dogs and headed north. Many folks putting lights on their boats. After passing under the bridge one of the dogs got lost their footing and went into the water. Showed rhaiza the overboard procedure, threw out a floating cushion, and got the dog back aboard none the worse for wear.

by Elliot Murphy at December 06, 2016 06:20 AM

December 04, 2016

Elliot Murphy

Dec 3rd, 2016

Last night we found some battery powered clamp on nav lights, tonight after grilling dinner for guests I got the lights rigged and took a short trip North on the grand canal. Many docks have winter lights up, and plenty of sailboats home for the winter. Lights worked well and nothing to worry about in the canal. Much work to do but this trip is helping my mind so much, I feel stronger than I have in years.

by Elliot Murphy at December 04, 2016 04:15 AM

December 02, 2016

Elliot Murphy

Dec 2, 2016

Boat didn't take on water overnight. Shortly after sunrise made some pour-over coffee and was about to head out, then thought it was worth inviting others. Rhaiza and Darcy were both up for a quick cruise. Got the bimini top installed, and drank coffee while looking around the nature preserve. Dogs loved the boat. nice 30 minute cruise before the sun started beating down.

by Elliot Murphy at December 02, 2016 05:28 PM

Dec 1, 2016

Arrived at rental house. Picked up boat from warehouse, helped the owner replace the windshield. Drove out to the causeway, launched, found my way through the canals to the house. Got a ride back to the ramp to recover the truck and trailer. Went to Wal-Mart and bought throwable life preserver cushions and portable navigation lights. Couldn't find a bilge pump so resigned to using a cup to bail as needed.

by Elliot Murphy at December 02, 2016 05:25 PM

October 17, 2016

Mark Shuttleworth

The mouse that jumped

The naming of Ubuntu releases is, of course, purely metaphorical. We are a diverse community of communities – we are an assembly of people interested in widely different things (desktops, devices, clouds and servers) from widely different backgrounds (hello, world) and with widely different skills (from docs to design to development, and those are just the d’s).

As we come to the end of the alphabet, I want to thank everyone who makes this fun. Your passion and focus and intellect, and occasionally your sharp differences, all make it a privilege to be part of this body incorporate.

Right now, Ubuntu is moving even faster to the centre of the cloud and edge operations. From AWS to the zaniest new devices, Ubuntu helps people get things done faster, cleaner, and more efficiently, thanks to you. From the launch of our kubernetes charms which make it very easy to operate k8s everywhere, to the fun people seem to be having with snaps at snapcraft.io for shipping bits from cloud to top of rack to distant devices, we love the pace of change and we change the face of love.

We are a tiny band in a market of giants, but our focus on delivering free software freely together with enterprise support, services and solutions appears to be opening doors, and minds, everywhere. So, in honour of the valiantly tiny leaping long-tailed over the obstacles of life, our next release which will be Ubuntu 17.04, is hereby code named the ‘Zesty Zapus’.

 

by Mark Shuttleworth at October 17, 2016 12:23 PM

October 15, 2016

Jonathan Lange

servant-template: production-ready Haskell web services in 5 minutes

If you want to write a web API in Haskell, then you should start by using my new cookiecutter template at https://github.com/jml/servant-template. It’ll get you a production-ready web service in 5 minutes or less.

Whenever you start any new web service and you actually care about getting it working and available to users, it’s very useful to have:

  • logging
  • monitoring
  • continuous integration
  • tests
  • deployment
  • command-line parsing

These are largely boring, but nearly essential. Logs and monitoring give you visibility into the code’s behaviour in production, tests and continuous integration help you make sure you don’t break it, and, of course, you need some way of actually shipping code to users. As an engineer who cares deeply about running code in production, these are pretty much the bare minimum for me to be able to deploy something to my users.

The cookiecutter template at gh:jml/servant-template creates a simple Haskell web API service that does all of these things:

As the name suggests, all of this enables writing a servant server. Servant lets you declaring web APIs at the type-level and then using those API specifications to write servers. It’s hard to overstate just how useful it is for writing RESTful APIs.

Get started with:

$ cookiecutter gh:jml/servant-template
project_name [awesome-service]: awesome-service
...
$ cd awesome-service
$ stack test
$ make image
...
sha256:30e4c9a5f29a2c4caa44e226859dd094c6ac9d297de0d1d2024e8a981a7c8f86
awesome-service:unversioned
$ docker run awesome-service:latest --help
awesome-service - TODO fill this in

Usage: awesome-service --port PORT [--access-logs ARG] [--log-level ARG]
                       [--ghc-metrics]
  One line description of project

Available options:
  -h,--help                Show this help text
  --port PORT              Port to listen on
  --access-logs ARG        How to log HTTP access
  --log-level ARG          Minimum severity for log messages
  --ghc-metrics            Export GHC metrics. Requires running with +RTS.
$ docker run -p 8080:80 awesome-service --port 80
[2016-10-16T20:50:07.983292987000] [Informational] Listening on :80

For this to work, you’ll need to have Docker installed on your system. I’ve tested it on my Mac with Docker Machine, but haven’t yet with Linux.

You might have to run stack docker pull before make image, if you haven’t already used stack to build things from within Docker.

Once it’s up and running, you can browse to http://localhost:8080/ (or http://$(docker-machine ip):8080/) if you’re on a Mac, and you’ll see a simple HTML page describing the API and giving you a link to the /metrics page, which is where all the Prometheus metrics are exported.

There you have it, a production-ready web service. At least for some values of “production-ready”.

Of course, the API it offers is really simple. You can make it your own by editing the API definition and the server implementation to make it really your own. Note these two are in separate libraries to make it easier to generate client code.

The template comes with a test suite that uses servant-quickcheck to guarantee that none of your endpoints return 500s, take longer than 100ms to serve, and that all the 201s include Location headers.

If you’re so inclined, you could push the created Docker image to a repository somewhere—it’s around 25MB when built. Then, people could use it and no one would have to know that it’s Haskell, they’d just notice a fast web service that works.

As the README says, I’ve made a few questionable decisions when building this. If you disagree, or think I could have done anything better I’d love to know. If you use this to build something cool, or even something silly, please let me know on Twitter.

by jml at October 15, 2016 11:00 PM