gampleman

[gampleman]

Design your own Runtime

The Elm Architecture is one of the great innovations Elm brought to the software industry. But one of the questions that often comes up is how to scale it up for bigger applications. In this article I want to show various ideas that allow you to enhance the Elm architecture to make developing larger applications easier.

But first, we should discuss the Runtime. Elm, as a programming language is completely pure (with some mostly insignificant exceptions). To make it useful, we pair it with the Elm runtime, that performs effects on our behalf and calls our code with the results.

We set this relationship up when we define main, by calling (typically) a function from elm/browser. These functions that elm/browser exposes are our hooks into the runtime, telling it what functions to call when, and from which functions to accept instructions as to which effects to perform.

However, these runtimes aren't all magic. In fact they form a hierarchy where each successive version can be implemented on top of the previous runtime. The most basic one we have access to is Platform.worker:

worker : { init : flags -> ( model, Cmd msg ) , update : msg -> model -> ( model, Cmd msg ) , subscriptions : model -> Sub msg } -> Program flags model msg

The first thing you'll notice is that there is no view function! But this function gives us the basics we need:

it tells the runtime how to initialise the main loop

it tells the runtime how to update the loop

it tells the runtime where to execute effects from and how to receive input from outside

Now imagine that Elm gave us the following function renderHTML : Html msg -> Cmd msg. We can now make our next runtime in the hierarchy, Browser.element:

renderAfterUpdate : (model -> Html msg) -> (model, Cmd msg) -> (model, Cmd msg) renderAfterUpdate view (model, cmds) = ( model, Cmd.batch [ renderHtml (view model), cmds ] ) element : { init : flags -> ( model, Cmd msg ) , view : model -> Html msg , update : msg -> model -> ( model, Cmd msg ) , subscriptions : model -> Sub msg } -> Program flags model msg element config = Platform.worker { init = config.init >> renderAfterUpdate config.view , update = \msg model -> config.update msg model |> renderAfterUpdate config.view , subscriptions = config.subscriptions }

(To see how you would actually build this using ports if you didn't have the above mentioned magical renderHTML function, I have written an article just about that.)

In a similar manner you can then build all the other functions in the Browser module, slowly layering on more functionality, that declaratively makes the applications more powerful. In the rest of this article, we are going to explore some ideas how you can also layer on more functionality and give your application superpowers.

Effect Types

I mentioned previously that your pure Elm program instructs the runtime which (side) effects you would like it to run. This is achieved through the Cmd type. However, the Cmd type has a few features that can make it challenging to use in some situations:

It is opaque, so there is no way to see inside it.

It can contain functions, so it can't even be reliably =='d.

Effect Types circumvent these problems, and involve creating a custom Effect type, that enumerates all the possible effects that your application can perform. Then you change the signature of your update functions to update : Msg -> Model -> (Model, Effect Msg). Finally, you make a wrapper that actually evaluates these effects by turning them into Cmds.

This has the following benefits:

Testing your application either as a whole or even just testing individual update functions becomes straightforward, as all you need to do is provide a fake implementation of the function that runs effects. You can check out avh4/elm-program-test for some testing goodness based on this idea.

You can treat things as effects that aren't. For example notifications are typically much easier to model as an effect that is managed by the runtime, even if the implementation is just some model field that get's modified and a regular Elm view. Likewise there can be top-level state (we'll talk about this more a bit later), that components might want to modify as an effect (for instance switching a colour theme).

You can describe effects at the right level of abstraction. For instance sometimes request/response pairs of ports are unavoidable, but you can't implement them as a Cmd Msg (meaning you can't make it a Cmd that gives you a Msg back, you have to manually manage the corresponding subscription). However, nothing is stopping you implementing an Effect that stores a "request" in the model, ads the corresponding subscriptions, handles matching request/response semantics by id, timeouts and error handling, and packages all this as a single Msg back to the caller.

In a similar manner, you can also do things like if an effect consists of making several HTTP requests in parallel, the machinery for that can also be abstracted away.

You can make your effects composable by implementing batch and map, or skip that if you don't want it. There is a lot of flexibility in the approach.

Declarative Effects

As mentioned above, Elm treats some effects specially by not using Cmds, most notably HTML rendering. Instead a declarative abstraction is offered, which runs a function of the model, then compares the current result with the previous result, and runs effects based on the diff.

In fact you could think even of subscriptions as implemented like that:

worker : { init : flags -> ( model, Cmd msg ) , update : msg -> model -> ( model, Cmd msg ) , subscriptions : model -> Sub msg } -> Program flags ({model : model, subs: List (Sub msg)}) msg worker config = primitiveProgram { init = config.init >> Tuple.mapFirst (\m -> {model = m, subs = []}) update = \msg model -> let newSubs = config.subscriptions model.model (added, removed) = diffSubs model.subs newSubs (newModel, cmds) = config.update msg model.model in ( { model = newModel, subs = newSubs } , Cmd.batch [ cmds , List.map addEventListener added |> Cmd.batch , List.map removeEventListener removed |> Cmd.batch ] ) }

The code above features a bunch of imaginary primitives and indeed all of this is actually implemented in Kernel code, but I think it illustrates how one might think of building declarative effects in their own app.

Some use-cases that I've seen is for example a function that renders web audio, but perhaps this could also be used for declaratively changing the URL for routing, or non-HTML based rendering (a PDF renderer?).

Automatic effects

Evan Czaplicki's TodoMVC implementation also sort of includes a custom runtime, and this does a simple thing: after anything happens, it serialises the model and saves it to localStorage. Similarly, your runtime can automatically do things, and your code doesn't have to worry about it.

Context

Another useful addition you can do (often in combination) with effects is providing your functions read only values from the runtime. For instance, it can be powerful to know what the window dimensions are in the view, or what the name of the current user is in the update, without having to write model logic to deal with these things is great.

view : Context -> Model -> Html Msg update : Context -> Msg -> Model -> (Model, Effect Msg) -- etc for init, subscriptions

You can of course provide Effects to allow the app to modify these values.

(Master) View Types

The standard Elm runtime mandates that you return Html from your views. However, Html is a format designed for writing scientific articles in the early nineties, and as such might not be the ideal thing to express highly dynamic applications in. Furthermore, to make HTML actually work practically, it has to be coupled to CSS, a language designed to separate the formatting of documents from the act of writing them. Thankfully, as elm-ui has demonstrated, we are not limited to the past.

You can design your own view language. This is particularly nice if you are already using a design system with predefined components and layout patterns. You can then translate these into a system of Elm modules that implement the system directly.

module View exposing (View, Attribute, Background, TextColor, Padding , render ) type View msg = Box { background : Background, padding : Padding} | Text {textColor: TextColor, text : String} -- ... | EscapeHatch (Html msg) type Background = Level1 | Level2 type TextColor = Primary | Secondary | Disabled | Active render : View msg -> Html msg

and so on.

Doing this has some interesting benefits. First of all, you can ensure that your teams produces highly cohesive interfaces, since the system either nudges or enforces this (depending if you provide an escape hatch).

Secondly, testing becomes more high level, since your views are now a custom type that you can write matching functions for trivially, and their semantics are more obvious (i.e. a tabbed interface will be a Tab constructor, rather than a see of divs with some arbitrary classes applies to them).

Taking this further

Once your views are defined in your own types, you can start building a lot of interesting capabilities on top of these. One experiment that I wrote allowed you to include local state for these view types.

The trick to doing that is to actually call the users view function in update, then traverse the resulting tree and match it up to the component state. Then you run each components update function on its own state and save the resulting tree in the model. When it comes time to calling view, you simply transform this resulting tree into Html. Most of the code is simply bookkeeping to match the view tree to the state tree. Is this is a good idea? I think a lot more testing would be needed to find out.

Another useful thing that is easy to implement is extending the context idea into the view:

withDeviceDimensions : ({ width : Int, height : Int } -> View msg) -> View msg -- view = row [ ] [ withDeviceDimensions (\{width, height} -> svgDrawing [ text_ [ x (width / 2) ] [ text "Hello from the middle of the screen!" ] ] )]

This can be particularly helpful for localisation, as it frees you from needing to pass the locale all over the place.

I would like to also see if these tricks could be nicely combined with animation, so we can have neat enter and leaving animations without giving up the Virtual DOM style of programming.

URL Coders

I have to admit that one of my least favourite core Elm apis is Browser.application. Even after years of using it, I still find the combination of onUrlRequest and onUrlChange and how they interact with pushUrl confusing. Every time I need to use a Url.Parser, I stare at its type signatures with puzzlement. Perhaps frontend routing is confusing simply by its nature.

However, that doesn't mean we can't try fixing it. I've written a very rough sketch of an alternative API, that would be suitable for some kinds of apps. I foresee it most useful for apps that truly have a central model, like calculators, dashboards, maps, etc. On the other hand content based apps might want to stick with the builtin (or just with server side routing).

The main idea is to treat URLs as a encoded version of the model. And for encoding and decoding we have very nice tools in Elm: encoders and decoders. Leonardo Taglialegne has extended this idea into that of Codecs, which combine encoders and decoders into a single structure, that (mostly) guarantees that they are mutually compatible. So the idea is to build a codec for URLs, but of course we typically don't want to store the whole model in the URL, hence we would call it a lossy encoding.

That means that the encoders would lose data from the model, and the decoders would need ways to recover a model (possibly not the same one). This recovery can either simply be providing a default value, or it can be a side-effect like reading something from the environment (i.e. flags or Browser.getViewport) or recovering the whole value by fetching the data from a server.

This would allow to get rid of init, onUrlRequest, onUrlChange, pushUrl, replaceUrl and Url.Parser at the cost of providing a single URLCodec, that unifies all those things.

Admittedly this is the least fleshed out idea out of all of the above, but the point is to encourage you to find better APIs that fit the app you want to write.

Make it your own

This wasn't meant as a list of things you need to do, but more a menu of delicius dishes that you can combine into making your application code clean and expressing the ideas that it represents rather than a lot of mechanical details mixed in with the business logic. Feel free to discuss this article on the Elm slack.

[gampleman]

Implementing VDOM in Elm

What if Elm didn't ship with a VDOM library? No HTML or SVG? You might be tempted that it wouldn't be very useful, but we can implement it ourselves using pure elm and ports. Ports? You heard right, good old trusty ports.

Simplest thing possible

So how can we render stuff? Well let's make a simple program to test:

module Main exposing (main) import MyHtml exposing (Html, program, div, text, href, a) import Time type Msg = Tick Float update : Msg -> Int -> (Int, Cmd Msg) update msg model = case msg of Tick t -> ( t, Cmd.none ) view : Int -> Html Msg view model = div [] [ text (model |> toString) , a [href "https://google.com"] [ text "GOOGLE IT"] ] main = program { init = ( 0, Cmd.none ) , update = update , subscriptions = \model -> Time.every Time.second Tick , view = view }

Looks pretty standard except for the funny import. Well, what does that look like?

port module MyHtml exposing (program, Html, Attribute, div, text, a, href) type Html msg = Node String (List (Attribute msg)) (List (Html msg)) | Text String type Attribute msg = Attr String String | Property String Json.Value

First, we defined some types to represent our HTML tree. Note that this is by no means complete, but can be easily extended to support things like SVG.

init : ( model, Cmd msg ) -> (model -> Html msg) -> ( model, Cmd msg ) init userInit userView = let ( initModel, initCmd ) = userInit in ( initModel, Cmd.batch [ render (userView initModel), initCmd ] ) update : (msg -> model -> ( model, Cmd msg )) -> (model -> Html msg) -> msg -> model -> ( model, Cmd msg ) update userUpdate userView msg model = let ( newModel, newCmd ) = userUpdate msg model in ( newModel, Cmd.batch [ render (userView newModel), newCmd ] ) program userProgram = Platform.program { init = init userProgram.init userProgram.view , update = update userProgram.update userProgram.view , subscriptions = userProgram.subscriptions }

Next let's define our program function. We're essentially wrapping the users program, but on both init and update we're adding an extra Cmd in to the mix:

render : Html msg -> Cmd msg render = encodeHtml >> renderPort port renderPort : Json.Value -> Cmd msg encodeHtml html = case html of Node name attrs children -> Json.object [ ( "type", Json.string "node" ) , ( "name", Json.string name ) , ( "attributes", Json.list (List.map encodeAttr attrs) ) , ( "children", Json.list (List.map encodeHtml children) ) ] Text str -> Json.object [ ( "type", Json.string "text" ), ( "value", Json.string str ) ] encodeAttr attr = case attr of Attr key val -> Json.object [ ( "type", Json.string "attribute" ), ( "key", Json.string key ), ( "value", Json.string val ) ] Property key val -> Json.object [ ( "type", Json.string "property" ), ( "key", Json.string key ), ( "value", val ) ]

So here we have encoded our HTML structure into JSON and sent it over a port. I'll spare you from future encoders, but they all look follow this same pattern. Let's see what the JavaScript side looks like:

<div id="output"></div> <script> var app = Elm.Main.worker() app.ports.renderPort.subscribe(function(html) { const output = document.getElementById("output"); while (output.firstChild) { output.removeChild(output.firstChild); } render(html, output); }); function render(struct, out) { switch(struct.type) { case "node": var el = document.createElement(struct.name); struct.attributes.forEach(attr => { switch(attr.type) { case "attribute": return el.setAttribute(attr.key, attr.value); case "property": return el[attr.key] = attr.value; } }); out.appendChild(el); struct.children.forEach(child => render(child, el)); break; case "text": var el = document.createTextNode(struct.value); out.appendChild(el); break; } } </script>

So whenever the renderPort triggers, we remove all contents from our destination node and use DOM apis to build the whole structure. Note that in DOM apis there are two flavors of attribute - actual atttributes that need to be set with setAttribute and properties that have specialised setters.

Does it work? Check it out:

Supporting events

Alright, we managed to get something on the screen, but we want to build a real app and so we need to support events coming in. We'll need to modify our approach a bit.

Let's make a little counter app to demo this:

module Main exposing (main) import MyHtml exposing (Html, program, div, text, href, a, onClick) type Msg = Inc | Dec main = program { init = ( 0, Cmd.none ) , update = \msg model -> case msg of Inc -> ( model + 1, Cmd.none ) Dec -> ( model - 1, Cmd.none ) , subscriptions = \model -> Sub.none , view = \model -> div [] [ text (model |> toString) , a [ onClick Inc ] [ text "+" ] , a [ onClick Dec ] [ text "-" ] ]

Event handlers in Elm are Json.Decode.Decoders that transform JavaScript event objects into the users custom msg type. But Json.Decode.Decoders are really functions under the hood.

Now a bit of trouble presents itself. We can't encode JSON Decoders into Json.Value objects and send them over ports. So how can we set up event listeners?

To solve this, we need to build up a dispatcher, which we can store in our model. So let's make our own model, that will wrap the users model:

import Dict exposing (Dict) type alias PrivateModel model msg = { userModel : model , handlers : Dict String (Decoder msg) }

The handlers key is a dispatcher datastructure: it holds the decoders the user specified stored under string keys. But where do those come from? We'll use a simple trick and compute a path through the DOM tree as their key. So for example the Inc onClick handler above, would have the key path div.0.a:click.

Next, we'll introduce a new representation, which we'll call SafeHtml. This is exactly the same as Html msg, except it drops the type variable and inside the coresponding SafeAttribute stores these key strings instead of the decoders. Note that we will only expose Html msg to the user of our library, SafeHtml is an implementation detail. We also add a new constructor for creating event listeners:

type Node handler = Node String (List (NodeAttribute handler)) (List (Node handler)) | Text String type NodeAttribute handler = Attr String String | Property String Json.Value | Event String Options handler type alias Options = { preventDefault : Bool , stopPropagation : Bool } type alias Html msg = Node (Json.Decode.Decoder msg) type alias Attribute msg = NodeAttribute (Json.Decode.Decoder msg) type alias SafeHtml = Node String type alias SafeAttribute = NodeAttribute String

We also update our encoders to only support SafeHtml.

Now we have the key data modeling pieces in hand. We now need a function that will take a Html msg and give us back both a handlers dispatcher and a SafeHtml tree that we can use for rendering:

extractListeners : String -> Html msg -> ( Dict String (Decoder msg), SafeHtml ) extractListeners prefix html = case html of Node name attrs children -> let key = prefix ++ "." ++ name safeAttrs = List.map (makeAttrSafe key) attrs listeners = List.filterMap getListener attrs kids = List.indexedMap (\index -> extractListeners (key ++ "." ++ toString index)) children childListeners = List.foldr (\( a, _ ) b -> Dict.union a b) Dict.empty kids in ( List.foldr (\( k, fn ) d -> Dict.insert (key ++ ":" ++ k) fn d) childListeners listeners , Node name safeAttrs (List.map Tuple.second kids) ) Text s -> ( Dict.empty, Text s ) makeAttrSafe : String -> Attribute msg -> SafeAttribute makeAttrSafe prefix attr = case attr of Event key options tagger -> Event key options (prefix ++ ":" ++ key) Attr k v -> Attr k v Property k v -> Property k v getListener : Attribute msg -> Maybe ( String, Decoder msg ) getListener attr = case attr of Event key _ tagger -> Just ( key, tagger ) _ -> Nothing

Now let's build it and use it:

subscriptions : (model -> Sub msg) -> PrivateModel model msg -> Sub (Maybe msg) subscriptions userSubscribtions model = let eventDispatcher ( key, event ) = Dict.get key model.handlers |> Maybe.andThen (\decoder -> Json.Decode.decodeValue decoder event |> Result.toMaybe ) in Sub.batch [ eventPort eventDispatcher, Sub.map Just (userSubscribtions model.userModel) ] port eventPort : (( String, Json.Value ) -> msg) -> Sub msg init : ( model, Cmd msg ) -> (model -> Html msg) -> ( PrivateModel model msg, Cmd (Maybe msg) ) init userInit userView = let ( initModel, initCmd ) = userInit ( handlers, safeView ) = extractListeners "" (userView initModel) in ( { userModel = initModel , handlers = handlers } , Cmd.batch [ render safeView, Cmd.map Just initCmd ] ) update : (msg -> model -> ( model, Cmd msg )) -> (model -> Html msg) -> Maybe msg -> PrivateModel model msg -> ( PrivateModel model msg, Cmd (Maybe msg) ) update userUpdate view maybeMsg model = case maybeMsg of Just msg -> let ( newModel, newCmd ) = userUpdate msg model.userModel ( handlers, safeView ) = extractListeners "" (view newModel) in ( { userModel = newModel, handlers = handlers } , Cmd.batch [ render safeView, Cmd.map Just newCmd ] ) Nothing -> ( model, Cmd.none ) program userProgram = Platform.program { init = init userProgram.init userProgram.view , update = update userProgram.update userProgram.view , subscriptions = subscriptions userProgram.subscriptions }

To handle the posibility that our dispatcher somehow gets out of sync with the DOM, we have to wrap the users message in a maybe. In the subscriptions call, we use the dispatcher to transform an incoming message (which is in the form of a (keyPath, eventObject) tuple) into the message the user expects. We handle the maybe in the new update function by simply ignoring the Nothing case. In both update and init we now call our extractListeners function to build up the dispatcher data structure.

And here's the updated JavaScript:

function render(struct, out, port) { switch(struct.type) { case "node": var el = document.createElement(struct.name); struct.attributes.forEach(attr => { switch(attr.type) { case "attribute": return el.setAttribute(attr.key, attr.value); case "property": return el[attr.key] = attr.value; case "event": return el.addEventListener(attr.key, e => { port.send([attr.value, e]); if (attr.stopPropagation) { e.stopPropagation(); } if (attr.preventDefault) { e.preventDefault(); } }); } }); out.appendChild(el); struct.children.forEach(child => render(child, el, port)); return; case "text": var el = document.createTextNode(struct.value); out.appendChild(el); return; } } var app = Elm.Main.worker() app.ports.renderPort.subscribe(function(html) { const output = document.getElementById("output"); while (output.firstChild) { output.removeChild(output.firstChild); } render(html, output, app.ports.eventPort); });

And that's basically all it takes to get events coming back in:

Making it Virtual

You may have noticed that we are re-rendering the entire DOM tree on every update. This is both not efficient and also potentially wrong, as it will lose focus on input elements and the like. The solution is to diff a new rendered view with the current one. This would create a bunch of patches that can then be applied to the real DOM for efficient updates.

The nice thing is that all but the application of patches can be done in Elm. So let's get started.

First let's look at describing changes between SafeHtml structures.

type Change = Change Patch | At Int Change | Batch (List Change) type Patch = Redraw SafeHtml | Facts (List ( Bool, SafeAttribute )) | TextChange String | Remove | Insert SafeHtml encodeChange : Change -> Json.Value encodeChange change = ... encodePatch : Patch -> Json.Value encodePatch patch = ...

I've spared you the implementation of the decoders in the interest of brevity. They look just the same as the other decoders.

Now let's take a look at how we might apply these changes to the existing DOM:

function render(struct, port) { switch(struct.type) { case "node": var el = document.createElement(struct.name); applyFacts(struct.attributes, el, port) struct.children.forEach(child => el.appendChild(render(child, port))); return el; case "text": return document.createTextNode(struct.value); } } function applyChange(change, element, port) { switch(change.type) { case "change": return applyPatch(change.patch, element, port); case "at": return applyChange(change.change, element.childNodes[change.index], port); case "batch": return change.changes.forEach(c => applyChange(c, element, port)); } } function applyPatch(patch, out, port) { switch(patch.type) { case "facts": return applyFacts(patch.facts, out, port); case "text": out.nodeValue = patch.value; return; case "redraw": return out.parentNode.replaceChild(render(patch.value, port), out); case "insert": return out.appendChild(render(patch.value, port)); case "remove": return out.parentNode.removeChild(out); } } function applyFacts(facts, el, port) { facts.forEach(attr => { switch(attr.type) { case "attribute": return attr.value == null ? el.removeAttribute(attr.key) : el.setAttribute(attr.key, attr.value); case "property": if (attr.value == null) { delete el[attr.key]; return; } else { el[attr.key] = attr.value; return; } case "event": if (attr.value == null) { el.removeEventListener(attr.key, el[attr.value]); delete el[attr.value]; } else { const handler = e => { port.send([attr.value, e]); if (attr.stopPropagation) { e.stopPropagation(); } if (attr.preventDefault) { e.preventDefault(); } }; el.addEventListener(attr.key, handler); // store a reference to the function so we can remove the handler el['handler-' + attr.value] = handler; } } }); } var app = Elm.Main.worker(); app.ports.renderPort.subscribe(function(change) { const output = document.getElementById("output"); applyChange(change, output, app.ports.eventPort); });

This might seem like a lot of code, but it's fairly simple. The Change datastructure allows to find the node that should change (that's what the At constructor is for). Then we apply one of the 5 possible changes. Note that there are more possible mutations that could increase efficiency, like a reorder change, but we've done only these for simplicity. Facts is a term used in Elm's virtual dom to refer to attributes, properties and event listeners.

Ok, let's try to get something on the screen:

initialRender : SafeHtml -> Cmd (Maybe msg) initialRender = Insert >> Change >> encodeChange >> renderPort type alias PrivateModel model msg = { userModel : model , handlers : Dict String (Decoder msg) , view : SafeHtml } init : ( model, Cmd msg ) -> (model -> Html msg) -> ( PrivateModel model msg, Cmd (Maybe msg) ) init userInit userView = let ( initModel, initCmd ) = userInit ( handlers, safeView ) = extractListeners "" (userView initModel) in ( { userModel = initModel , handlers = handlers , view = safeView } , Cmd.batch [ initialRender safeView, Cmd.map Just initCmd ] )

Note that we store a reference to the current view. Let's do some diffing against it:

wrapAt : Int -> List Change -> List Change wrapAt i changes = case changes of [] -> [] list -> [ At i (batchIfNecessary changes) ] batchIfNecessary : List Change -> Change batchIfNecessary changes = case changes of [] -> -- This should never happen Batch [] x :: [] -> x list -> Batch list diff : SafeHtml -> SafeHtml -> List Change diff before after = if before == after then [] else case ( before, after ) of ( Text bstr, Text astr ) -> [ Change (TextChange astr) ] ( Node bName bAttrs bChildren, Node aName aAttrs aChildren ) -> if aName == bName then let attrsDiff = if aAttrs == bAttrs then [] else List.map2 diffAttrs bAttrs aAttrs |> List.concat |> Facts |> Change |> List.singleton childrenDiff = if bChildren == aChildren then [] else diffChildren 0 bChildren aChildren in [ batchIfNecessary (attrsDiff ++ childrenDiff) ] else [ Change (Redraw after) ] _ -> [ Change (Redraw after) ] diffAttrs : SafeAttribute -> SafeAttribute -> List ( Bool, SafeAttribute ) diffAttrs before after = if before == after then [] else [ ( False, before ), ( True, after ) ] diffChildren : List SafeHtml -> List SafeHtml -> List Change diffChildren index before after = case ( before, after ) of ( [], [] ) -> [] ( b :: efore, [] ) -> At index (Change Remove) :: diffChildren (index + 1) efore after ( [], a :: fter ) -> Change (Insert a) :: diffChildren (index + 1) before fter ( b :: efore, a :: fter ) -> case diff b a of [] -> diffChildren (index + 1) efore fter diffs -> At index (batchIfNecessary diffs) :: diffChildren (index + 1) efore fter

Again a fair amount of code, but the idea is pretty simple - we traverse both trees simultaneously and record changes as we see them. This code could be made more sophisticated/performant, but I tried to not make it too complicated.

I also tried to remove pointless wrapping structures, so that the changes output is easy on the eyes.

OK, let's get the diffing wired up into our update function:

render : SafeHtml -> SafeHtml -> Cmd msg -> Cmd (Maybe msg) render before after cmd = case diff before after of [] -> Cmd.map Just cmd changes -> changes |> batchIfNecessary |> At 0 |> encodeChange |> renderPort |> (\renderCmd -> Cmd.batch [ renderCmd, Cmd.map Just cmd ]) update : (msg -> model -> ( model, Cmd msg )) -> (model -> Html msg) -> Maybe msg -> PrivateModel model msg -> ( PrivateModel model msg, Cmd (Maybe msg) ) update userUpdate view maybeMsg model = case maybeMsg of Just msg -> let ( newModel, newCmd ) = userUpdate msg model.userModel ( handlers, safeView ) = extractListeners "" (view newModel) in ( { userModel = newModel, handlers = handlers, view = safeView }, render model.view safeView newCmd ) Nothing -> ( model, Cmd.none )

And that's basically it:

I've prepared a Gist version for easier reading of the completed source code.

Discussion

Now all this is neat, but so what? I wrote this for a few reasons:

It's meant as a bit more advanced tutorial on some cool ports techniques.

You are not tied to the built in rendering system. If you want to build your own rendering engine in Elm, I've just shown you how. I hope this can encourage some experimentation with cool techniques. One of the popular ones these days is to not diff against a Virtual DOM, but against the real DOM. You can try that pretty easily.

You can customise the way the rendering works. Need a crazy component (say a map library) to work within your tree? All you need to do is implement a new Node constructor.

After reading this, you should be much better equiped to go and read the real implementation. It's much harder to read, but most of the concepts are the same.

However, a few caveats are in order.

This will not beat the native version in performance anytime soon. We need to convert all of the Elm datastructures into native JS ones and that is pretty costly.

I've not widely tested this. It's meant as educational material, not a production ready library.

Hope you enjoy playing with it. Let me know on the Elm slack if you can use this for some cool experiments :)

[gampleman]

A quick article showing how you can extend Lo-Dash with custom utility functions and how to test them in a generic, safe way.

[gampleman]

An in depth piece on some front-end dev-ops stuff I've been working on for the last six months.

[gampleman]

Orthodox Poirot.

[gampleman]

I wrote a blog post on our work blog about a project I created recently.

[gampleman]

Hi Jakub! I just came across your Automator action to change text encoding. It is great, almost the one that I need, but unfortunately it does not have an option for changing encoding from Unicode-16 to other encodings. I am struggling to find an action that would do it. I am totally ignorant in these things, so I cannot program it myself. Maybe you can suggest me how I can do it? Or where I can get it? I would really appreciate your help! Natalia

Four years ago I published a small Automator action for batch converting encodings. About time for an update. So I wrote a much quicker, open-source version of the thing. It should support now all encoding that NSString supports natively:You can [download this version](https://github.com/gampleman/ConvertCharset.action/releases/download/v2.0.0/Convert-Charset.action.zip) and check out the source code here

[gampleman]

Coupling

Today I was debugging some code. I spent the whole day on it, I ended up writing only 2 lines of code to fix the problem. Where did I spend all of my time? Trying to understand what the code did and how information actually travelled through the system. I'd like to discuss a few ways software systems communicate information and the impact of those choices on the maintainability of that software.

Criteria

We can abstract away and say that each method of communication introduces some degree of coupling - that is how much the two software components depend on each other.

Coupling has a few advantages and disadvantages. Loose coupling allows components to be more easily taken apart and reused with other components, decreasing the amount and complexity of doing that. Tight coupling on the other hand tends to be logically simpler.

Method calls

The simplest method is for one object to call methods on another object. This usually adds incredibly tight coupling making rewriting of both components necessary. Sometimes this also induces circular dependency issues, when object A needs to call methods of object B who needs to call methods on object A.

The principal method of reducing coupling is a variant of Duck Typing: where component A doesn't need to know or care anything about the exact type or implementation details of object B as long as object B can understand some methods that object A requires.

There are many variations of this idea e.g.: contracts, interfaces or protocols.

Method calls, unlike all of the remaining techniques holds a unique advantage: when reading the code, you can easily work out the flow within your program (there are still asynchronous issues, but at least it is clear which code will handle the future). All you need to know is the type of the variable and the method name, and then you can usually easily see the code responsible. Furthermore support in debugging tools is usually excellent.

Events; Callbacks

This technique works by having object A have a method that allows object B to subscribe its method to be called by object A whenever it decides to do so. In pseudocode:

window = new Window # window has a method called display button = new Button("Click to see a window") button.onClick(window.display) # when the user clicks the button, the method window.display will be called

Events can have various implementations that are better or worse. You can have very direct event distribution methods like the pseudocode above, or you can have quite indirect ones, like a global Notification Center object that all other components subscribe to or even a remote web service communicating with different components. The direct method we can see above is almost as good as method calls as we can see which object is talking to which object, the only thing that becomes more complicated is when does the call occur.

However when there is some central delivery mechanism (or perhaps code relies on bubbling mechanisms or similar), it becomes increasingly difficult to figure out what code is going to handle it. If you see code like:

NotificationCenter.broadcast(myEventNameVariable, eventPayload)

It takes quite a while to find the relevant code that responds to that. What is also implicit in these systems is that where using methods you would have code like this:

class A method doSomethingWithMethods() b.foo() c.bar() d.bum() class B method foo() doFoo() class C method bar() doBar() class D method bum() doBum()

With events you can rewrite that as:

class A method doSomethingWithEvents() NotificationCenter.broadcast('somethingDone') class B NotificationCenter.on('somethingDone', doFoo()) class C NotificationCenter.on('somethingDone', doBar()) class D NotificationCenter.on('somethingDone', doBum())

As you can see, with the first code we can clearly see that calling A.doSomethingWithMethods will first execute A.doFoo then B.doBar and finally C.doBum, with events we, by looking at A.doSomethingWithEvents we have no idea what is going to happen. We have to do a global search through our codebase to find all the places that the event string occurs. We have no guarantee in which order will the methods be executed. Furthermore, when in the first case we get an error on line 3 of method A, we know that object B is likely innocent, the likely culprit is object C. In the second case we will be lucky if we can trace it to object A at all, let alone have an idea to which of the listeners belongs the blame.

When should one use such a system then? In highly decoupled systems or as an alternative to local events. As such bubbling is often a good solution, as a component for which tighter coupling is appropriate can subscribe directly to the object, whereas a loose component can subscribe at the top-level. (e.g. I want to make a backup whenever the user saves something, but I have a fully generic backup solution, therefore I could subscribe to a top-level backup event as I don't care if the file being saved is a text document or a video file.)

Value observing; binding

Some languages support adding observers to variables - whenever a variable on a object changes, some object can run some code in response. If this is doubled, then it is called binding, where changing one variable automatically changes the value of another.

This looks like a really nice idea and typically allows for impressive demos where very little code can achieve a lot. But when you have a large and complex system, you start to believe that value-observing is a terrible idea - especially between objects. In most mutable languages we are used to assigning to variables all the time and it is generally considered fairly safe (except for loosing the previous value). Often we spend significant time debugging code until we realize that actually assigning to a variable triggered some code that triggered an asynchronous request that causes a logic error. Perhaps this technique could be more tolerable if the variable had to explicitly declare itself observable before this started working.

My recommendation: don't use value-observing other then inside the same object, and prefer easier to understand method such as setters and getters.

Conclusion

Consider the ease of debugging later when writing your code. Don't just add observers all over the place because it looks easier than defining a bunch of methods. Just because you made a brand-new notification service doesn't mean you need to use it for two halves of the same component. That's like two people in the same room sending each other text messages: possible, but awkward.

[gampleman]

This essay by Kathy Sierra examines the price we pay for decisions and the consequences of that on interface design. A must read for anyone in the field.

For those not in the field, this article describes the same cognitive process and the consequences on the judicial system and society at large. It's a bit long, but very much worth the read.

[gampleman]

Check out my sister's new site, where she features some of her work; a sample of which you see above.

[gampleman]

I've been somewhat industrious and made a small website that will convert a tumblr blog into a podcast you can subscribe to in iTunes. Let me know what you think.

[gampleman]

What do you want?

I have only one desire to be holy   Set apart for you, Master.As I perceive a faint wisp of your glory   I grovel in the dust.I long for you to touch me to crush my head   As a man crushes a beetle.But how can the Holy mingle with the loathsome?   And yet I demand no less.

[gampleman]

An Academic Poem

"The florist loved Mary" 1 "and"2 "Mary loved the florist." 1 "Pinning them down as attracted or unattracted, interested or uninterested, they enter into a new kind of arena, of all-or-nothing commitments." 3 "An almost comically tricky object for understanding and analysis." 3 "It does indeed start this snowball of epistemic virtue!" 3 "The florist loved Mary." 1 "Mary was the agent and the florist the patient." 1 "Love worries that I implicitly cleave to a classical view," 3 "that Mary is" 1 "the one I want to pursue" 9 "The symbol loosens the bond between agent and world" 9 "It is pretty clear that it is simply a bluff". 4 "Love sure knows how to hurt a guy!" 3 "Communication works best among those who not only share a language but who share a lot" 5 "more"2. "Communication involves an exchange of signals with a particular physical shape. They influence by impacting sensory surfaces." 6 "But they also play an irreducible role as the material symbols they are." 9 "It does indeed start this snowball of epistemic virtue!" 3 "I am both following and steering your own cognitive activities, as you are both following and steering mine." 7 "Now for a confession:" 9 "A very abstract relationship, not a substantial or physical one" 6 "we have" 2 "I experience the feeling of sadness" 8 "as essentially transformative" 9 "I come to understand what it was like for you." 8 "Love sure knows how to hurt a guy!" 3 --- ### References Apart from their aesthetic value, they are fine readings in Philosophy of Mind. 1. [Bechtel](http://mechanism.ucsd.edu/research/DERIVATI.pdf) (1994). 2. [O'Reagan & Noë](http://nivea.psycho.univ-paris5.fr/Manuscripts/ORegan;Noe.BBS.pdf) (2001). 3. [Clark](http://philpapers.org/rec/CLAILS) (2004). 4. [Stitch](http://www.jstor.org/discover/10.2307/186966?uid=3738032&uid=2&uid=4&sid=21101519992667) (1978). 5. Cummins (1996). 6. [O'Brien & Opie](http://ac.els-cdn.com/S0271530902000101/1-s2.0-S0271530902000101-main.pdf?_tid=1be16232-3e33-11e2-95b3-00000aacb35d&acdnat=1354640180_121cc7da4f61972793852b06f38e35a9) (2002). 7. Churchland (2002) as cited in O'Brien & Opie (2002). 8. [Davies & Stone](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.127.8921&rep=rep1&type=pdf) (2001). 9. [Clark](http://lac-repo-live7.is.ed.ac.uk/handle/1842/1439) (2006).

[gampleman]

Writing your own Canvas Scene Graph

Writing a scene-*what*? The HTML5 Canvas api provides primitive methods to express shapes, strokes and fills, which is all fine and dandy, but in a lot of situations we tend to think of drawing as composed of objects. E.g. to draw what I see from my window I need to draw a tree, a house and some grass, not a sequence of paths. This is true even more when we start animating, typically objects that are part of another object move when the parent object moves. One way to abstract this is to have a graph of objects where some objects are contained in other objects and each object takes care of drawing itself. This is called scene graph rendering. In this article I will show how to build your very own scene graph renderer for the HTML5 canvas. If you are simply looking for a full-featured, pre-built solution, I recommend checking out [CAAT](http://labs.hyperandroid.com/static/caat/). But sometimes you need something easily customizable/lightweight. We will build a single CoffeeScript class that implements everything necessary. ## The basics The first thing that any computer graphics program has to deal with are coordinate systems. We want each object to have it's own independent coordinate system and itself be defined in terms of it's parents. In general this indepence is one of the best reasons for using a scene-graph solution, since canvas tends to have a lot of global state, and you might often face the situation that changing one part of your composition affects a very different part. We wish to avoid that. The simplest way to solve this is to have each object be it's own canvas. So let's start coding: class Entity width: 0 height: 0 constructor: (@width, @height) -> # Create the canvas we will be rendering the object to @canvas = document.createElement('canvas') @canvas.width = @width @canvas.height = @height # Call this function to render out the object, returns a Canvas instance render: -> ctx = @canvas.getContext('2d') # Clear the canvas, important for animation ctx.clearRect(0, 0, @width, @height) @draw(ctx) @canvas # return the canvas # This function should be overriden in subclasses draw: (ctx) -> Now each subclass has to only implement the `draw` method with it's own set of drawing primitives. ## Composition We hover are still not a graph, there is no way in which objects can have no children. First we need to add a few more properties to our class: x: 0 y: 0 children: [] parent: null `x` and `y` are numbers that determine where will the object be drawn in the `parent`'s coordinate system. Let's implement a default `draw` method, that will render the children: draw: (ctx) -> for child in @children ctx.drawImage(child.render(), child.x, child.y) false # return some value, otherwise CoffeeScript will return an array The `drawImage` method takes an Image, Canvas or Video object and draws it at specified coordinates. Since `render` returns a canvas instance with the object drawn in it, we can now render it in the parent canvas. A simple demo of this class:

## Rotations To make this even more worth it, we would like to support rotating any object and having all it's children be rotated as well. Again we need to add a rotation attribute: rotation: 0 and tweak our `draw` function: draw: (ctx) -> for child in @children if child.rotation isnt 0 ctx.save() ctx.translate(child.x, child.y) ctx.rotate(child.rotation) ctx.drawImage(child.render(), 0, 0) ctx.restore() else ctx.drawImage(child.render(), child.x, child.y) false Canvas provides us with all the magic we need. We `save` the current context, then shift our origin point to the child's coordinates (NB: we are going to be rotating children around their origin point, perhaps you would like to implement rotation around center point). Then we do the rotation and drawing and then we simply restore our context to it's original point of origin. ## Shaping up the API First of all since most entity objects will want to override draw in one way or another we might simply use a [Kestrel](https://github.com/raganwald/homoiconic/blob/master/2008-10-29/kestrel.markdown) in the constructor and allow the user to optionally provide it when initiating the class: constructor: (@width, @height, @draw = @draw) -> @canvas = document.createElement('canvas') @canvas.width = @width @canvas.height = @height The `@draw = @draw` might look a bit awkward at first glance, but it assigns as a default our own implementation of `draw` if the user hasn't provided one. The js looks like this: function(draw) { this.draw = (draw != null ? draw : this.draw); } Next let's have a single function for instantiating new Entities and simultaneously adding them as children: add_child: (width, height, draw) -> child = new Entity width, height, draw @children.push child child.parent = @ child # return `child` so that we can do stuff like `collection = scene.add_child 30, 30` You might want to create another class that wraps up some common functionality as creating the top level `Entity` (typically called `Scene`) and setting its canvas as the one actually displayed in the document and setting up animation loops and so on. I'll leave that as an exercise for the reader (hint: look at the fiddles, there you have the basics). ## Caching If you draw expensive things in your objects (and some things in canvas are pretty expensive like shapes with multiple gradients/shadows), you might not want to redraw them 60 times per second. Our architecture allows us to prevent that rather easily. Again we add two more attributes: # This is what the user sets perform_caching: no # This tracks whether or not we should render is_cached: no Now let's modify our `render` function: render: -> return @canvas if @perform_caching and @is_cached ctx = @canvas.getContext('2d') ctx.clearRect(0,0,@width, @height) @draw(ctx) @is_cached = true @canvas We skip all the work when caching is enabled, since the canvas still contains all that was drawn into it. ## Wrapping up And that's basically all there is to it. There are a myriad of features you can add like more transformations apart from rotations, primitive entity subclasses (think something like `Sprite` entity), etc. The complete class is here:

[gampleman]

What bothers me about ACTA, SOPA, etc.

There is a significant problem with these laws (acts, agreements, whatever) even if you don't care about IP legislation at all. The current entertainment industry arose under a certain set of conditions. These condition have changed and the industry is failing to keep pace1. So instead of transforming into something modern they attempt to change the conditions back to their outdated state via the law. Let me make an analogy: imagine that Catholic priests started lobbying for the state to ban atheism because they were losing their jobs. The current legislation is the same thing for the entertainment industry. Many people view the current protests as some sort of hippie free-lunch2 absurdity. What is in fact at stake is however a core value of capitalism: if you can't make a profit with your product, you don't make the government protect your market, you make your *product better* or you shrivel up and *die*. ---- 1. Actually it's [not even doing so bad](http://torrentfreak.com/what-piracy-the-entertainment-industry-is-booming-120130/), but that's a separate issue. 2. The Pirate culture tends to elicit such views with slogans such as "Sharing is caring" .

[gampleman]

I don't believe that 2012 will be the end of the world, but it's a fun topic.

So: Good luck with surviving the apocalypse and any other similarly worthwhile endeavors in which you may partake! Happy 2012!

[gampleman]

Zdeněk Miler 1921-2011