Compile Time

Coupling, in the form of importing modules, is the key to causing large jumps in compiled files and eventually a blowout in compile time. Anyone who's experienced their workflow grind to a halt knows what this feels like. elm-module-graph¹ is an excellent tool in diagnosing where your dependencies come from. Making modules Simple² is the key to having a constant compile time as your application grows.

There are two kinds of issues when trying to improve compile time. Compiler/OS optimisation issues and your application architecture. The former has well documented workarounds³ and are likely to be addressed in the 0.19 release. The latter is the focus of this section as it will affect most codebases beyond roughly 10k lines. If your application is less than this, or your compile time is under 10 seconds, focus on features and do not waste time fiddling with compile time optimizations. Read something useful like Refactoring to reduce time spent on bugs.

Do you have a coupling problem?

Here's the test:

1. Compile your app
2. Touch a commonly edited module ( a page, a service, model, controller etc... )
3. How many files recompile?
4. Repeat this for a few files and get an average.

So that was a bit of a lighthearted take on what can turn into a very frustrating problem down the line. The point is that a healthy app should really only be compiling 3-5 files for most changes regardless of the size of the app. This means when you hit 1000 files and 100k LoC, you're still only compiling 3-5 files. Of course this does not hold for your common files or library files because by definition, they will be imported by alot of modules but more on strategies to mitigate that in File Structure.

To give you an idea, our application of 450+ files with 45k LoC typically recompiles 3-6 modules with page changes. Components cause a jump to about 100+ taking over 3 minutes to complete. Touching Alfred.elm or our Types.elm definition recompiles over 200 files and we take a short coffee break. Sadly unavoidable.

Our story...

We had ~350 files in ~30k LoC. A typical change affected a 65+ files and compiled in ~2 minutes. This meant adding a Html.div or a Debug.log took 2 minutes each time. This had a HUGE effect on workflow and team morale. We literally stopped working and gave birth to elm-hack (ported a version⁶ to my game), our infamous compile aide. I won't go into the details there, my talk⁴ with slides⁵ goes into a bit more detail how we were able to get from 2 minutes down to 2 seconds. This bought us enough time to get to a point where management trusted us enough to let me fix the core issue. Fast forward 3 months, and a complete re-architect of the root modules, we're now at ~45k LoC in 426 files. A typical change will affect 3-4 files and take 15-30 secs.

(Aside: "Complete re-architect" sounds scarrrry, ooohhhhh, a COMPLETE re-architect, aaarrrhhhh. But we're in Elm. And I had done all the refactorings I mentioned in the refactoring section. I spent 2-3 days to do some major 1-2k lines of reshuffling of modules. Then I spent the rest of the week and another 3-4k lines to rewrite routing, page loading, port existing pages over. One week, handful of bugs, not the end of the world. There is a very good reason experienced coders fear rewrites or changing core architecture, we've all experienced the pain of going through a long regression trail after such changes. Elm rewrites the rules. Refactor fearlessly.)

Case study ( Quotes )

To explain why coupling is bad for compile time, I'm going to use a work related example. In health insurance, there is a concept of Person, these people take up a either a single, couple or family Membership.

-- Person.elm
type alias Person =
    { id : Int
    , -- more fields
    }

-- Membership.elm
type alias Membership =
    { id : Int
    , -- more fields
    }

There is a large list of pages which handle various aspects of both:

-- Medicare.elm
import Person exposing (Person)

-- ClearanceCertificate.elm
import Person exposing (Person)

-- Rebates.elm
import Membership exposing (Membership)

-- etc...

A feature of insurance is we get a quote for a single person or a membership.

-- Quote.elm
type QuoteID
    = PersonID Int
    | MembershipId Int

type alias Quote =
    { id : QuoteId
    , -- more fields...
    }

Now, since one can make a quote in Person or Membership, it makes sense to add it to the respective models.

-- Person.elm
type alias Person =
    { id : Int
    , quote: Quote
    , ...
    }

-- Membership.elm
type alias MemberPolicy =
    { id : Int
    , quote: Quote
    , ...
    }

Visually this looks like the following:

Great, we've got some code sharing here. Both Person and Membership now share the Quote component and we're happy! ( Of course, the avid reader would know that's not how the story goes.)

touch Quotes.elm
elm-make ...
Compiling 33 modules

According to my highly accurate 'Do you have a Coupling problem?' chart, we ranked (30+):

There is some serious coupling here. The xkcd on compiling is no longer funny because you’re about to lose your job from lack of productivity.

So we had a common component, it was added to the model where it was needed most, then the view and update logic are shared. What's the problem?

Here's some food for thought:

touch Person/Page1.elm
elm-make ...
Compiling 1 module

touch Membership/Page15.elm
elm-make ...
Compiling 1 module

touch Membership/Model.elm
elm-make ...
Compiling 15 modules

When you touch a module (Quote.elm), it will re-compile ALL modules (all pages, person/membership models) that directly or indirectly import the changed module.

When does it not do this?

Myth #1: If I change a non-exposed function in Quotes.elm it will not re-compile all modules that import it.

Myth #2: If I make my Quote type opaque and only export the opaque type, it will not re-compile all modules that import it.

Myth #3: If I add a comment to Quotes.elm, it will not re-compile all modules that import it.

Fact: Doing anything to Quotes.elm will re-compile all modules that import it.

This is the reason for all non-compiler/OS related compile time blowouts and it can happen very quickly, just with the introduction of one feature, badly coupled.

There is an excellent tool for looking at this specific scenario of coupling in your app, elm-module-graph¹. And for the case study, it produces the following graph:

So Quotes is the highlighted module. It has 11 blue modules lit up, these are the modules that either import it directly or indirectly. There are 3 black lines that comes out of quotes, these are the modules that import Quotes directly. The rest of them come out of Model.Person because Model.Person imports Quotes.

This is how it looked after I removed Quotes from the Person/Membership models and passed it into functions that required it.

So now Quotes is directly imported by Membership and Person, two TEA modules and this is imported by Main.

touch Quotes.elm
elm-make ...
Compiling 4 modules

Let's count that, Quotes, Membership, Person, Main. Four!

Healthy! Read this section to laugh at my struggles but otherwise you're wasting time here.

So with this pattern, even if your app reaches 426 files, your compile time will not grow with it.

Footnotes

¹. elm-module-graph: https://github.com/justinmimbs/elm-module-graph ↩

². Rich Hicky's Simple Made Easy: https://www.infoq.com/presentations/Simple-Made-Easy ↩

³. Compile time issues and workarounds: https://gist.github.com/zwilias/7ed394ec0e9c6035e1874d19b721e294 ↩

⁴. Elm Remote Meetup talk - https://youtu.be/ulrukPRYsws?t=50m4s ↩

⁵. Elm Remote Meetup slides - https://docs.google.com/presentation/d/10vN7eLr3qsd4nK2zcxUHgbu68fO_yzAA-b7Hf7kEcik/edit?usp=sharing ↩

⁶. elm-hack.sh, a script to compile single modules VERY fast - https://github.com/mordrax/cotwelm/blob/master/hack.sh ↩

⁷. What we do when Elm is compiling - https://xkcd.com/303/ ↩