Dance, Computer, Dance

by Ray Grasso

Adding Webpack to Middleman's External Pipeline

18 February, 2017

I use Middleman to build most of my content-focused websites. With the upgrade to version 4 comes the opportunity to move the asset pipeline out to an external provider such as Webpack.

I struggled to find good examples of how to integrate Webpack 2 with Middleman 4 so I’m documenting the approach I used here. For example code refer to middleman-webpack on Github.

Points of Interest

Build and development commands for webpack are in package.json.

"scripts": {
  "start": "NODE_ENV=development ./node_modules/webpack/bin/webpack.js --watch -d --color",
  "build": "NODE_ENV=production ./node_modules/webpack/bin/webpack.js --bail -p"
},

The external pipeline configuration in Middleman just calls those tasks.

activate :external_pipeline,
           name: :webpack,
           command: build? ? "yarn run build" : "yarn run start",
           source: ".tmp/dist",
           latency: 1

set :css_dir, 'assets/stylesheets'
set :js_dir, 'assets/javascript'
set :images_dir, 'images'

Assets are loaded by Webpack from the assets folder outside of the Middleman source directory1. Webpack includes any JS and CSS imported by the entry point files in webpack.config.js and generates bundle files into the asset paths Middleman uses.

module.exports = {
  entry: {
    main: './assets/javascript/main.js',
  },

  output: {
    path: __dirname + '/.tmp/dist',
    filename: 'assets/javascript/[name].bundle.js',
  },

  // ...

}

The config for Webpack itself is fairly straightforward. The ExtractText plugin extracts any included CSS into a file named after the entry point it was extracted from.

module.exports = {
  // ...

  plugins: [
    new ExtractTextPlugin("assets/stylesheets/[name].bundle.css"),
  ],

  // ...
}

This means you can include your styles from your JS entry file like normal and Webpack will extract the styles properly2.

Using the standard Middleman helpers to include the generated JS and CSS bundles allows Middleman to handle asset hashing at build time.

<head>
  <%= stylesheet_link_tag "main.bundle" %>
</head>

<body>
  <%= javascript_include_tag "main.bundle" %>
</body>

Finally

If you want to add modern JS and CSS to a bunch of statically generated pages then Middleman and Webpack works fine.

If, however, you are looking for a boilerplate for building a React SPA then something like react-boilerplate or create-react-app is likely a better fit.

  1. To avoid asset files being processed by both Webpack and Middleman. 

  2. Images are currently managed via Middleman and not Webpack. 

Structuring a Large Elm Application

21 October, 2016

I’m building an application in Elm and have been working on a strategy for breaking it down into smaller pieces.

My preferred approach is a few minor tweaks to the pattern used in this modular version of the Elm TodoMVC application1.

The Structure

The file structure is as follows.

$ tree src
src
├── Global
│   ├── Model.elm
│   ├── Msg.elm
│   └── Update.elm
├── Main.elm
├── Model.elm
├── Msg.elm
├── TransactionList
│   ├── Components
│   │   ├── FilterForm.elm
│   │   └── TransactionTable.elm
│   ├── Model.elm
│   ├── Msg.elm
│   ├── Update.elm
│   └── View.elm
├── Update.elm
└── View.elm

Global contains global state and messages, and TransactionList is a page in the application.

The top level Model, Msg, Update, and View modules stitch together the lower level components into functions that are passed into the top level Elm application (as shown below).

--
-- Main.elm
--
import Html.App as Html
import Model
import Update
import View

main : Program Never
main =
    Html.program
        { init = Model.init
        , update = Update.updateWithCmd
        , subscriptions = Update.subscriptions
        , view = View.rootView }

--
-- Model.elm
--
module Model exposing (..)

import Global.Model as Global
import TransactionList.Model as TransactionList

type alias Model =
    { global : Global.Model
    , transactionList : TransactionList.Model
    }

init : ( Model, Cmd msg )
init =
    ( initialModel, Cmd.none )

initialModel : Model
initialModel =
    { global = Global.initialModel
    , transactionList = TransactionList.initialModel
    }
  
--
-- Msg.elm
--
module Msg exposing (..)
import Global.Msg as Global
import TransactionList.Msg as TransactionList

type Msg
    = MsgForGlobal Global.Msg
    | MsgForTransactionList TransactionList.Msg

One of the things I like about this pattern is how readable each top level module is with import aliases.

View, Update, and Global State

The view and update functions compose similarly but I pass the top level model down to both so that they can cherry pick whatever state they need.

The lower level update functions can look at all the state and just return the piece of the model they are responsible for. For example the Global model can have common entities and state specific to the transaction list live in the TransactionList model.

Views are similar in that they can take state from the global model as well as their own model and render as necessary.

--
-- Update.elm
--
module Update exposing (..)

import Msg exposing (Msg)
import Model exposing (Model)
import Global.Update as Global
import TransactionList.Update as TransactionList

updateWithCmd : Msg -> Model -> ( Model, Cmd Msg )
updateWithCmd msg model =
    ( update msg model, updateCmd msg )

update : Msg -> Model -> Model
update msg model =
    { model
        | global = Global.update msg model
        , transactionList = TransactionList.update msg model
    }

updateCmd : Msg -> Cmd Msg
updateCmd msg =
    Cmd.batch
       [ TransactionList.updateCmd msg
       ]
       
--
-- View.elm
--
module View exposing (..)

import Model exposing (Model)
import Msg exposing (Msg)
import TransactionList.View as TransactionListView
import Html exposing (..)
import Html.Attributes exposing (..)

rootView : Model -> Html Msg
rootView model =
    div [ class "container" ] 
        [ TransactionListView.view model ]

This approach seems to be working pretty well so far and it seems like adding routing shouldn’t be too difficult.

A Step in the Right Direction

28 July, 2016

I pulled the pin on working in the React/Redux space a few months ago after I became tired of the churn. Things were moving quickly and I found myself spending more time wiring together framework code than writing application code. This kind of thing sneaks up on you.

One glaring ommission was a preconfigured and opinionated build chain. I moved from starter kit to starter kit chasing the latest webpack-livereload-hot-swap-reload shine. Each kit was subtly different to the one before it. Not just their build components either. I missed having agreed-upon file conventions on where to store your actions, reducers, stores, and friends. It made me appreciate the curation provided by the Ember team in their toolchain.

The creation of Create React App (triggered by Emberconf no less) is a step in the right direction. Bravo.

Infrastructure Koans

21 May, 2016

Envato teams are responsible for the operation of the systems they build.

My team is trying something different to help onboard new people. We’re creating a set of infrastructure koans for them to complete. The koans are tasks that—once completed—will help folks navigate our infrastructure and systems, and thereby acquire skills that are essential for supporting our services.

When someone joins the team a new issue is created in one of our team’s Github repos using the koans document as a template. Once the new team member has completed all of the koans they are added to the on-call rota and assigned a buddy who can help if things get tricky whilst on call.

The koans are not meant to be layed out step by step unless the task is complex or requires unusual knowledge. We hope this encourages folks to explore and internalise more than they would if following a todo list.

Some Example Koans

Set yourself up on PagerDuty and read through past incidents.

View metrics for each of our systems in New Relic.

  • What is the average response time?
  • What does CPU, Memory, and I/O utilisation look like on each server?
  • What are the slowest transactions for the service? Dig into each transaction and see where the time is spent.
  • Check the error analytics tab and look for any relationships between errors and past deployments.
  • Check the availability and capacity reports.
  • Look for trends in the web transactions and database reports.

Look up each of our services in Rollbar.

  • What are the two most common errors being reported?
  • Drill into the details of a recorded error.
  • Are these errors we can live with? Should we create a task to fix them?

Open the AWS CloudWatch console.

  • Look through the available dashboards and metrics.
  • What CloudWatch alerts do we have configured for our production systems?

Open the AWS ECS console.

  • How many task definitions do we have? How many available versions exist for each of them?
  • Which systems make up each of our service clusters?
  • How many repositories do we have in ECR?

Look through our Stackmaster templates and find the results of building stacks from them in CloudFormation.

Access our ELK cluster and run some queries.

Run queries against our production database replicas.

Decrypt some database backups.

SSH into various servers in our infrastructure.

A Docker Container for River5

30 March, 2016

I’m rebuilding a VPS that I use for a bunch of my websites, side projects, and experiments. It hosts some static sites via Apache, a few rails apps, a node app, Postgres, and other bits and bobs. I want maintenance of configuration to be simpler in the future so I’m giving Docker a crack.

One of the side effects of using Docker should be that I can mess about with different tools and rollback to a clean state easily.

My first effort in learning Docker has been to create a container for hosting the River5 river-of-news aggregator by Dave Winer.

It’s up on Github and ripe for pull requests.