Revisiting gulp. Realtime JSX compilation and browserification.
Today I decided that it was worth investing some time in revisiting the build tools that are used across the Double Negative web properties.
We use gulp pretty much universally across our projects. In fact not only do most projects use gulp, but most projects use very similar build processes full stop.
It thus seemed pretty stupid that each project maintained its own completely independent build process such that when I made 'big' changes and/or learned new things I would have to implement it across every project. I would inevitably forget, and then spend time down the line debugging bugs and problems that I had already fixed numerous times before.
What I wanted to achieve
I remember watching Bret Victors Inventing on Principle and thinking that the kind of development processes he demonstrates are what I want to have in place for Double Negative.
That is to say that I want to be able to make a change and see what it does in 'as realtime as possible'.
That distinction in itself is very important. Previous 'versions' of my build processes have simply recompiled all the JSX files after a single change simply because it was not that slow - a few seconds. But.. those few seconds really (and surprisingly) mess with your head and your workflow.
I was aware that Gulp 4 had functionality pertaining to sequential execution of tasks. Having however previously had issues with Gulp 4, I wanted a solution that worked with Gulp 3.
I stumbled upon the run-sequence package as outlined in this StackOverflow answer. It does what it says on the tin, but I had a few issues in that it was seemingly considering incomplete asynchronous tasks complete when they were not. My code was being browserified before it had all been compiled from JSX.
I investigated asynchronicity and noted from this answer that a
return statement informs gulp that the task in question is over. This resolved my issue.
This posed a problem in that each time a JSX file finished compiling it was considered a JS file edit, and as such the watch on JS files was triggered. If multiple JSX files (or files that depended on them changed) then the second watch task would be triggered multiple times.
I resolved this in a non-perfect, but clean and effective way. Two watch tasks, and a global boolean indicating the status of ongoing tasks.
My final code looked like this:
It works as expected.
This is another case whereby I feel that had I read through the gulp documentation in advance, things would have progressed a lot quicker.
I always forget that let is block scoped. That is why I define
taskOngoing'globally' outside of my 'watch' task body.
And some of the intricacies of gulp:
- Tasks need a return statement to execute synchronously.
Take for example the line commented with 'THIS line is discussed below'. If you follow your log output (use gulp-util) this log will appear (in what might seem) the wrong place. Given that the tasks in my sequence are asynchronous this log may well be outputted prior to any logs contained within the sequenced tasks that only run asynchronously and after the previous task has completed.
Now when I begin a development session I simply make sure that my 'watch' task is running (
The initial problem
I noted that one part of the initial problem that I was trying to solve was that I used similar build processes across sites.
The final part of the puzzle was implementing this 'generic' build process as a git project in and of itself and including it within our projects as a git submodule.
The obvious down side to this is that it implies that my build process for all sites is exactly the same when it is in fact not. My resolution to this is simply to maintain a branch of the build project for each project. Generic changes to the build process can be merged into the individual branches, and the status of such merges can be easily and logically tracked from the repository.
I skipped over some implementation details of the individual JSX/browserify tasks.
Some of the NPM package that I use are:
gulp-dependencies-changed - for discerning which files have dependencies that have changed.
gulp-newer - for discerning which JSX files are 'newer' than the JS files that they compile to.
Take a look, and have a play. If you have any issues, or questions then let me know and I will do my best to point you in the right direction.