Speeding up HTML page creation for Webpack

There is a very popular webpack plugin for building HTML pages, it is called: html-webpack-plugin. Judging by the slogan, the main motivation for this plugin is "simplifying the creation of HTML pages for bundles". In my case, this was exactly what I needed, since I needed to create a static site consisting only of html/css/js files + various resources, such as pictures, fonts, etc.

Having tried this plugin in work, everything suited me. For templating I used Pug, but there were no problems with this. Rendering pug templates was done using pug-loader for webpack. Plugin configuration was quite simple, it was necessary to specify the relative path for the generated HTML file filename, the path to the page template template, and the array of bundles chunks, which should be inserted into page.

Configuration Example:

new HtmlWebpackPlugin({
    filename: 'relative/path/to/file.html',
    template: '/path/to/template.pug',
    chunks: ['chunk1', 'chunk2']
});

At the output, we received ready-made HTML pages with already inserted page bundles. But, for some reason I did not understand, creating one page took too long, something around 3000ms. The more pages there were, the longer it took to wait for the creating. This did not suit me in any way, since in the case of static sites, the number of pages is constantly growing!

Such a picture was when my site had 34 pages. The HTML creating took almost 2 minutes...

After reading articles on accelerating the build process, I found that can parallelize this process. That is, depending on the capabilities of your processor, several parallel processes will be launched for the build. Thus, if you have a 6-core processor, then theoretically, you can speed up the build by 6 times. The first attempt was thread-loader, but unfortunately I didn’t get the result from this loader, I don’t even remember why... Fortunately, in the articles I came across a plugin happypack. With happypack I was able to reduce the build time by about 3.5 times, which could not but rejoice!

Time was reduced by about 3.5 times. I expected a 6-fold acceleration (6-core processor), but still the costs of creating separate processes took their toll...

However, the total build time directly depended on the number of pages, and it still took 500 to 1000 milliseconds to generate each page. That is, 100 pages => 100 seconds, 1000 pages => 1000 seconds. Obviously, with such numbers, the question arises about the appropriateness of the chosen method of generating the site, but I did not want to abandon webpack and I could not accept the fact that it takes so much time to render a simple pug template! I didn’t want to dive into the source code of the plugin html-webpack-plugin, so I was looking for a replacement for it.

And then I met in the webpack documentation a mention of one completely unpopular plugin, it was called htmls-webpack-plugin. The difference in names is only in one letter: htmlS-.... Features of the plugin inspired hope:

• Simple and flexible, you can control anything, no need to configure many plugins;
• Support for multiple HTML by default;
• Fast, almost 20 times faster html-webpack-plugin for 20+ pages.

Configuration Example:

new HtmlsWebpackPlugin({
	htmls: [{
	    src: '/path/to/template.pug',
	    filename: 'relative/path/to/file.html',
	    render: (file, params) => {
		    return pug.renderFile(file, params);
		}
	}]
});

After integration and verification, my expectations were met, build speed was reduced by another 5 times! Now the generation of one page took only about 170 ms - basically, this time was spent on generating HTML from pug template using pug.renderFile(). But, unfortunately, the generation process fell out of loader-flow and happypack could not optimize this process... That is, parallel generation of templates did not work at the moment.

For parallelization in this approach, I decided to use the npm-package worker-farm. Developer htmls-webpack-plugin provided asynchronous HTML rendering mode, which made it easy to organize a farm of pug renders, which, like with happypack, in parallel, in several streams, generated HTML. Speed increase was almost 2-fold, which reduced the time required to build one HTML page to the cherished 100ms.

We are close to the cherished 100ms to the page!

Conclusion

Thus, in the end, I reduced the build time of one page from the original 3000ms to 100ms, which is a very good result in my opinion. If this isn’t enough for you, in this case you can either look for a faster template engine (maybe Pug is not the fastest, did not compare), or generally work with pure html!

Pivot table for 34 pages.

Nuances:

• Since we are personally creating HTML pages, webpack now knows nothing about our templates. What does this lead to:
  1. When changing the template files, it is necessary to restart the build process, because it itself will not catch the fact of changing the file and HMR will not work;
  2. In development mode, you need to cache the generated HTML so that when you change the bundle files (js / css, etc.), you do not have to re-render templates that have not changed. The production build does not need a cache.
• For fun, I conducted a stress test of my configuration. Building of 1088 pages took 1 minute 27 seconds, that is, the create time of one page decreased to a record 80ms! But the subsequent increase in pages did not give an increase. Building of 2176 pages took 3 minutes and the same 80ms on one page... Apparently this was all that my processor was capable of!

Versions:

{
    "webpack": "4.41.2",
    "html-webpack-plugin": "3.2.0",
    "htmls-webpack-plugin": "1.0.9"
    "happypack": "5.0.1",
    "worker-farm": "1.7.0"
}