The other day I set up automatic testing of my website using Google Chrome Lighthouse. I’m also running my own Lighthouse CI server to store test result history and show trends. Here’s an outline of what this means, some of the results I’ve obtained so far, and my thoughts on the usefulness of Lighthouse.

What is Lighthouse?

Lighthouse is a tool developed by Google which audits websites and gives them a score in four categories: Performance, Accessibility, Best Practices, and SEO (search engine optimization). It’s also built into the developer console in Chrome, so you can run it manually on any website you want directly inside your browser.

I have already been using Lighthouse manually to do occasional tests on my website and its theme, bobatheme. However, I recently came across this GitHub Action. It caught my eye because I was already using GitHub Actions to perform a few basic Hugo¹-related audits found in a thread on the Hugo forum.

Setting it all up

My goal is to run Lighthouse’s audits on an environment as similar to my real website’s deployment as possible.

First, I created a separate lighthouse job on my audit workflow on GitHub Actions. This job will do all of the testing. Since it’s a separate job, it runs in parallel with my existing audit task.

Next, I copied my existing build steps verbatim. This makes the site effectively identical to my real deployment on bbaovanc.com. One of the crucial steps is to pre-compress all the files so their compressed versions can be served without the server having to compress them on-the-fly, saving valuable processing power.

Serving the compressed files is crucial because it simulates the real world transfer size of the files. In addition, Lighthouse will recommend that you serve the files compressed.

Inside the audit environment I use the same webserver (Caddy) and a similar configuration (compared to the one running on bbaovanc.com) to what I have running on bbaovanc.com. Caddy also is able to easily run the demo site with HTTPS, even on localhost. That allows it to test certain elements (such as the comment section) that wouldn’t be allowed to load if it were plain, unsecured HTTP.

I selected a few significant pages on my website so Lighthouse can test a wide range of the features in my website’s theme. The results of these tests are then uploaded to my Lighthouse CI server. There you can view the audit history and see differences in results between builds.

What I found from the tests

Performance impact of a YouTube video

I found that embedding a YouTube video using the built-in shortcode in Hugo made the website drastically slower. It’s a lot clearer in the audit environment where everything is throttled down a bunch, so the performance impact is more clear. Below is a screenshot of the difference in score between my blog post with a YouTube embed and a similarly sized blog post which has no video.

I can probably optimize this a little by making a copy of the built-in Hugo YouTube shortcode, and modifying it to defer the loading of the remote scripts. That will have to be a future project, and I’ll probably write a post if I ever end up testing it.

My thoughts on Lighthouse

You can make the argument that the results from Lighthouse don’t really mean much in the real world. On modern devices and a decent internet speed most websites will load just fine.

Although most pages on my site have a perfect score from Lighthouse, I’m not going to go way out of my way in order to keep it that way. After all, Lighthouse is just a tool intended for “improving the quality of web pages.”² That doesn’t mean it needs to be perfect.