Development Update

Well, my plan to write much more frequent blog posts fell through, unfortunately. When I made the plan to divide the development work into milestones, I had the ambition and expectation that each milestone would be completed significantly faster.

So why is it taking so long? There's no one simple answer to that - there are several factors at play:

1. Underestimating how many fundamental issues needed to be resolved

So, if you have to redesign 100 pages, the first page will only take 1% of the time, right? No, that's not how it works in reality. It's been a huge undertaking just to reach the point where we could get started on redesigning the first few pages, as there were many fundamental issues that needed to be resolved first.

And, as I've tried to explain before, it's not just a visual redesign. It is a comprehensive restructuring and rewriting of the entire Virtual Manager platform, which we will benefit from in the long term.

However, in order to get some new content in the game sooner, I've decided to adjust the development plan. I'll write more about that further down.

2. Only 1 full-time employee

For those who don't know, I am the only full-time employee here. I am the company's owner, developer, accountant, secretary, cafeteria lady, designer, etc. etc. In addition, I have FC Apollyon, who works a few hours a week as an administrator, and our freelance developer, who I hire for limited periods.

Obviously this sets a limit to how fast things can go, since I'm alone with the development most of the time. However, our freelancer has just returned, and I have arranged to contract him for 300 hours over the next 3 months.

3. Design is hard

We've had to come to the realisation that design is pretty dang hard when you get into the weeds, and we don't have anyone on the team who is particularly good at it.

For me, this means that I often get stuck and frustrated. This often results in me giving up and instead doing backend stuff that may not be necessary right now.

Although I'm investigating the possibilities of hiring a designer to go over a limited number of pages, I do also need to get better at it myself.

It's unrealistic to think that we could afford for a designer to do everything, because there are MANY more pages in Virtual Manager than you'd think. As of today, the game contains 462 unique pages and 240 partials (partials are fragments that are inserted into the pages and in some cases reused in several places).

That's why I've started a self-study in design, which will hopefully make it easier for me to redesign a page without getting stuck. In addition, the work should naturally go faster the more pages we finish and as we build a "catalog" of solutions to common UI challenges.

4. Life circumstances

In addition to the above factors, we also can't get around the fact that I don't work as efficiently as when I started at the company right after university. Back then, code was pouring out of me, and I even couldn't help but work on it in my free time, especially when I was given the task of making the new match engine (the one with the little round tiles for players) to replace the old text-based one. We developed it amazingly quickly in just a few months. I made the match engine itself (backend, AI, etc.) while the game's founder, Rasmus (The Dudes), made the visual part.

It should be said, however, that some of the breakneck pace we managed in the first few years after I was hired came at the expense of a some amount of technical debt - a debt that we are still paying off, in the form of all the work we are currently putting into this extensive rewrite.

But time went on, and after working here for 12 years, I finally bought out the previous owner. At that very moment, COVID-19 hit, and my plans to rekindle my inspiration and work from exotic locations instead turned into me just sitting around at home. Too many years of working from home, a couple of long-term illnesses, and a bad bike crash that resulted in a fractured pelvis have left me with bad habits and in poor physical shape. "Fat and lazy" some might be cheeky enough to say, but I do have to admit that things just move a bit slower these days.

I was hesitant to write this section because I don't want it to sound like I'm fishing for sympathy. I'm not. These are my personal challenges, and I'm the only one who can solve them. I included it only for the sake of those who wouldn't be satisfied with anything less than the full explanation. These are just facts of life, and they're all things I'm working to improve.

The plan for the near future

It's clear that it has been too long without noticeable improvements to the game. Therefore, I have decided cut short the work on Milestone 1 and wrap up the things we have in progress as soon as possible. The things we are still missing in Milestone 1 will be moved to a later date.

After that, we will start working on Milestone 2, which, as you know, is all about tactics. I have written about it before, but in short, it involves splitting up and replacing a number of the current tactics settings with much more detailed settings.

It's a feature that I have had in mind and looked forward to for several years, and it's an area where we're much more at home than with design. In the meantime, I will continue to studying UI design, so that I'll be better equipped when we return to it.

A look at a few things from the redesign

...now, let's take a look at some of the things we've done on Milestone 1.

Clubhouse

One of the first pages we redesigned was Clubhouse -> Overview.

On this page we just went with the simple solution and kept the structure and content from the old one, since it actually works OK. The page has been reimplemented in our new design system, and most importantly, it is responsive; i.e. it adapts smoothly to different screen sizes.

As the screen gets smaller, the content shrinks until the space is too tight for the content. Then the side and top navigation are hidden away in the drop-down menu, making room for us to keep the 2 columns. At the same time, the content area switches to go all the way to the edges of the screen.

As the screen gets even smaller, we eventually reach a point where the 2 columns collapse into 1.

Shoutbox

As I also mentioned in the last blog post, ALL javascript is being discarded and rewritten in a way that works with our new platform. That meant we also had to rewrite the shoutbox. It looks like itself, but is now fully integrated into our new platform, and you can, for example, scroll between messages by swiping your finger on mobile.

Dark mode

Many people have asked for a dark mode in Virtual Manager. We have previously experimented with it internally, but it became too messy and disjointed, because the old site was simply not built for it.

With the redesign, dark mode is built in from the start. In our design system, all the main colours are defined as variables that are named after their area of ​​application. Text and background colours are defined in matching sets that have enough contrast to keep the text easily readable. So instead of specifying that a section should have black text with a light grey background, for example, we instead specify that it should use the colour "base-300" as the background and "base-content" as the text colour.

This means that we can define the named colours in two versions: light mode and dark mode - and thus we have solved the problem of supporting dark mode once and for all. In the dark mode palette, the two named colour variables are then defined as an almost black for the background, and a light grey for the text.

If we use the named colours consistently, we don't need to think about dark mode on every single page - it just works. A simple javascript component can then switch between the 2 palettes based on what you have chosen.

Sharp focus on performance

As I mentioned in my last blog post, our current project is not just about a new design. It is a comprehensive rewrite of the entire game platform, and central to that is a sharp focus on performance and lightning-fast response times.

On mobile, 1-2 seconds of response time is simply not good enough. We need to get below 500ms on most pages, and preferably even less. This requires us to bake performance into the system from top to bottom, in a way that was not possible on the old site.

To achieve this, we need to optimise every link in the chain.

• The database must deliver data as quickly as possible, and must be spared when we can.
• The web servers must generate pages lightning-fast every time you click.
• If you are performing actions that are very computationally intensive - e.g. training - they should be performed asynchronously in the background. You should get immediate feedback that the training is in progress, and then you should be automatically forwarded to the result, the moment it is ready.
• We should send as little data to you as possible, which is especially important if, for instance you are on the bus going through an area with poor coverage.
• Once you have received the page content, we need to make sure that your device does not have to spend a lot of time parsing and displaying it. Again, this is especially important on mobile.

Caching

Caching is one of the most important tools to help us achieve this, both by sparing the database and making it less work for web servers to generate the page you have requested.

Previously, we have only added caching on a few problematic pages to fix immediate problems. With the redesign, caching is baked in everywhere from the start, and we cache as much as possible.

The technique we use is called fragment caching, and it works by storing fragments of pages on a lightning-fast cache server, so that we don't have to generate the same piece over and over again, as long as the data they are made from hasn't changed. This way, both the database and web servers have to spend less effort generating the pages, especially if the cached fragments are shared by all users.

If we take a look at Clubhouse -> Overview

1. Top navigation
2. Side navigation
3. Your club's info box
4. League table
5. Every match in "Current matches"
6. Every message in the shout box
7. Vifa rank progress

There are many pitfalls when caching, but we already have experience with most of them from the past.

The first pitfall is that you can easily end up showing data that is too old. Our system solves the problem by each cache fragment having a key that contains the time that the data object we are displaying was last updated. A concrete example would be the matches in the "Current matches" section. Each match has a timestamp that the system automatically updates with "now" whenever something changes. The same applies to most of the other data objects you can imagine in the game, such as clubs, leagues, players, teams, forum posts, etc. etc.

When the web server has to piece together the Overview page, it checks for each match in the list whether there is a cached fragment with a key where the match's last update time matches. If it doesn't, the fragment is rendered as normal, but it is stored in the cache so that it is ready for the next time.

The outdated fragments are automatically cleaned up by the cache server, since it clears out the least recently used objects when it is running out of space.

In reality, caching is more complex than the example, since we also have to take things like different languages into account. That's why a language code is also included in all cache keys.

Dates and time represent an even bigger challenge. First, different languages ​​have different date and time formats. This problem is already solved by including the language code in all cache keys.

It becomes much more difficult when we have to take time zones into account. We could include the time zone as another component in the cache keys, but every time we add something else to the cache keys, the number of different versions of fragments that we have to store in the cache gets multiplied. We would prefer that as many users as possible share the same fragments in the cache.

In addition, there is the problem with relative times, for example when we specify a time like "yesterday", "in about 5 minutes", etc. This is a problem because the wording has to change continuously. It's not the timestamp in the data object that's changing - it's just time passing. So here we don't even have the option to fix it by adding something else to the cache key.

Previously, we have simply avoided caching times, but with the redesign I insisted that we find a solution, once and for all.

We have done this by implementing a way to insert times on the pages, which contain the time in UTC, together with instructions to a Javascript component on how to format it. Javascript runs on your own machine, and the component formats the time based on your time zone every time you retrieve a page.

This way, everyone can share the same cache fragment, even if they are in different time zones, and it also solves the problem with relative times.

We also cache all of the navigation. As you've already seen, the side navigation moves into a drop-down menu on the left side on small screens. Unlike the side navigation on desktop screen sizes, this navigation panel contains links to all main sections on the entire site including sub-pages in a tree structure. We have to include the entire navigation menu content on every page view, even though most of it will be hidden on large screens. We do this because it needs to be available immediately if you change screen size, for example by turning your tablet from landscape to portrait. That makes it even more important that we cache it.

The side navigation you see on large screens is actually the same as the one in the drop-down menu. We just use styling to position it and make the irrelevant sections temporarily invisible on large screens. In order to cache the navigation, we had to find a solution to highlighting the link to the page you are currently on. On the old site, navigation isn't cached, so it's generated fresh on the server with each page view, and the highlighting is inserted directly. But, since we're trying to reuse the same fragment on all pages, we cannot insert highlighting directly. We would have to cache a version of the navigation for every possible page, to get the correct highlighting.

This has required a new navigation backend and a new Javascript component that can automatically highlight the correct link in the navigation, control which sections should be invisible on large screens, and at the same time ensure that the relevant main section is always expanded in the menu on mobile.

The last example is the way we handle content needs to look different depending on which user is viewing it. One example is the league table. Here, your own club needs to be highlighted, and once again we had to find a solution that is cache friendly and allows us to send the same cached fragment to everyone, while at the same time highlighting your own club. We solved that in much the same way as with times and dates. We insert extra data into the league table for each club, which a Javascript component on your own machine can then read and ensure that your club is highlighted.

Significant reduction in CSS and Javascript

With our new platform, we have also achieved a significant reduction in the amount of CSS and Javascript we send. Even though your browser automatically caches JS and CSS the first time you visit the site, your device still has to spend time loading and parsing all of it on every page view. On a desktop PC, this may not matter that much, but once again, it is something that can make the experience worse on mobile.

A comparison of the old and new version of Clubhouse -> Overview shows how significant the reductions are:

	Old page	New page
CSS	852 kB	155 kB
JS	1239 kB	545 kB

CSS has been reduced by 82% and Javascript has been reduced by 56%.

These numbers only include the components we send ourselves - not third-party components, such as Google Charts, which is used for VIFA progression chart.

These were just a few examples of the work we put into making the game's new platform lightning fast. As I've already mentioned, the redesign is a major restructuring, with performance being a main focus.

The coming weeks

As I mentioned, we're currently wrapping up the things we're working on in Milestone 1 so we can move on to the tactics improvements. I expect to start working on the tactics editor in 1-2 weeks.

I haven't yet decided if I'll write a blog post about more of the things we've done in Milestone 1 before then.