Microservice Magic

Photo by Teng Yuhong on Unsplash

The first law of distributed systems is, don’t distribute! - M. Fowler

If you still feel the need to consider moving to a distributed architecture then microservices is a good alternative to a monolith.

Why microservices?

Robust and Stable

Microservices encourages a design where components are highly cohesive and have low coupling. This is an implicitly stable design.

Scalable

A monolith needs to be scaled as one entity, in a microservices architecture, every service and its underlying database can be scaled independently.

Ease of deployment

Modifying a monolith in production is like changing the wheels of a car as its moving. Deploying changes in a microservices architecture is more like a tire change at a F1 pitstop.

Teams can quickly make changes, fix bugs without stepping on each other teams toes.

Why are organisations adopting Microservices?

Here is a quote from Sam Newman that pins this down.

...the single biggest reason that organisations adopt microservices is to allow for more developers to work on the same system without getting in each others way

Should you use Microservices?

Are you a small team?

There is a vast ecosystem of great open source tooling that can help you setup your own microservice architecture. However, if you are a small team, its probably not a good idea to “roll your own”.

We made this mistake. Our log aggregation SaaS was initially built with a self assembled stack that was really a distributed monolith. We spent weeks on plumbing and reliability rather than on features that delivered actual business value. Juggling all the moving parts in our stack took a lot of effort.

Small teams should really invest time on the business value of their software and use managed services by AWS, GCP or Azure to take care of the microservices infrastructure.

Are you a large team?

If you are a large team and having difficulties or even teething pains with your monolith, then it would be a good idea to consider incremental steps towards microservices. One of the best analogies, I’ve heard here is to think of microservices as a volume dial. Changes should be introduced step by step and not 0 to 10.

Key design decisions for modelling microservices in the next post.

🖖

The monolith architecture

Photo by Carl Nenzen Loven on Unsplash

Think of a monolith architecture as a system that needs to be changed and deployed as a single entity. Development teams work on different modules of this entity, Testing teams run tests on the system and then it is eventually deployed as a single entity to users.

Contrary to what many think, a monolith is a great way to build software. It is simple to develop and easy to monitor, debug and test. There is strength in its simplicity.

However, there are downsides as well. It is hard to change a single module independent of other modules. It can be difficult to scale, as the database can be tricky to decompose.

I have seen teams work on monolith software that was in large parts ‘spaghetti’ mixed with a healthy portion of legacy code that was never refactored. This monolith required massive effort just to get it work together more modern software.

Distributed Monolith? Just say no

One situation that you want to avoid is a distributed monolith. This is a monolith where teams have managed to separate their software into modules, however the modules are still tightly coupled. This means that you cannot change, deploy and release software modules independently. If thats the case, then your team is not really gaining any benefit from a distributed system. Just the overheads.

I must emphasise again that a Monolith does not automatically imply a poorly designed, legacy system. Many successful startups like Netflix, Airbnb etc. all started with Monolith Architectures. Basecamp is a great example of a well executed and intentional monolith.

Distributed systems and specifically Microservices are often seen as an alternative to Monoliths. More on that and how you should approach such a transition in the next post.

Air Quality, Athletes and the Hong Kong Marathon.

Like running 🏃🏻‍♂️🏃🏼‍♀️? Read this.

We look at how athletes are affected by air quality and our experience from the Hong Kong Marathon.

Air Quality and Athletes

There are 3 main reasons why athletes are at an increased risk from air pollution. source

1. Increased ventilation. During training an athlete can consume 20 times more air than a normal person at rest.

2. A greater portion is inhaled via the mouth bypassing the normal nasal filtering mechanisms.

3. A higher intake velocity means the deeper parts of the respiratory tract are affected.

Here are some studies that have even collected empirical evidence of how athletes performance is affected by air pollution.

• Ozone is particularily damaging. People achieved a lower aerobics score on days with high ozone, Cakmak et.al.

• Football performance dramatically reduced due to high PM in German stadiums Lichter et.al.

• Inhalation during exercise and rest Leandro et.al

a traditional 30-min of moderate aerobic exercise session might induce inhalation of high levels of pollutants when performed at dirtiest cities.

Minute ventilation (or air intake per minute, typically 6 litres) was compared in cities categorised by WHO as clean (white bar) or dirty (black bar). Notice the dramatically high jump in inhaled PM in polluted cities.

Major cities in Europe may not face high PM levels however the levels of NO2 have been found far in access of WHO standards.

image source: https://pubmed.ncbi.nlm.nih.gov/30018189/#&gid=article-figures&pid=figure-3-uid-2

The Hong Kong Marathon

Every year over 70,000 athletes take part in the Hong Kong Marathon. The high humidity and difficult course, makes it one of the most challenging courses.

Image: worldsmarathons.com

However, like most major cities around the world, Hong Kong is not immune to air pollution. The government listened to the voices of the athletes and people and took concrete measures.

Starting 2013, civic organisations started to monitor air quality during the Marathon and tried to investigate the “on track" air quality.

Image: Hong Kong Marathon, West Kowloon highway under heavy air pollution, 2006.

Source: Wikipedia. (CC BY-SA 3.0)

Real time "on-the-track" Air quality monitoring

Is it safe to run?

The City university of Hong Kong and the Hong Kong Environment Protection Department collaborated to release a real time Air Quality Index (AQI) for the Hong Kong Marathon.

This was the first such measurement with a completely new generation of monitoring devices specifically developed for the project. It was able to accurately measure the AQI 'on the track'.

This real time AQI data was released by the sponsor, Standard Chartered together with the current weather information along the race track.

The basic concept was that people could monitor the air quality in real time to determine whether to continue in their competition or not.

Sheng Ye, Research Scientist at Signify GmbH was a staff member for this project.

One of the critical segments of the race was the western harbour tunnel. Let's look at that as an example.

The undersea western harbour tunnel

Throughout the Hong Kong Marathon route, the "critical" path is the long-path undersea Western Harbour Tunnel. Tunnels are commonly recognised as an air pollutant "trap", especially the undersea ones. Many air pollutants as well as CO2 are heavier than air, and get trapped in tunnels and accumulate within.

The Hong Kong government closed the tunnel traffic ahead of time and used mechanic ventilation to significantly improve the in-tunnel air quality for participants.

Here is an image from our real time AQI measurements for the undersea tunnel. Clear reductions of several air pollutants can be observed during the fan-on period, while participants ran through the tunnel.

Image: https://www.mdpi.com/sensors/sensors-16-00211/article_deploy/html/images/sensors-16-00211-g010.png

Sun, L.; Wong, K.C.; Wei, P.; Ye, S.; Huang, H.; Yang, F.; Westerdahl, D.; Louie, P.K.K.; Luk, C.W.Y.; Ning, Z. Development and Application of a Next Generation Air Sensor Network for the Hong Kong Marathon 2015 Air Quality Monitoring. Sensors 2016, 16, 211. https://doi.org/10.3390/s16020211

Summary

Athletes are a high risk group when it comes to air pollution. If you frequently run long distances in urban areas, it helps to keep an eye on the real ground level air pollution along your route and the ozone levels for the day. Predictions as offered by many weather apps are often not 'ground level accurate' so do keep that in mind if you rely only on them.

Finally, large sporting events like the olympics and city marathons are an opportunity for stakeholders to apply large scale measures to improve air quality for the entire community. For the long run :-)

The city of Beijing has demonstrated this is possible by reducing the pollution by 30% during the 2008 Olympic games. Let's see what Paris has in store for 2024.

Authors:

Sheng Ye

Sanjay K.