KBC is known as one of the largest banks in Belgium. They have a mobile application for banking, payments, ... Over the years, KBC has added several services to their online platform, including Goal Alert. With this feature, they are obviously pleasing all of their soccer fans.
The backend of Goal Alert had one big mission: provide football data to our mobile app and withstand peaks of thousands of requests when a highlight happens during a game. The following architecture was created to serve as a stable and scalable solution.
Caching plays a big role in our backend solution. To ensure we can keep up with the requests during a game, most of the football data is cached in a Redis database. When an API endpoint is called, most of the time data is fetched from Redis instead of our Postgres database. But we even go one step further, all requests pass an AWS API Gateway. This gateway triggers a Lambda function which also checks if it can give the result back from caching, so in many cases, a request from a user doesn’t even hit our backend.
In order to process all the football data coming in from external services, we run several workers that do the hard work. Queues (in our case bull, a Redis-based queue for Node) are filled with jobs and are picked up one by one by an available worker. These workers are completely separated from the API layer. In case the load goes up by the workers, it won’t interfere with the API layer and vice versa.
If everything goes well we have happy football enthusiasts in our app, but what if something goes wrong? We try to monitor as much as we can in our backend by sending crucial information and error messages to New Relic hosted by a third party. A game starting too late, a highlight or summary video that didn't come through? These are all events that are monitored and will raise an alert in NR to ensure we are on top of problems and provide the smoothest user experience possible.
In addition to the API layer, our backend provides to the mobile app, it also functions as an aggregator between different external services.
OPTA
OPTA delivers all the football data that the mobile app requires: players, goals, cards, fouls, … all this data is available in XML feeds and is sent to an endpoint that we provide in our backend. All files are kept in our database to make sure we don’t miss any data when parsing the XML files must go wrong. Parsing the XML files goes through a queuing process and is picked up by several workers to process the data as fast as possible.
WSC
WSC delivers the video material for the Goal Alert app. For every highlight (red cards, goals, …) we receive a small clip of the event. At the end of each game, we also receive a summary video.
BrightCove
BrightCove is a video platform where you can host your own video content. We use BrightCove to securely save the videos WSC delivers. These videos are made available on a CDN of BrightCove which ensures good performance through its caching features. Also, a thumbnail of the video is automatically created when we upload the video.
KBC API layer
KBC has exposed an API layer that provides our backend with 2 services. The first service gives our backend a connection for authenticating KBC users. The second service is used to send push notifications to mobile users and inform them about highlights for their favorite team or game.
The Goal Alert backend is hosted on Heroku, a cloud-based Platform-as-a-Service. Since we have to withstand a serious amount of traffic during a game, we have multiple dynos running. Based on the metrics we receive from Heroku and other monitoring tools, we use a horizontal scaling strategy to make changes to our current capacity.
Also, our Postgres database is highly scaled and has multiple followers to ensure there is no bottleneck in fetching data from our database. As indicated in the caching layer topic, we make use of serverless Lambda functions hosted on AWS to execute scripts behind our API Gateway. No setup of servers or operating systems ensures our developers can solely focus on code and functionalities that make the difference.