As in most companies, at the end of the year everything seems to slow down. December is the month of closing budgets, focusing on platform stability and starting of holidays season that results with a pause in release cycles. Last year, we took this time and allowed ourselves to do a small experiment, a fun engineering project, which we affectionately called PIGS and I would like to share with you its results.
A few years ago, one of the product teams released Portable Infoboxes. PI’s are the tables that usually contain rich metadata related to the article topic, they have key — value table structure, are usually placed in the top right corner of article and are built using wikitext templates. If you’d take a closer look you would notice th…
Read more >
Athena is a convenient big data query engine offered by AWS. It quickly got adopted in our company by multiple departments, however, its serverless nature has also created the risk of our Athena bill getting out of hand.
Our data set contains terabytes of data and it’s easy to write a very expensive query. The users can avoid such a scenario by using partitions which allow them to limit the amount of accessed data to only the time period of interest. However, when working on a report in an external tool or implementing some complex queries, it’s easy to make a mistake or forget about partitioning. Also, external tools often do not show the amount of data scanned, so it’s not so easy to keep a track of it.
We decided to create a tool which wi…
Read more >
At FANDOM we believe our applications should be resilient, that is they should be able to recover from failures and while the failure occurs still be useful for the users. Health checks are one of the ways to make applications self-healing.
The idea is that the health checks detect when an instance of the application reaches an invalid state. A supervisor monitors those health checks and attempts to fix any issues. Depending on your platform the action might be different.
Many frameworks (like Dropwizard and Spring) and platforms (like Kubernetes and Marathon) encourage you to implement health checks for your web applications.
When a health check fails, AWS will stop sending requests to that instance. That’s pretty straightforward — we suspec…Read more >
As the GDPR deadline has come and past, our team has reflected on its efforts of building the front-end library. The task was to build a front-end modal which FANDOM would integrate across its web products. For obvious reasons (a $20 million fine), we set a strict zero tolerance level for any failures. Our goal from the outset was to create the most well tested piece of code at FANDOM.
The first problem we had to tackle was being able to reliably detect where in the world the user currently viewing a given page is visiting from. Our network provider Fastly provides a Geo Detection service through its Varnish servers. This exposes a wealth of information, but most importantly, an ISO 3166-2 country code associated with the client IP address.…
Read more >