The technical stack behind Wikipedia, 20 years of success

On January 15, 2001, Wikipedia was born, created by Jimmy Wales and Larry Sanger, the founders of the company Bomis and the online encyclopedia Nupedia.

Nupedia was at the time an encyclopedia written by experts following a very formal process, unlike Wikipedia whose spearhead would be collaboration, a philosophy that makes it a success today.

Originally, Wales and Sanger intended to profit from Wikipedia, but the growing concerns of the community regarding, in particular, the appearance of ads on the platform pushed Wales to give up his desire to monetize the encyclopedia and migrate wikipedia.com to wikipedia.org.

In 2003, Nupedia was definitively absorbed by Wikipedia, and all its content was injected into the latter, which would keep growing and expanding until it became the 13th most visited site on the Internet in 2020 (Alexa rank).

The same year, the WikiMedia foundation was created so that the management of Wikipedia (and a few other services) could be delegated and carried out independently.

The technical stack

It's interesting to see that over the years, the more new technologies appear on the web, the more it seems that "old" web technologies are no longer powerful enough to run large platforms.

And yet, you'll see that one of the most visited sites in the world is based on a "simple" PHP / MySQL stack!

Let's go over this in order, Wikipedia runs on a free and open-source platform called MediaWiki developed specifically for the launch of Wikipedia.

This platform is developed in PHP, with the majority of the codebase running on PHP version 7, and uses MySQL as its database.

Note that some of Wikipedia's database servers run on MariaDB, a free and open-source fork of MySQL (the latter being owned by Oracle).

Article search is performed using Elastic Search, the "caching" strategy uses Apache Traffic Server (ATS) as well as Varnish, and the application server is an Apache server.

In 2015, the WikiMedia foundation had 520 servers in their Ashburn datacenter, not counting servers located elsewhere for data redundancy.

These servers mostly run on Linux Debian, but some of the oldest machines run on Ubuntu (perhaps no longer the case today).

You can find some of this information on the dedicated Wikipedia page.

A few statistics

A technical stack is interesting, especially if we can get some data on the average load it's capable of handling, and fortunately WikiMedia is very transparent about this:

• More than 55 million articles worldwide (i.e. 228 million pages in total)
• Around 3 billion edits in total
• More than 93 million registered users
• The site is available in 317 languages (note that Wikipedia is not the most translated site in the world, that would be jw.org with 1025 different languages)
• The text database of the articles (without revisions) for the English version weighs 78GB (and can be downloaded right here), the complete version of the English database weighs more than 10 TB.
• In 2020, en.wikipedia.org totaled 94,313,066,229 page views! (source)

In 2020, the foundation employed more than 300 people with an annual revenue exceeding 100 million dollars, mainly based on donations.

Fun fact

Wikipedia is a battlefield for many bots, automating article edits on sometimes controversial subjects. Some of them get stuck in loops, modifying the same data over and over, sometimes for several years.

An article from The Guardian talks about it, I invite you to read it if you're interested!