Created by young technologist Juan Benet, the InterPlanetary File System or IPFS is a revolutionary idea that can disrupt and decentralise the world of file-sharing in unimaginable ways. No wonder it’s irking storage monopolies and censor-happy regimes.
There are two internets: centralised and decentralised. If you’re new to the game, here’s a short guide. Centralised internet is where the network revolves around a single server. All the key processing works go to a single server. When a user connects to the network, she basically sends a request to the central server and gets things done. The user doesn’t perform the action directly. Need examples? Think about accessing Gmail, or most Google services or most popular applications and storage systems that you can think of now.
Now, let’s start explaining decentralised internet or interconnected networks with an example: the bitcoin. In a decentralised network, work is shared by more than one machine. Here we don’t really depend on a single central server. Most tasks are distributed and, hence, there is shared responsibility and ownership.
So far, the story of the Internet has been that of centralised networks. Big companies, with their central servers and server farms, have controlled access, storage and distribution of data.
Decentralisation was a taboo, at least until a few years ago. Change arrived when a bunch of technologists sitting in different corners of the world started thinking about breaking the tenets. They argued that decentralised networks could offer more reliability, scale and privacy. They make everyone accountable for every action performed on the network. Shared ownership means shared responsibility.
But that’s easier said than coded
Decentralisation means placing the collective before the self. Profits take a backseat on most occasions. Freedom gets priority over control and micromanagement. Decentralisation leaves less room for manipulation and surveillance. It is anchored on transparency and accountability. Take the bitcoin. Created as a response to the global financial meltdown of 2008, the decentralised digital currency is powered by blockchain, which is a decentralised, distributed and public digital ledger that features open records or blocks.
The fact that decentralised networks make failures rare, considering that they are not dependent on single servers to handle all the work, makes them attractive to new generation businesses. Decentralized networks scale faster. Want more power, add more machines.
Several similar products and services came up as a result of the open internet movement. Today, there is some consensus on why the world will be better off without the centralised internet and technologies. Supporters of the decentralised internet include the inventor of the World Wide Web, Tim Berners-Lee; the father of the internet, Vint Cerf; Brewster Kahle, founder the Internet Archive; and more.
All roads lead to the centre
Even though there has been a healthy shift towards decentralisation, the exponential growth in the business of centralised internet and the immense wealth and political clout that the Big Tech commands pose gargantuan challenges to the healthy evolution and expansion of decentralised web. The pushback is quite astonishing. It takes immense courage, time and efforts today to make an impact with an open or decentralised technology product.
One of the major areas of technology where centralisation still plays a commanding role is storage. Earlier, the world of computing was not generating much data. Whatever was created was promptly stored locally, using legacy storage (local hardware, etc.). But with the advent of cloud computing and storage, the game has been tilted heavily in favour of the companies. Storing and distributing data is an activity that is monopolised by storage giants. From Google to Apple to Amazon to Oracle and Microsoft, all tech giants are big players in the centralised cloud market. No doubt, storing and sharing data is at the mercy of the Big Tech now.
The InterPlanetary File System wants to change all that for good. Originally written by a young technologist from Stanford, Juan Benet, the IPFS entered the world of computing and file sharing in 2015. It is an open-source project made by Protocol Labs, which Benet founded. For starters, this is a peer-to-peer distributed file system, which aims to link all computing devices with the same system of files. Like the world wide web? Yes, it is like the web in some ways.
But the IPFS has many differences to the web. In a way, the IPFS is the bitcoin of the world wide web. According to Benet, the IPFS is more like a giant BitTorrent swarm. Users of the ‘torrent’ would know a swarm as a pool of people sharing a torrent file. IPFS is a similar system. It helps you exchange digital objects within one Git repository (a kind of file vault). Several dozen developers work for IPFS now.
How does it actually work?
To understand this, we need to look at how we share data online. In computing, people (machines) follow several rules when they share data (files) or interact with each other over the web. Geeks call them the communication protocols. Each protocol signifies clear-cut tasks. Systems and networks interact with each other based on these protocols while exchanging data. When you’re sending an email, downloading or uploading music, streaming Netflix, you follow these protocols. Your browsers, machines, internet service providers, etc. understand and respect these protocols to make information delivery easier and hassle-free.
A prominent protocol being used today is the Hypertext Transfer Protocol (HTTP). You can spot this right at the start of the address (URL) of the website you’re reading now. Now, there are two kinds of networks at play here. The first is a client-server network and the other is the peer-to-peer network. When you use Gmail, you’re entering a client-server network. When you’re downloading content from Torrent, you’re in a peer-to-peer network.
In a client-server environment, the server agent knows everything about the data shared. This includes the location, address of the parties involved and more. The server controls this operation entirely. This also means these activities can be manipulated if needed. This makes such interactions less private and less secure.
But haven’t things been hunky-dory all this while?
Yes. That’s because the HTTP is pretty good with sharing small amounts of data (text and photos). If you’ve been using the internet for sharing files you’d know that if you want to send an email with a large file, it is pretty difficult. You need to use a different client or system for that. If you remember, one of the factors that made Torrent popular was its ability to share huge files over the web easily and in a decentralised way.
With the arrival of high-speed computing, the amount of data being generated every moment has increased exponentially. Now, we stream 4K videos over the web; big data applications crunch gargantuan amounts of data every nanosecond; the growth in the number of web-connected appliances (Internet of Things), etc. make data sharing a big deal today. Given the value being attributed to storing and sharing data, centralised internet becomes a problem as it helps a few companies, especially the cloud tech giants, monopolise the entire data ecosystem.
What’s new about IPFS?
The IPFS moots an alternative.
Mainly, the IPFS has a few interesting and unique compartments. It uses something called Distributed Hash Tables or DHT. This means every piece of data is shared across a network of computers. A hash table is a data structure. Here, this decentralised data is coordinated and is made searchable for all. The nodes that store the data are not centrally coordinated. When one node fails, the same data can be accessed via another. This means the system is more reliable.
Like BitTorrent, the IPFS handles millions of nodes. A new data exchange protocol helps it perform the task efficiently. The protocol, BitSwap, enables the exchange of data, based on a peer-to-peer storage system built on IPFS, called Filecoin. The IPFS also uses something called Merkle DAG (in blockchain tech, a Merkle tree is a structure that allows for efficient and secure verification of content in a large pool of data). A Merkle DAG is a mix of a Merkle Tree and a Directed Acyclic Graph (DAG).
Minus jargon, this means the system ensures that what we share over IPFS is clean and not corrupt. Another interesting fact is that since IPFS does not need access to Internet protocols, like HTTP, users can distribute data in networks that are built on another network (eg. VPNs, services like Skype, etc.).
Together, these faculties make the IPFS a powerful and modern tool for decentralised data sharing. Obviously, governments and big technology companies are not going to like it as it allows for open access to content and leaves little room for censorship. Elements such as self-certifying file systems or SFS take away the need for special permissions for data exchange. This saves time and energy and makes sharing data over the IPFS is like sharing data over a local network: secure and fast.
You will be hearing a lot about the IPFS in the months to come, especially given the heated global debate over Big Tech monopolising data sharing and storage and how humanity is going to pay a hefty price for that. As they say, watch this space.