“When you have IPFS, you can embark looking at everything else te one specific way and you realize that you can substitute it all” — Juan Benet
A Less Technical Treatment to IPFS
from John Lilic
This section will attempt to provide a high level insight spil a prelude to my colleague, Dr. Christian Lundkvist’s, deep dive technical summary below.
IPFS began spil an effort by Juan Benet to build a system that is very quick at moving around versioned scientific gegevens. Versioning gives you the capability to track how states of software switch overheen time (think Git). IPFS has since become thought of spil the The Distributed, Voortdurend Web , “IPFS is a distributed opstopping system that seeks to connect all computing devices with the same system of files. Ter some ways, this is similar to the original aims of the Web, but IPFS is actually more similar to a single bittorrent swarm exchanging git objects. IPFS could become a fresh major subsystem of the internet. If built right, it could complement or substitute HTTP. It could complement or substitute even more. It sounds crazy. It is crazy.”
At its core, IPFS is a versioned verkeersopstopping system that can take files and manage them and also store them somewhere and then tracks versions overheen time. IPFS also accounts for how those files budge across the network so it is also a distributed opstopping system.
IPFS has rules spil to how gegevens and content stir around on the network that are similar ter nature to bittorrent. This opstopping system layer offers very interesting properties such spil:
– websites that are downright distributed
– websites that have no origin server
– websites that can run entirely on client side browsers
– websites that do not have any servers to talk to
Instead of referring to objects (pics, articles, movies) by which server they are stored on, IPFS refers to everything by the hash on the verkeersopstopping. The idea is that if ter your browser you want to access a particular pagina then IPFS will ask the entire network “does anyone have this verkeersopstopping that corresponds to this hash?” and a knot on IPFS that does can terugwedstrijd the verkeersopstopping permitting you to access it.
IPFS uses content addressing at the HTTP layer. This is the practice of telling instead of creating an identifier that addresses things by location, we’re going to address it by some representation of the content itself. This means that the content is going to determine the address. The mechanism is to take a opstopping, hash it cryptographically so you end up with a very puny and secure representation of the opstopping which ensures that someone can not just come up with another opstopping that has the same hash and use that spil the address. The address of a opstopping te IPFS usually starts with a hash that identifies some root object and then a path walking down. Instead of a server, you are talking to a specific object and then you are looking at a path within that object.
HTTP vs. IPFS to find and retrieve a opstopping
HTTP has a nice property where te the identifier is the location so it is effortless to find the computers hosting the opstopping and talk to them. This is useful and generally works very well but not te the offline case or ter large distributed screenplays where you want to minimize geyser across the network.
Ter IPFS you separate the steps into two parts,
- Identify the opstopping with content addressing
- Go and find it — when you have the hash then you ask the network you’re connected to ‘who has this content? (hash)’ and you connect to the corresponding knots and download it.
The result is a peer to peer overlay that gives you very prompt routing.
To learn more, observe the Alpha Movie.
IPFS by Example
from Dr. Christian Lundkvist
A technical Examination and IPFS (InterPlanetary Opstopping System) is a synthesis of well-tested internet technologies such spil DHTs, the Git versioning system and Bittorrent. It creates a P2P swarm that permits the exchange of IPFS objects. The totality of IPFS objects forms a cryptographically authenticated gegevens structure known spil a Merkle DAG and this gegevens structure can be used to monster many other gegevens structures. Wij will ter this postbode introduce IPFS objects and the Merkle DAG and give examples of structures that can be modelled using IPFS.
IPFS is essentially a P2P system for retrieving and sharing IPFS objects. An IPFS object is a gegevens structure with two fields:
- Gegevens — a blob of unstructured binary gegevens of size <, 256 kB.
- Linksaf — an array of Verbinding structures. Thesis are linksaf to other IPFS objects.
A Verbinding structure has three gegevens fields:
- Name — the name of the Listig.
- Hash — the hash of the linked IPFS object.
- Size — the cumulative size of the linked IPFS object, including following its linksaf.
The Size field is mainly used for optimizing the P2P networking and we’re going to mostly overlook it here, since conceptually it’s not needed for the logical structure.
IPFS objects are normally referred to by their Base58 encoded hash. For example, let’s take a look at the IPFS object with hash QmarHSr9aSNaPSR6G9KFPbuLV9aEqJfTk1y9B8pdwqK4Rq using the IPFS command-line implement (please attempt this at huis!):
The reader may notice that all hashes start with “Qm”. This is because the hash is te actuality a multihash, meaning that the hash itself specifies the hash function and length of the hash te the very first two bytes of the multihash. Ter the examples above the very first two bytes ter hex is 1220, where 12 denotes that this is the SHA256 hash function and 20 is the length of the hash ter bytes — 32 bytes.
The gegevens and named linksom gives the collection of IPFS objects the structure of a Merkle DAG — DAG meaning Directed Acyclic Graph, and Merkle to represent that this is a cryptographically authenticated gegevens structure that uses cryptographic hashes to address content. It is left spil an excercise to the reader to think about why it’s unlikely to have cycles ter this graph.
To visualize the graph structure wij will visualize an IPFS object by a graph with Gegevens te the knot and the Linksom being directed graph edges to other IPFS objects, where the Name of the Listig is a label on the graph edge. The example above is visualized spil goes after:
Wij will now give examples of various gegevens structures that can be represented by IPFS objects.
IPFS can lightly represent a verkeersopstopping system consisting of files and directories
A puny opstopping (<, 256 kB) is represented by an IPFS object with gegevens being the opstopping contents (plus a petite header and footer) and no linksom, i.e. the linksom array is empty. Note that the opstopping name is not part of the IPFS object, so two files with different names and the same content will have the same IPFS object representation and hence the same hash.
Wij can add a petite opstopping to IPFS using the directive ipfs add:
Wij can view the opstopping contents of the above IPFS object using ipfs cat:
Viewing the underlying structure with ipfs object get yields:
Wij visualize this verkeersopstopping spil goes after:
A large verkeersopstopping (>, 256 kB) is represented by a list of linksaf to opstopping chunks that are <, 256 kB, and only minimal Gegevens specifying that this object represents a large verkeersopstopping. The linksaf to the verkeersopstopping chunks have empty strings spil names.
A directory is represented by a list of linksom to IPFS objects signifying files or other directories. The names of the linksaf are the names of the files and directories. For example, consider the following directory structure of the directory test_dir:
The files hello.txt and my_file.txt both contain the string Hello World!\n. The verkeersopstopping testing.txt contains the string Testing 123\n.
When indicating this directory structure spil an IPFS object it looks like this:
Note the automatic deduplication of the verkeersopstopping containing Hello World!\n, the gegevens ter this opstopping is only stored ter one logical place ter IPFS (addressed by its hash).
The IPFS command-line device can seamlessly go after the directory listig names to traverse the verkeersopstopping system:
IPFS can represent the gegevens structures used by Git to permit for versioned verkeersopstopping systems. The Git commit objects are described ter the Git Book. The structure of IPFS Commit object is not fully specified at the time of this writing, the discussion is ongoing.
The main properties of the Commit object is that it has one or more linksom with names parent0, parent1 etc pointing to previous commits, and one verbinding with name object (this is called tree ter Git) that points to the opstopping system structure referenced by that commit.
Wij give spil an example our previous opstopping system directory structure, along with two commits: The very first commit is the original structure, and ter the 2nd commit we’ve updated the verkeersopstopping my_file.txt to say Another World! instead of the original Hello World!.
Note also here that wij have automatic deduplication, so that the fresh objects te the 2nd commit is just the main directory, the fresh directory my_dir, and the updated opstopping my_file.txt.
This is one of the most arousing use cases for IPFS. A blockchain has a natural DAG structure ter that past blocks are always linked by their hash from straks ones. More advanced blockchains like the Ethereum blockchain also has an associated state database which has a Merkle-Patricia tree structure that also can be emulated using IPFS objects.
Wij assume a simplistic monster of a blockchain where each block contains the following gegevens:
- A list of transaction objects
- A listig to the previous block
- The hash of a state tree/database
This blockchain can then be modeled te IPFS spil goes after:
Wij see the deduplication wij build up when putting the state database on IPFS — inbetween two blocks only the state entries that have bot switched need to be explicitly stored.
An interesting point here is the distinction inbetween storing gegevens on the blockchain and storing hashes of gegevens on the blockchain. On the Ethereum toneelpodium you pay a rather large toverfee for storing gegevens te the associated state database, te order to minimize bloat of the state database (“blockchain bloat”). Thus it’s a common vormgeving pattern for larger chunks of gegevens to store not the gegevens itself but an IPFS hash of the gegevens ter the state database.
If the blockchain with its associated state database is already represented te IPFS then the distinction inbetween storing a hash on the blockchain and storing the gegevens on the blockchain becomes somewhat blurred, since everything is stored ter IPFS anyway, and the hash of the block only needs the hash of the state database. Te this case if someone has stored an IPFS verbinding te the blockchain wij can seamlessly go after this verbinding to access the gegevens spil if the gegevens wasgoed stored te the blockchain itself.
Wij can still make a distinction inbetween on-chain and off-chain gegevens storage however. Wij do this by looking at what miners need to process when creating a fresh block. Ter the current Ethereum network the miners need to process transactions that will update the state database. To do this they need access to the total state database ter order to be able to update it wherever it is switched.
Thus ter the blockchain state database represented ter IPFS wij would still need to tag gegevens spil being “on-chain” or “off-chain”. The “on-chain” gegevens would be necessary for miners to retain locally te order to mine, and this gegevens would be directly affected by transactions. The “off-chain” gegevens would have to be updated by users and would not need to be touched by miners.
by Dr. Christian Lundkvist, Director of Engineering, and John Lilic, ConsenSys Enterprise