Loading...
Support 200000 nodes
Maximum file size is 1024MB
Maximum peer data storage is 1 TB
Total network capacity = 1TB * 1024 * 1024 * 200000 = *** MB of storage
Typical peer network speed for upload/download is 1MBit/sec
Total network speed is = 200000 * 1 / 8 = ** Mbyte/sec
def joinNetwork(networkId: String)
def leaveNetwork(networkId: String)
def listNetworks(): List[String]
def broadcastMessage(message: String)
def discoverPeers(): List[Peer]
def getNetWorkId(): String
def connect(peer: Peer)
def listPeers(networkId: String): List[Peer]
def listFileURIs(): List[String]
def fileterFiles(fileFilter: String): List[String]
def hasChunks(fileMetaData: MetaData): List[FileChunk]
def requestFile(fileURI: String, peerId: Peer)
def requestFileChunk(fileURI: String, chunkNumber: Int, peerId: Peer)
def offerFile(fileURI: String): Array[Byte]
def offerFileChunk(fileURI: String, chunkNumber: Int): Array[Byte]
def listChunks(fileURI: String, peer: Peer): List[FileChunk]
def generateKeyPair(): PublicKey
def encryptFile(fileURI: String, key: PublicKey): String
def decryptFile(bytes: Array[Byte]): String
def verifySignature(data: Array[Byte], signature: String, publicKey: String): Boolean
def checkHash(data: Array[Byte], hash: String): Boolean
def compareFiles(fileURI: String, otherFileURI: String, peer: Peer): List[MerkleNode]
Because our system is all incapsulating we store all required info for a peer in Peer data table. In a non p2p network some of the data would be stored in global registries like DNS for example.
Peer : +int ID
Peer : +int NetworkId
Peer : +int IpV4Address
Peer : +int MacAddress
Peer : +String Name
Peer : +String Email
Peer : +String PrivateKey
PeerRegistry <|--Peer
PeerRegistry : +int NetworkID
PeerRegistry : +int PeerID
FileMetaData: +string URI
FileMetaData: +int FileID
FileMetaData: +int MaxChunkNumber
FileMetaData: +int ChunkSize
FileMetaData: +DateTime DateCreated
FileChunk <|--Peer
FileChunk <|- FileMetaData
FileChunk : +int ChunkNumber
FileChunk : +int PeerID
FileChunk : +int MetaDataID
FileCHunk : String CheckSum
You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...
New node joining the nwetork
1) If global peer registry is known a new node contacts it using a well known l3 address and gest a list of networks togehtere with the list of l3 addresses of peers
2) Peer uses the network of his choocing (any peer l3 address will suffice) and sends an invite message with it's own name, l3 address and network id
3) The nodes in the network get a new node joining message and confirm or reject it
4) Network ocnfirmation or rejection is achived by having an established quorm among the nodes that belong to a particular network using PAXOS or RAFT algorithm
5) When the decision has been established a new node receives confirmation
6) New nodes l3 address is propagated to all the nodes inside a network
7) New node has joined a network
1) If global peer registry is not known the node uses l3 broadcast address to discover any peers that belong to some network
2) After peer is discovered the node asks the peer what network it belongs to
3) After that a new node follows steps described in the case where peer registry is known to join a network
How file download is being done
1) Node asks peers in a network to list files by filter
2) Some of the nodes respond with the list of avaialble chunks for a file together with a metadata containing the information about how many chunks in total compose a file
3) Node downloads avaialble file chunks from peers and asks other peers for a missing chunks
4) Once all chunks are collected the node deems file download as a success
To establish a secure connection between peers
1) Uscing any l3 protocol (TCP or UDP) a node establishes a physical connection with a peer
2) Node asks for supported crypto information and aux information
3) Peer sends it's digital certificate
4) Node validates the signature in the certificate, validating the authenticity of the peer. Node also receives servers public key in the response
5) Node generates session private key, encrypts it with peers' public key and sends it to the peer
6) Peer decrypts session private key using it's own private key
7) Two parties now have a session private key that they can use for secure data encryption/decryption between themselfs only
8) A secure communication has been established
Every node essentially represents a combined database, router and fault detection mechniasm all at once. It consists of
1) A database of known peers together with their addresses
2) A database of stored files it can share with others
3) Routing database to forward packets to other peers
TO have a scalable DB spread across the whole network we use a distributed hash table structure (DHT) that maps keys to values in a distributed and efficient manner.
To prevent multiple peers sharing the same file chunk we use the following strategies:
1) File merging and data compaction using anty enthrope protocol. Every peer talks with another in the background and presents a merkle tree (not the whole data copy) to the node for sample. Using merle tree we can quickly check which part of the file is present or not present on an ode without doing a full file scan. To add fault tolerance we control how many copies of the same chunk can be present on an eighbouring peers at a time. If the number is > 1 our system is fault tolerant to failures as now if a node with ap rticular chunk crashes we can still have x > 1 nodes that hold the same chunk
To have a secure connection between a pair of nodes we use SSL which does a secure key exchange before starting sending data
Nodes check the life status of neighbours by using a heartbeat protocol. tThis is where a node sends periodic status check messages to every oother node to check if the node is alive or dead.
We don;t use permisionless node nerollment beacause we want our p2p network to be secure and compliant.
The core of decentrilized p2p system is providing redundancy and fault tolerance in the event where a central service would be a single point of failure. The tradeoff is that this setup increases latency for all the writes and reads because they have to be propagated to the other parts of the system which might not be geographically close. We can aleviate this by having writes done in async manner, but then we will run into inconsistency issues. The other tradeoff is overall complexity fo the system as a whole and difficulty in monitoring as now the system consists of many moving parts and can';t be easily represented as a one whole pice. Increasing in scalabilty by adding nodes might be more complex than in centrilized system as load management is not direct because every nodes potentially does everything and can't be specialized like in a centrilized system/.
HIgh traffic due to nodes frequently entering/leaving the network
Difficulty accessing a particular file chunk if it's not yet replicated to other nodes
Network partitioning leaving a system in semi-disconnected state and thus adding inability for users to get a particular file chunkj if they reside in partitioned network
In case of network outages it could take a long time for node to discover peers in the first place
Add prizes for joining the network and maybe provide free compute or file resources to the most prolific participants
Add global monitoring service that will show network utilization for all peers
Adaptive peer recomendation based on nodes location and network characteristics