User:Gonnabee/sandbox/Storj DCS

From Wikipedia, the free encyclopedia



To keep things organized, please use following links:

Sources For links to sources
Discussion For discussion, only article related
 Gonnabee (Talk) 14:50, 23 January 2022 (UTC)

Storj DCS
Type of site
Decentralized Cloud storage
Available inEnglish
Predecessor(s)Tardigrade
Country of originUSA
Area servedWorldwide
OwnerStorj Labs Inc.
ProductsDecentralized Cloud storage
ServicesAmazon S3 compatible decentralized object storage
URLstorj.io
IPv6 supportNo
CommercialYes
RegistrationRequired (included in free tier layer)
LaunchedMarch 19, 2020; 4 years ago (2020-03-19)
Current statusActive

Storj (/storage/) DCS formerly known as Tardigrade (named after the famous and basically indestructible water bears) is a cloud service offered by Storj Labs Inc. that provides an Amazon S3 compatible decentralized object storage. DCS stands for Decentralized Cloud Storage.[1]

In contrast to conventional usually centralized cloud storage providers, Storj Labs Inc. does not own or operate any data center to store its customers data.[2] Instead, the Storj DCS object storage space consists of a network of currently approximately 12.800 single and geographically divers storage nodes[3]. These nodes are operated by independent individuals called Storage Node Operators (SNO) who get paid by Storj Labs Inc. for providing storage space and bandwidth for Storj DCS.

Storj Labs Inc.launched the current version (V3) of its decentralized storage initially as Tardigrade on 03/19/2020.[4] In 2021 the service has been rebranded and re-launched as Storj DCS on 04/20/2021 with a new pricing structure and a new corporate identity.[5]

History  Not done[edit]

Inception[edit]

Storj was founded in 2014 by Shawn Wilkinson after sparking an idea about disrupting cloud storage with Bitcoin technology. Together with co-founder James Lowry their presentation of a working proof of concept software demo has won 1st place in the Texas Bitcoin Conference 2014 hackathon contest and $250.000 as prize.[6][7]

Initial idea[edit]

Similar to Uber or Airbnb, Storj is a player in the sharing economy space. Like Uber drivers chauffering people in their private cars or Airbnb users renting out private rooms, Storage Node Operators rent out unused hard disk space and bandwidth of their existing predominantly residential computer and internet infrastructure.[8]

Similar services at inception Not done[edit]

By the time of Storjs inception, the idea of a distributed storage cloud operated by home users had already been adopted by:

  • Space Monkey, a startup founded in 2011 that offered its customers a 1 TB hard disk as local storage for rent. When the drive was connected to the internet, it became connected to all other Space Monkey hard drives forming a distributed storage network in which all user files placed onto the local drive would get encrypted and replicated to other remote drives, which safeguarded from data loss in case of hardware failures or disaster.[9] Space Monkey was acquired by Vivint Smart Home Inc. in 2014.[10]
  • Vivint Smart Home Inc., a public smart home company in the United States and Canada, used the Space Monkey technology after its acquisition in 2014 for storing those large amounts of data that accrue in smart homes from IoT devices and camera recordings, such as surveillance or doorbell cams.

Early concepts and implementations  Not done[edit]

Metadisk[edit]

Tardigrade V1[edit]

Tardigrade V2[edit]

Prior to the actual V3 version of the Storj DCS network, extensive experience has been gathered with version V2 in which node operators were called farmers while storage space customers were called renters. Back then the Storj storage network consisted of a system for contracting and negotiating on top of a distributed hash table (Kademlia). On this permissionless platform renters and farmers would broadcast their contract offers and then negotiate directly with interested parties to enter into a contract and execute it.

Farmers were paid through Storj token (SJCX), which at that time was a Counterparty token on top of the Bitcoin blockchain.

After extensive alpha and beta testing, developers at Storj Labs Inc. came to the conclusion to drop the existing platform and to completely rebuild the system with an improved concept. Due to a massive increase in Bitcoin fees, payment was moved away from Counterparty token to Storj ERC-20 token (STORJ) on Ethereum. This switch was made with the intention to assure the ability to process mass micropayments to a large number of node operators in the future.

Actual Product (V3)  Not done[edit]

The current version V3 is a complete rebuild after extensive experience has been gathered with version V2. The new concept geared towards better scaling and to provide Amazon S3 compatibility.

 Comment: This area could include:

  • Differences to conventional centralized providers
  • Benefits, advantages
  • Maybe also something about incidents[11] and if files have been lost so far
  • Problems, issues, disadvantages, e.g. Problem Eth fees

Technology  Not done[edit]

General overview[edit]

The Storj DCS object storage is a network of computers with different roles to allow uploads and downloads and to maintain a zero knowledge state as well as to ensure the durability and integrity of the uploaded data.

Independent node operators provide the entire storage space and up- and download bandwidth. High availability database clusters, called satellites, keep track of all data in the distributed network and of the availability and integrity of the data by constantly auditing the nodes. Repair workers maintain the redundancy of data in the network by recreating and redistributing data pieces whenever the availability falls below a specified threshold.[12]

Upload[edit]

Before an object gets uploaded to the network, it gets encrypted with AES-256-GCM symmetric encryption and split into segments. Each segment gets split into 80 pieces (shards) that are erasure coded with the Reed Solomon erasure coding scheme. These shards get uploaded to 80 different accidentally selected nodes which are located all over the world.

Download[edit]

Whenever a client requests to download data from Storj DCS, then it will download the erasure coded pieces, reconstruct segments from them and rebuild the file from the segments. Because the pieces are erasure coded, 29 shards out of 80 are sufficient to fully rebuild a complete segment.

Storage[edit]

The erasure coded pieces are stored on nodes operated by independent node operators (SNO). The node operators provide disk space and bandwidth for up- and download and get paid with Storj ERC-20 tokens for providing this service. To maintain data availability and integrity, Storj Labs Inc. is running a ledger on so called satellites, which are high availability database clusters which are located independently from each other in different areas of the world. The satellites constantly check the nodes online status, the availability and integrity of single pieces via audits to make sure that there are always enough intact pieces available to rebuild any given data segment at all times. If the number of pieces fall below a certain threshold, the repair worker servers will get triggered for re-creation and re-distribution of pieces.

Repair[edit]

To safeguard customer data against node churn, node unavailability or nodes losing data permanently repair workers will recreate and redistribute pieces, whenever the ledger suggests, that the number of pieces for a specific segment has fallen below a certain threshold. Then the repair worker server will reconstruct that segment and recreate new pieces from it which will then get redistributed to new nodes. This process assures that there are enough pieces for reconstruction for any segment any time.

Redundancy[edit]

Storj DCS ensures redundancy by erasure coding and not by data replication. Every file gets encrypted and split into segments. Every segment gets split into 80 unique erasure coded pieces from which any 29 pieces are sufficient to reconstitute the whole segment. 29 out of 80 results in an expansion factor of 2.75.

Amazon S3 compatibility  Not done[edit]

Amazon S3 API, but also compatibility in terms of performance, durability, functionality (such as native streaming), security, and privacy.

Security Model  Not done[edit]

  • Access Grants
  • Zero trust and zero knowledge Architecture  Not done
    • Zero knowledge: Nodes don't see or impact the data in any way
    • Zero trust: Anyone trying to access data is authenticated every single time they make a request

Programming language and license  Not done[edit]

The Storj DCS code is Open Source[13] and written in Golang.

Common misconceptions about Storj DCS  Not done[edit]

Storj DCS does not operate a blockchain.

The Storj token is only used to transfer payments to node operators.

No data gets stored in any blockchain in any way.

There is no data replication, all stored pieces are unique. Redundancy is achieved through erasure coding.

Usage  Not done[edit]

Use Cases[edit]

Backups, research data sets[14], video streaming[15][16], large file transfer[17], multimedia storage[18], persistent storage[19]

Adoption[edit]

Slow for enterprises therefore aiming at developers[20]

Network statistics[edit]

Storj DCS network statistics[3][21]
Date Active nodes Network capacity Free capacity Number of objects Number of pieces (shards)
01/2022 12.767 15,61 Petabytes 6,14 Petabytes 401 Million 31 Billion

(At the time of writing Storj DCS consists of 12.767 active nodes, a total network capacity of 15,61 Petabytes whereas 6,14 Petabytes are available. 401 million objects are being stored resulting in 31 billion pieces (shards).[3])  Comment: Redundant to the table and will be removed if table remains.

Aggregated real time data about the Storj DCS network is available on the Storj homepage.[3] On 08/24/2021 Storj launched the Storj DCS Public Network Statistics web page where various statistics about its Storj DCS service gets periodically updated.[22] Independent web pages use this data to publish statistical data about Storj DCS. [21][23]

Customers[edit]

Storj DCS is currently particularly aimed at developers[20] to have apps and services built on top of it. There are all kind of customers which includes: The Internet Archive[24], CIMMYT, Boonji Project, Pocket Network, PixelExperience, Filebase[25], tribesocial[26]

Integrations  Not done[edit]

Storj offer various integrations: Filezilla, Fastly, Duplicati

Regulatory constraints  Not done[edit]

Regulations maybe not ready for web3 style distributed and encrypted zero knowledge storage.

Competitors  Not done[edit]

Decentralized storage providers with own technology[edit]

Other decentralized cloud storage providers are:

information Note: Add IPFS too?

Decentralized providers based on Storj V2 technology[edit]

Centralized storage providers[edit]

Other centralized cloud storage providers are:

See also  Not done[edit]

References[edit]

  1. ^ "Introducing Storj DCS—Decentralized Cloud Storage for Developers". www.storj.io. Retrieved 2022-01-16.
  2. ^ "How Storj Is Building a Storage Cloud Without Owning a Single Disk". Data Center Knowledge. 2019-08-22. Retrieved 2022-01-17.
  3. ^ a b c d "Storj - Decentralized Cloud Storage". www.storj.io. Retrieved 2022-01-15.
  4. ^ "General Availability for Tardigrade is Here". www.storj.io. Retrieved 2022-01-15.
  5. ^ "Storj Launches Storj DCS, The Developer's Gateway to the Decentralized Internet". www.businesswire.com. 2021-04-20. Retrieved 2022-01-15.
  6. ^ "Shawn Wilkinson's entrepreneurial journey". Atlanta Tech Village. 2018-04-11. Retrieved 2022-01-15.
  7. ^ Gray, Cameron. "Storj Vs. Dropbox: Why Decentralized Storage Is The Future". Bitcoin Magazine: Bitcoin News, Articles, Charts, and Guides. Retrieved 2022-01-16.
  8. ^ "How Storj Is Building a Storage Cloud Without Owning a Single Disk". Data Center Knowledge. 2019-08-22. Retrieved 2022-01-15.
  9. ^ "Kickstarter Tech Project of the Week: Space Monkey". PCMAG. Retrieved 2022-01-15.
  10. ^ Tilley, Aaron. "Smart Home Company Vivint Just Bought Cloud Storage Startup Space Monkey". Forbes. Retrieved 2022-01-15.
  11. ^ "Storj DCS status page". Retrieved 2022-01-17.{{cite web}}: CS1 maint: url-status (link)
  12. ^ "Storj Final.mp4 | Storj DCS". link.ap1.storjshare.io. Retrieved 2022-01-15.
  13. ^ "Storj". GitHub. Retrieved 2022-01-15.
  14. ^ https://www.storj.io/documents/storj-univ-of-edinburgh-report.pdf
  15. ^ "Sintel.2010.1080p.mkv | Storj DCS". link.us1.storjshare.io. Retrieved 2022-01-16.
  16. ^ "Big_Buck_Bunny_4K.webm | Storj DCS". link.eu1.storjshare.io. Retrieved 2022-01-16.
  17. ^ "tearsofsteel_4k_DCP.zip | Storj DCS". link.eu1.storjshare.io. Retrieved 2022-01-18.
  18. ^ "tearsofsteel_4k.mov | Storj DCS". link.eu1.storjshare.io. Retrieved 2022-01-16.
  19. ^ "boonjiprojectcom | Storj DCS". link.eu1.storjshare.io. Retrieved 2022-01-17.
  20. ^ a b "Storj Crowdsources Infrastructure to Undercut Cloud Giants". Data Center Knowledge. 2021-06-29. Retrieved 2022-01-17.
  21. ^ a b "Storj DCS network data". storjstats.info. Retrieved 2022-01-15.{{cite web}}: CS1 maint: url-status (link)
  22. ^ "Storj DCS Public Network Statistics". stats.storjshare.io. Retrieved 2022-01-15.{{cite web}}: CS1 maint: url-status (link)
  23. ^ "Storj Net Info". storjnet.info. Retrieved 2022-01-15.
  24. ^ "Internet Archive". Twitter. Retrieved 2022-01-16.{{cite web}}: CS1 maint: url-status (link)
  25. ^ "Hello, Storj!". Filebase Blog. 2020-12-18. Retrieved 2022-01-15.
  26. ^ "Tribe Social Storage is now 100% Decentralized (with Storj)". tribesocial.io. Retrieved 2022-01-15.{{cite web}}: CS1 maint: url-status (link)

External links  Not done[edit]

 This article incorporates text by © copyright Blender Foundation | www.sintel.org available under the CC BY 3.0 license.

 This article incorporates text by © copyright 2008, Blender Foundation / www.bigbuckbunny.org available under the CC BY 3.0 license.

 This article incorporates text by © Blender Foundation | mango.blender.org available under the CC BY 3.0 license.

 This article incorporates text by © Blender Foundation | mango.blender.org available under the CC BY 3.0 license.

Category:Distributed data storage Category:Web3 Category:Decentralization Category:Cloud Storage

Retrieved from "https://en.wikipedia.org/w/index.php?title=User:Gonnabee/sandbox/Storj_DCS&oldid=1068238185"