NetFS

Download only what you use... share what you have

In anything you do, imagine the world is watching and act accordingly.

 
 

NetFS Peer-to-Peer Network Filesystem

Summary

This P2P file system aims to enable high-speed uploading of commonly shared data from peers to your machine as you need it.  Your disk then caches this data for your use, and for use by others when they need it.  As a second-generation protocol,  benefiting from the BitTorrent experience, file download speeds will be greatly increased, and support for live video streaming enabled.  Entire file systems will be supported, along with rudimentary version control.  ISP caching will be supported, lowering overall costs, and improving the on-line experience.

Draft Protocol Specification

The preliminary spec has been completed, and a compatible client is currently under development.  Anyone wishing to contribute should check out the following link, and feel free to join the discussion list, or e-mail me directly at bill@billrocks.org.

Draft Protocol Specification

Use Cases

  • Samsung wants to us a high-end OS on its new wifi-enabled G4 smartphone.  It only has 512 meg of flash devoted to program memory, as the remaining 15 gig is for music and other media.  So conserving file system space is important to Samsung's users.  But Samsung wants to run a high-end Ubuntu-Mobile distribution, and doesn't want to pick-and-choose between default installed packages.  NetFS allows Samsung to effectively install the whole distribution.  As users use their device, only files they actually use get downloaded (over the G4 network) and locally cached, creating a 10X improvement in the number of applications they can easily access.
  • Mary wants to try running Ubuntu on her new workstation, which is always on-line.  She hates that she will have to figure out what packages Ubuntu leaves out compared to the distribution she currently runs - Fedora.  As a high-profile coder, Mary needs many of Fedora's packages that are missing in Ubuntu's default installation.  So, she installs Ubuntu-Bloated-Edition, built on NetFS.  By default, it contains every Debian/Ubuntu package available, more than 20,000 packages.  She saves time, disk space, network bandwidth, and most importantly - she wont have to poke around and figure out what to install.
  • Project Gutenberg's e-book servers are costing the charitable organization too much money, while users experience long download delays.  BitTorrent can't help with these small files, so Gutenberg and it's mirrors bear the full cost.  Gutenberg asks users to install NetFS, and to download e-books that way.  Bandwidth costs drop, while user download speeds improve.
  • Nicole runs a popular on-line video site from her basement.  Her 40KB/sec upload speed has become a major obstacle, and she doesn't want to upgrade to a T1.  She encourages her viewers to use NetFS, and enjoys watching her user base grow by 10,000X, all driven from an old PC and a DSL line.
  • A popular ISP is getting killed with BitTorrent traffic, which keeps encrypting streams and changing ports, so traffic shaping and file caching isn't working.  The ISP encourages users to switch to NetFS for their legal file downloads, and runs a file caching server, reducing expensive external network traffic, while improving their customer's on-line experience.
  • A popular Linux distro creates a live CD for installation.  Rather than the usual install-everything, then update-all, thus re-installing most things, they put their packages in a NetFS directory.  The live CD boots, and installs the current version of all packages on the first go, eliminating the need to update after a clean install, while reducing the size of the live CD to only 64MB.  Users download packages at full bandwidth, yet cost the distro almost nothing.  While running off-line, users have full access to every package they installed, exactly as if they'd never used NetFS.  Overall installation time drops.
  • A popular file download site suffers from frequent DDoS attacks, frustrating users.  After switching to NetFS, users can still download, even when the main site is off-line.  Download times improve as well.
  • A sports enthusiast tires of poor TV coverage of local college games, and takes it upon himself to fix it.  From his cell phone, Tom broadcasts his team's game to the entire world, costing him nothing.

Benefits vs Bittorrent

  • Support for data streaming, such as live video
  • Dynamic incremental updates for whole file systems, rather than just large file downloads
  • Much faster download time for DSL/Cable users (see BTslave for reasons why)
  • Mush faster download times for large newly-released files (again, see BTslave)
  • Simpler browsing for files.  Just use a file-finder program
  • Interconnected web of file systems enables NetFS surfing, and Google-like indexing
  • Open-source spec and implementation
  • Users are generally available for file sharing when on-line, greatly improving download times and file availability
  • Friendlier ISP caching support
  • Publishers can sign entire file systems
  • Built-in revision control, and ability to switch between versions
SourceForge.net Logo

 

Download

Sourceforge.net

Forum

Billrocks.org

Dumb Ideas

Copyleft 2006 All rights approved.