4 x 2.5" HDD RAID5 RASPBERRY PI
STORJ FARMING
very simple software setup
low-intermediate hardware build
For more info of what the STORJ network is, take a look at https://storj.io/
The guide is in two parts, a brief hardware overview, and a full software installation guide.
This guide last updated 22nd December '17. (Added new Raspbian OS "Stretch" and 2 GB Swapfile)
The guide is in two parts, a brief hardware overview, and a full software installation guide.
This guide last updated 22nd December '17. (Added new Raspbian OS "Stretch" and 2 GB Swapfile)
HARDWARE:
So, there are many projects in the crypto-currency scene that either require a large amount of storage for a blockchain or alternatively can rent out storage for income (Sia/Storj). And others like Burst actually mine on HDD. My thinking was that 2nd hand laptop HDD's would be the most compact but most importantly the cheapest method of storage. Also 2.5" SATA only requires a 5v input. (3.5" HDDs require 5v and 12v inputs) making laptop drives readily compatible with USB. If only there were adaptors for this.....There are! and they're very cheap also. |
This prototype is constructed using 4 x 40GB drives. As this is an experiment I wasn't going to go nuts with my overheads. They were £3.50 each from CEX which conveniently gives a 24 month warranty which may come in useful if this is a success and would take most of the risk away from farming data on 2nd hand drives.
In theory, if it works with the cheap drives, it's just a case of scaling them to 2TB or 4TB drives with no additional software or hardware requirements. Just plug and play.
I've gone back to my trusted rack method of acrylic and motherboard risers, giving good ventilation and aesthetics at low cost. It's also very easy to work with and modular for future expansion, just add another layer.
The 6A power supply provides the 5v through the white wires, and are soldered directly to a USB hub (soldered to the barrel connector pins, with the hub sandwiched between the 4 drives). I show the construction below, layer by layer.
All drives are using the pig-tail type SATA adaptor to get their power from the hub only. The +5v pin on the USB to the Pi has been disconnected to prevent back-feeding. The USB hub is 7 port and so is also powering the Pi through it's micro USB connection.
The power supply is a 5v 6A DC out, 110/220v AC in and should be sufficient to power all drives and the pi. Each drive has a power rating on it's label of approx 0.7A (varied between the brands of drive I have, .7 is an average)
In theory, if it works with the cheap drives, it's just a case of scaling them to 2TB or 4TB drives with no additional software or hardware requirements. Just plug and play.
I've gone back to my trusted rack method of acrylic and motherboard risers, giving good ventilation and aesthetics at low cost. It's also very easy to work with and modular for future expansion, just add another layer.
The 6A power supply provides the 5v through the white wires, and are soldered directly to a USB hub (soldered to the barrel connector pins, with the hub sandwiched between the 4 drives). I show the construction below, layer by layer.
All drives are using the pig-tail type SATA adaptor to get their power from the hub only. The +5v pin on the USB to the Pi has been disconnected to prevent back-feeding. The USB hub is 7 port and so is also powering the Pi through it's micro USB connection.
The power supply is a 5v 6A DC out, 110/220v AC in and should be sufficient to power all drives and the pi. Each drive has a power rating on it's label of approx 0.7A (varied between the brands of drive I have, .7 is an average)
All images are thumbnails. Click to enlarge for detail.
Summary of parts (UK sellers):
4 x 40GB 2.5" HDDs £14 CEX link here
Raspberry Pi 3b £28.79 ebay link here
16Gb SD card (class 10) £5.20 ebay link here
Naked 7 port USB Hub £7 MOD MY PI link here
2 x A3 size, 3mm thick Acrylic £6 ebay link here
15cm USB to micro USB (pi power)£1 ebay link here
4 x Sata -> USB adapters £6.80 ebay link here
5v 6A 30W DC Power Supply £8.50 ebay link here Suitable plug required extra
M3 Nylon spacers/risers sets with screws/nuts. Many used and assembled to get desired height. Multiple kits approx £8 worth. £3.86 Free delivery ebay.
Total cost for this prototype approx £90
Estimated cost for 8TB version £325
Estimated cost for 16TB version £420
**Note all second hand drives from CEX come with 24 month warranty! (I don't work for them or get commission. It's just hands down the cheapest and most reliable place for components for crypto builds)
4 x 40GB 2.5" HDDs £14 CEX link here
Raspberry Pi 3b £28.79 ebay link here
16Gb SD card (class 10) £5.20 ebay link here
Naked 7 port USB Hub £7 MOD MY PI link here
2 x A3 size, 3mm thick Acrylic £6 ebay link here
15cm USB to micro USB (pi power)£1 ebay link here
4 x Sata -> USB adapters £6.80 ebay link here
5v 6A 30W DC Power Supply £8.50 ebay link here Suitable plug required extra
M3 Nylon spacers/risers sets with screws/nuts. Many used and assembled to get desired height. Multiple kits approx £8 worth. £3.86 Free delivery ebay.
Total cost for this prototype approx £90
Estimated cost for 8TB version £325
Estimated cost for 16TB version £420
**Note all second hand drives from CEX come with 24 month warranty! (I don't work for them or get commission. It's just hands down the cheapest and most reliable place for components for crypto builds)
SOFTWARE:
Step (1) Preparing the Pi
Once again with this project the preparation steps are the same as for other nodes, I have made a beginner friendly guide with images, step-by-step that shows the basics of formatting the SD card and installing the disk-image. "Getting started" Link opens a new window.
Getting Started
Then continue below...
Once again with this project the preparation steps are the same as for other nodes, I have made a beginner friendly guide with images, step-by-step that shows the basics of formatting the SD card and installing the disk-image. "Getting started" Link opens a new window.
Getting Started
Then continue below...
Step (2) WiFi - Optional
Wifi is available once the Pi is rebooted. Either reboot now using
sudo reboot and remove the ethernet cable. Or continue, and reboot later as it is required after the next step. |
With the Pi turned on, booted and you logged in with SSH we can start the setup.
First WiFi network connections are stored: sudo nano /etc/wpa_supplicant/wpa_supplicant.conf Go to the bottom of the file and add the following text: network={ ssid="YOUR WIFI ROUTER NAME" psk="YOUR WIFI ROUTER PASSWORD" } **Note: Enter details between the quotes, leave the "" in save and exit using ctrl+O Then 'enter' ctrl+X |
Step (3) Expand filesystem
So that the Pi makes full use of the SD card you've inserted, enter:
sudo raspi-config The default password for obvious security reasons should be changed. That is done in this menu. select "1 change user password", and follow the on-screen instructions. This will be the new password when using PUTTY to SSH into the Pi. Because we installed a 2GB image onto the card, the Pi may think that the card is only 2GB in size. So we tell it to expand the file-system (returns it to it's full size allowed): select "7 advanced options" select "A1 expand filesystem" select 'finish' system will reboot, (remove the ethernet cable if you didn't at the previous step). And log back in with SSH and PUTTY once reboot is complete. |
Step (4) Updates
The image we downloaded may have an update or two since they released it:
sudo apt-get update
sudo apt-get upgrade
select 'y' to accept the storage requirement
sudo apt-get update
sudo apt-get upgrade
select 'y' to accept the storage requirement
Step (5) Configuring the HDDs to RAID5 (or any HDD configuration you prefer)
As this project is entirely storage centered there a quite a few steps, simple steps, to configure it.
First we can install a program called MDADM (Multiple Disk or Device Administration, so the wiki says?).
This will be what manages the 4 drives used here, you can use as many drives as you like if you can find a way to power and connect them!
The program is installed simply with:
sudo apt install mdadm
In install select <ok>
and "all" as the default is fine.
As this project is entirely storage centered there a quite a few steps, simple steps, to configure it.
First we can install a program called MDADM (Multiple Disk or Device Administration, so the wiki says?).
This will be what manages the 4 drives used here, you can use as many drives as you like if you can find a way to power and connect them!
The program is installed simply with:
sudo apt install mdadm
In install select <ok>
and "all" as the default is fine.
Now we have mdadm installed we can make the RAID. There are 3 commands shown to the left, first lsblk shows what USB devices are connected and recognized. Here listed as sda, sdb, sdc and sdd. (The mmcblk0 and its partitions are the SD card) Your Pi will 99% of the time list USB devices as shown to the left. Make a note of the letters a,b,c,d etc it has assigned. The next command builds the RAID drive: |
sudo mdadm --create /dev/md0 --level=5 --raid-devices=4 /dev/sda /dev/sdb /dev/sdc /dev/sdd
Here, customize as you see fit. I'll explain:
"/dev/md0" is what the cluster of drives will be labeled as once the RAID is built. You can change this if you have another preference.
"--level=5" is the level of RAID that will be built, look at Standard RAID levels, WIKI for other options, simply change the number.
"--raid-devices=4" is the number of HDDs you want mdadm to use to create your RAID. (usually all of them)
and after that, separated by spaces, is the current label of every drive you want mdadm to use. If your Pi has labelled them something different this is where you would your specific drive labels.
Once you enter your command the HDD activity lights should flicker manically as it builds the drive. It does this in the background.
To check on it's build progress before progressing to the next step we use :
cat /proc/mdstat
As shown at the bottom of that last screenshot, devices used, a % progress of the build and an estimated time of completion.
**Note this prototype is 4x40GB drives, making /dev/md0 120GB usable storage and 40GB as unusable parity. It estimates 1 Hr for the build. This setup in theory will work exactly the same if four 500GB or 2TB drives were used, but expect the build time to rise accordingly.
Next, now that we have our RAID, we should create a partition and format it.
Here, customize as you see fit. I'll explain:
"/dev/md0" is what the cluster of drives will be labeled as once the RAID is built. You can change this if you have another preference.
"--level=5" is the level of RAID that will be built, look at Standard RAID levels, WIKI for other options, simply change the number.
"--raid-devices=4" is the number of HDDs you want mdadm to use to create your RAID. (usually all of them)
and after that, separated by spaces, is the current label of every drive you want mdadm to use. If your Pi has labelled them something different this is where you would your specific drive labels.
Once you enter your command the HDD activity lights should flicker manically as it builds the drive. It does this in the background.
To check on it's build progress before progressing to the next step we use :
cat /proc/mdstat
As shown at the bottom of that last screenshot, devices used, a % progress of the build and an estimated time of completion.
**Note this prototype is 4x40GB drives, making /dev/md0 120GB usable storage and 40GB as unusable parity. It estimates 1 Hr for the build. This setup in theory will work exactly the same if four 500GB or 2TB drives were used, but expect the build time to rise accordingly.
Next, now that we have our RAID, we should create a partition and format it.
This then adds the partition layer (p1) to each drive when lsblk is used again.
Use:
sudo mkfs.ext4 /dev/md127p1
To format the new partition to ext4. (Again change md127p1 to what ever your partition is called) Use command lsblk for your label)
Use:
sudo mkfs.ext4 /dev/md127p1
To format the new partition to ext4. (Again change md127p1 to what ever your partition is called) Use command lsblk for your label)
Because mount commands must be run as "sudo" superuser it has admin permissions. To be safe and avoid any problems we change the owner to "pi" or whatever your username is, with: sudo chown ${USER} -R /home/${USER}/storj |
You'll notce the MOUNTPOINT column is empty.
First make a folder to moun the partition to: mkdir storj Mount the partition for our use by entering sudo mount -o rw,auto /dev/md127p1 /home/${USER}/storj An explanation: -o rw,auto (-o,gives options, rw means read/write permissions, auto should automount) /dev/md127p1 (what to mount, enter the md<num><p1> you get from lsblk command) /home/${USER}/storj (where to mount it) |
Our drive is completly good to go! MOUNTPOINT lists the partition as mounted to our desired destination!
Step(5) Installing "Screen"
Screen is a very useful program. It allows for multiple instances of terminal to be run, disconnected from the Pc they were started on and re-joined later. This will allow you to start the synchronization process of the blockchain for example, turn off or close the terminal window, and the sync process will continue on the Pi. Without it when you close the terminal window the process stops.
To install screen:
sudo apt-get install screen
Using screen is very easy.
The program is started using:
screen bash
This opens a new terminal window. It gives this window an ID number so we can find it later.
** For info, don't do this yet, but, to issue commands to "screen" press ctrl+a together. This enters command mode. Then pressing "d" detaches the screen to come back to later. I'll tell you when to do this further down the guide.
Screen is a very useful program. It allows for multiple instances of terminal to be run, disconnected from the Pc they were started on and re-joined later. This will allow you to start the synchronization process of the blockchain for example, turn off or close the terminal window, and the sync process will continue on the Pi. Without it when you close the terminal window the process stops.
To install screen:
sudo apt-get install screen
Using screen is very easy.
The program is started using:
screen bash
This opens a new terminal window. It gives this window an ID number so we can find it later.
** For info, don't do this yet, but, to issue commands to "screen" press ctrl+a together. This enters command mode. Then pressing "d" detaches the screen to come back to later. I'll tell you when to do this further down the guide.
Step(6) Install Storj
Most of this step from https://docs.storj.io/v1.1/docs/setting-up-storj-share-on-a-raspberry-pi
First some dependencies are needed:
wget -qO- https://raw.githubusercontent.com/creationix/nvm/v0.33.2/install.sh | bash
Close your PUTTY window with the X in the top-right corner. Open a new PUTTY session to allow the next step:
nvm install --lts
sudo apt-get install git python build-essential -y
npm install --global storjshare-daemon
Create a large swapfile:
The Raspberry Pi comes with 1GB of RAM. Whilst this is enough to run one farming node, it struggles with multiple requests. To alleviate this as best we can we can use some of the spare SD card space as an extra RAM when it's under high demand. Known as a swapfile. To set-up:
sudo nano /etc/dphys-swapfile
Brings up the screen below.
Most of this step from https://docs.storj.io/v1.1/docs/setting-up-storj-share-on-a-raspberry-pi
First some dependencies are needed:
wget -qO- https://raw.githubusercontent.com/creationix/nvm/v0.33.2/install.sh | bash
Close your PUTTY window with the X in the top-right corner. Open a new PUTTY session to allow the next step:
nvm install --lts
sudo apt-get install git python build-essential -y
npm install --global storjshare-daemon
Create a large swapfile:
The Raspberry Pi comes with 1GB of RAM. Whilst this is enough to run one farming node, it struggles with multiple requests. To alleviate this as best we can we can use some of the spare SD card space as an extra RAM when it's under high demand. Known as a swapfile. To set-up:
sudo nano /etc/dphys-swapfile
Brings up the screen below.
Just change the default size of 100(MB), to 2000 as shown to the left
This is the largest the Pi will support. Effectively now having 3GB of RAM (but it's a band-aid on the problem). save, exit, ctrl+o ctrl+x Then, to build the new swap file for use: sudo dphys-swapfile setup and sudo dphys-swapfile swapon |
This is where we differ from the main guide. I found that some folders were missing making storj unable to start due to an incorrrect path. So we make some folders:
mkdir .config
cd .config
mkdir storjshare
cd storjshare
mkdir configs
cd
storjshare create --storj=ethereum-storj-token-address --storage=/home/pi/storj
So here I have entered the command above, with my ethereum address I've created for this.
In reply it generates a config file with some generic settings. This config file has a long randomly generated file name. Shown to the left. ending in .json It also opens this file in a default text editior (for raspbian this is "Vim"), I find vim not very intuitive so I usually quit using :qw and re-open the file to edit in this next step. On quiting Vim copy and paste this file name and path and hold it in a notepad file, we need it for 2 things later. |
I then use nano to edit the configuration file we just saw. I find nano editor much more intuitive:
nano .config/storjshare/configs/37d606d4243a0d6bb876af201ef4d810d4f1637b.json
(of course change the file name to yours!)
And you'll see something like this:
nano .config/storjshare/configs/37d606d4243a0d6bb876af201ef4d810d4f1637b.json
(of course change the file name to yours!)
And you'll see something like this:
And contains all your adjustable settings.
For now you are probably only concerned with.
For now you are probably only concerned with.
Next we'll start the daemon up, (nearly there!)
storjshare daemon
After a few seconds it reads back " * starting daemon in background "
All good.
My file permissions changes again, so to be sure:
sudo chown ${USER} -R /home/${USER}/storj
And finally run the Farm! (With your config name)
storjshare start --config /home/pi/.config/storjshare/configs/cc63ffa1267a5d66cde5bdaddbfb5893ffb05559.json
storjshare daemon
After a few seconds it reads back " * starting daemon in background "
All good.
My file permissions changes again, so to be sure:
sudo chown ${USER} -R /home/${USER}/storj
And finally run the Farm! (With your config name)
storjshare start --config /home/pi/.config/storjshare/configs/cc63ffa1267a5d66cde5bdaddbfb5893ffb05559.json
To check on your farm's status use:
storjshare status
storjshare status
Seeing connections to peers is good. A low Delta figure is also good.
It then just takes time to acquire contracts.
**edit - This screenshot shows the Raspberry Pi Farm performing MUCH better than previous versions of StorjShare. Their latest releases and methods of data distribution are now making this Raspberry Pi solution far more feasible. I will continue to update this screenshot as I get better data with time to as accurately as I can reflect the success/failures of this project.
I hope this guide has been useful.
Dan
It then just takes time to acquire contracts.
**edit - This screenshot shows the Raspberry Pi Farm performing MUCH better than previous versions of StorjShare. Their latest releases and methods of data distribution are now making this Raspberry Pi solution far more feasible. I will continue to update this screenshot as I get better data with time to as accurately as I can reflect the success/failures of this project.
I hope this guide has been useful.
Dan