Replies: 1 comment 11 replies
-
|
@stuberman |
Beta Was this translation helpful? Give feedback.
-
|
I am not sure what "replication(1)" is. I have not run lotus bench against my miner. |
Beta Was this translation helpful? Give feedback.
-
|
@stuberman Can you explain how Fils are locked up as collateral, what are the steps, commands etc. Thank you |
Beta Was this translation helpful? Give feedback.
-
|
I know about half of that answer, in reality someone needs to do some research and provide a complete answers such as how collateral gets incrementally returned or what happens when a Precommitted sector fails or a Commit fails and the sector never goes on chain for proving. I suggest you fill out a bounty form that would encourage someone to write up such a guide. The Storage Provider Working Group will review this and if approved pay for the work to be done. See Bounty Program to create an issue for review |
Beta Was this translation helpful? Give feedback.
-
|
Please I need your help. This is how i set up. before i run the lotus daemon for the first time i apply my env after download the proof my lotus run an error: ERROR: failed to start json-rpc endpoint: could not listen: listen tcp4 127.0.0.1:1234: bind: address already in use |
Beta Was this translation helpful? Give feedback.
-
|
@theasianbull You have a problem with your configuration in lotus, not a hardware prep issue. You can use Slack to diagnose your ports. |
Beta Was this translation helpful? Give feedback.
-
I am writing this guide to help people new to Filecoin mining who are interested in running a small lotus node or lotus-miner. My intention is to help people avoid some of the mistakes I made when trying to preplan the hardware and software configuration before actually installing Lotus.
There are many ways to run Lotus and my approach is not to provide the bare minimum but a reasonably small machine that will have enough resources to mine at small scale and not crash due to limited resources, Lotus code has become more efficient and some of the hardware will perform better than it did in mid 2020, while at the same time the network is also growing very quickly and putting some strain on other resources (such as chain growth).
There are a variety of platforms to build your lotus node or lotus miner, my recommendations are based on my experience using Ubuntu 20.04 server, without any GUI in order to not waste resources.
My first combined miner, that is everything was on a single machine, lotus daemon, lotus-miner and sealing worker, consisted of a x570 motherboard with AMD Ryzen 9 3900x CPU. The motherboard supports PCIe 4.0 for nice IO and the Ryzen CPU is fast and supports the needed SHA_NI extensions needed by Lotus. Lotus-miner needs 128GB ram or more plus plenty fo swap if you only have 128GB ram - so I allocated 200GB for swap. Speaking of disk, some of the processes need very fast disk including sealing disks and swap, followed by chain and proof parameters. Ideally you consider the channels used for IO to minimize contention and avoid putting all of these high utilization tasks on the same disk. You will also need an appropriate GPU with more than 8GB of ram, I chose a 1080Ti to beginning mining and it has served me well. Since mining involves sealing data into 32GiB or 64GiB sectors you not only need large sealing disks but lots of room for long term storage of those sectors. You can start with a single large HDD and add more as you grow. I began with five 14TiB Western Digital data center grade UltraStar HC530 drives. I chose them because they are rated for constant duty and are highly reliable. After formatting this gave me a capacity of 60TiB. Later I upgraded my case to a SuperMicro 24 drive bay case, and arranged the drives in 6x14TB ‘vdevs’ (virtual devices) for easy expansion. I used ZFS raidz for a single drive redundancy per vdev (yielding 60TiB per dev). I also used the ZFS feature for a hot online spare which allows additional HDDs to be used automatically if a drive fails. The key to success in Filecoin is high uptime, availability and reliability. Other aspects of my initial build were high speed Internet (1Gbps up/down) with unlimited usage - I will push 6 TB per month; and UPS to provide stable power for my machines and Internet and network. I am using several 1000VA ‘smart UPS’s’ which condition the power for good clean power and also connects to each server to perform a graceful shutdown if power is lost for an extended time. Think ahead when planning this but I know that many people will not want to invest too much to start until they really understand how the system works. For this reason I highly suggest you first practice on the test network without having to buy real FIL and waste real money. On the test network you can really tune your machine in terms of configuration, hardware, drivers, ram speed, etc.
BTW - I use the XMP 2.0 profile of my ram and do not overclock it or the CPU. Before I buy the hardware I always check the specs on the motherboard to make sure everything is compatible.
Step 1. Layout the disk arrangement to prevent headaches as you start mining: (all formatted to ext4 except swap and boot which is vfat)
Disk 1 (500 GB)
/boot partition (500MB should be more than enough)
/ (root) this is where your OS goes (10GB), perhaps also your /home for the lotus source code (10GB) and a directory for your proof parameters (158GB) - so a minimum of 200GB preferably fast disk for the proofs
/swap with only 128GB ram you will need a lot os fast swap (200GB perhaps on its own partition)
Disk 2 (1 TB)
/.lotus your chain will grow quick, as I write this it is growing at 38 GB/day, never let if fill the disk, expect to shut down the daemon and miner to reimport a pruned chain around 47 GB)
Disk 3 & 4? (2x2TB or 1x4TB)
/worker1 and /worker2 - I like to allocate 4 TiB of raw disk per 128 GB of ram - why? Because the ram limits how many sectors you can seal at the same time. Each sector occupies as much as 550GB of disk during sealing. This needs to be your fastest disk, NVMe using PCIe Gen4.0x4 is about as good as it gets. Since these disks take a beating, having two disks allows you to lose a disk and still seal even if your rate is much slower. You can use a single 4TiB disk, it all depends on your hardware, what sorts of slots you have available and interfaces. Yes, you can do a single 2TiB sealing disk… but you really won’t like how much time your miner is idling waiting for a task to finish.
ZFS pool
This is where I put my HDDs, as many as you like, you won’t need most of them until you have sealed sectors. I predict this miner will need about 50 days to seal 10TiB worth of sectors which is the minimum needed to be eligible for mining block rewards. I like ZFS because it is easy to configure and does not require hardware adapters and is Ubuntu ready. There are great resources on the Internet for tuning ZFS for maximum performance.
Step 2. Put all of that hardware together. This is the fun part. Motherboard, disks, ram, power supply, cables, CPU and fan… and that GPU...
Most of the hardware is pretty straightforward. You need to edit the /etc/fstab file to ensure all of your disks mount on boot.
But with an Nvidia GPU you need to get your hands dirty. Ubuntu likes to use the generic ‘nouveau’ driver but you really need to install the Nvidia proprietary driver. I use 455.38. Are you used to editing the GRUB boot loader? No? You are in for a wild ride. Why can’t Nvidia make this easy? I don’t know but be prepared to spend some time getting it to work, Google is your friend. You will know you are successful when you can run the command “nvidia-smi”. I like to run the driver with the no ‘UI’ flag -
sudo sh NVIDIA-Linux-x86_64-455.38.run --ui=noneStep 3. Have you figured out where all of the lotus paths will be?
You already have the disks sorted out - now decide what your paths will look like (or use the defaults) - here are mine:
/ is my Linux OS path
/home is part of the first disk and root partition - it is where the GitHub source sits. The default path is ~lotus
In my home directory, I have a subdirectory called “parms” which holds the proof parameters and the parent cache. If you do not specify the path the parameters will be in /var/tmp/filecoin-proof-parameters.
Next I chose a different disk and partition for my .lotus build so that I can manage my chain and not put my OS at risk. /.lotus is the path I chose. If you do not specify a path it will put .lotus in your home directory.
Next you define the sealing worker path. I use a separate disk designated /Lotus The actual path is /Lotus/worker
The GPU needs to be able to be locked to a single task. I use the path /Lotus/tmpdir for the tiny files it needs.
And finally I need to specify where the miner (storage) repo goes for long term storage. That path is /pool/storage. If you don’t specify the path the default is .lotusminer in your home directory.
Finally designate a backup path for lotus backups. Each of my backups is about 27MB
Step 4. Set up a script to set your env vars.
Mine is located in /etc/profile.d/lotus.sh It loads whenever I login and start the miner. I start the miner manually and not systemd. Remember to chmod the file as executable. Always source it after changing it or simply logout and login again. You can see the paths… the other lines you can search in the online docs - they work for me but you might need to adjust if you have significantly different hardware.
Step 5. Where will those config.toml files get created?
For me the lotus daemon config is /.lotus/config.toml [default would be ~.lotus/config.toml]
The miner config is in /pool/storage/config.toml [default would be ~.lotusminer/config.toml]
Step 6. What if you only want to run a lotus node and no miner?
You will need a chain so that part stays as well as the .lotus directory. You won’t need the miner path or sealing/worker disks or path. You need a lot less ram (32GB minimum) and smaller CPU (8 cores) and less Internet bandwidth.
Step 7. A bigger miner - adding a remote worker
The day may come where you want to grow your power faster, like seal 10TiB in 2 weeks rather than 2 months. What I did was continue to use my miner as described above, but move all of the sealing processes to a dedicated remote sealing worker. Here is my layout:
Hardware ThreadRipper 3990x with 256GB ram (max) and 2080Ti GPU
Note that I have two paths for sealing repo so I have to put them both in the command line when I start the worker:
nohup lotus-worker --worker-repo /seal/worker --worker-repo /seal2/worker run --listen 0.0.0.0:3456 > worker.log 2>&1 &Beta Was this translation helpful? Give feedback.
All reactions