Linux
This page covers importance of understanding the Linux operating system including installation, configuration, environment variables, performance optimization, and performance analysis.
Last updated
Was this helpful?
This page covers importance of understanding the Linux operating system including installation, configuration, environment variables, performance optimization, and performance analysis.
Last updated
Was this helpful?
Becoming a storage provider requires a team with a variety of skills. Of all the technical skills needed to run a storage provider business, storage knowledge is important, but arguably, it is even more important to have deep understanding of the Linux operating system.
Where most enterprise storage systems (NAS, SAN and other types) do not require the administrator to have hands-on Linux experience, Filecoin does require a lot more knowledge about Linux. For starters, this is because Filecoin is not just a storage system. It is a blockchain platform that offers decentralized storage. As a storage provider, you must ensure that your production system is always available, not just providing the storage.
Although Lotus also runs on Mac, production systems generally all run on Linux. More specifically, most storage providers run on Ubuntu. Any Linux distribution should be possible but running Ubuntu makes it easier to find support in the community. Every distribution is a bit different and knowing that all components have been built and tested on Ubuntu, and knowing you have the same OS variables in your environment as someone else, lowers the barrier to starting as a storage provider significantly. Go for Ubuntu Server and choose the latest LTS version.
Install Ubuntu LTS as a headless server. This means there is no desktop environment or GUI installed. It requires you to do everything on the command line. Not having a desktop environment on your server(s) has multiple advantages:
It reduces the attack surface of your systems. Fewer packages installed means fewer patches and updates, but more importantly, fewer potential vulnerabilities.
As you will be running several tasks on GPU (see ), it’s best to avoid running a desktop environment, which might compete for resources on the GPU.
Exclude the nvidia-drivers and cuda packages from your updates using set. Once you have a working setup for your specific GPU, you will want to test these packages before you risk breaking them. Many storage providers may need to since some operating systems do not include this package by default.
All installation tasks and operational activities happen from the CLI. When installing and upgrading Lotus, it is recommended to build the binaries from source code. Upgrades to Lotus happen every two months or so. If you are unable to perform a mandatory Lotus upgrade, you may become disconnected from the Filecoin network, which means you could be penalized and lose money, so it’s vital to keep Lotus up-to-date.
Configuration parameters for the Lotus client are stored in 2 places:
into config.toml files in ~/.lotus, ~/.lotusminer and ~/.lotusworker
into environment variables in ~/.bashrc if you are using Bash as your shell
Configuration parameters, and most environment variables, are covered in the . More specific environment variables around performance tuning can be found on the repository on GitHub.
Lotus needs to open a lot of files simultaneously, and it is necessary to reconfigure the OS to support this.
This is one of the examples where not every Linux distribution is the same. On Ubuntu, run the following commands:
The commands used are:
Some storage providers fine-tune their setups by enabling CPU-core-pinning of certain tasks (especially PC1), as a starting storage provider it’s not necessary to do that level of tuning. It is essential, however, to have some level of understanding of the to know how to prioritize and deprioritize other tasks in the OS. In the case of Lotus workers you certainly want to prioritize the lotus-worker process(es).
Diagnosing performance bottlenecks on a system is vital to keeping a well balanced .
There are many good resources to check out when it comes to Linux performance troubleshooting. Brendan Gregg’s is an excellent introduction. Each one of these commands deserves a chapter on its own but can be further researched in their man pages.