⚝
One Hat Cyber Team
⚝
Your IP:
216.73.216.23
Server IP:
178.33.27.10
Server:
Linux cpanel.dev-unit.com 3.10.0-1160.108.1.el7.x86_64 #1 SMP Thu Jan 25 16:17:31 UTC 2024 x86_64
Server Software:
Apache/2.4.57 (Unix) OpenSSL/1.0.2k-fips
PHP Version:
8.2.11
Buat File
|
Buat Folder
Eksekusi
Dir :
~
/
usr
/
local
/
src
/
netdata
/
health
/
guides
/
netdev
/
View File Name :
1min_netdev_budget_ran_outs.md
### Understand the alert Your system communicates with the devices attached to it through interrupt requests. In a nutshell, when an interrupt occurs, the operating system stops what it was doing and starts addressing that interrupt. Network interfaces can receive thousands of packets per second. To avoid burying the system with thousands of interrupts, the Linux kernel uses the NAPI polling framework. In this way, we can replace hundreds of hardware interrupts with one poll by managing them with a few Soft Interrupt ReQuests (Soft IRQs). Ksoftirqd is a per-CPU kernel thread responsible for handling those unserved Soft Interrupt ReQuests (Soft IRQs). The Netdata agent inspects the average number of times Ksoftirqd ran out of netdev_budget or CPU time when there was still work to be done. This abnormality may cause packet overflow on the intermediate buffers and, as a result, drop packet on your network interfaces. The default value of the netdev_budget is 300. However, this may not be enough in some cases, such as: - Multiple interfaces operating at 1Gbps, or even a single interface at 10Gbps. - Lower powered systems processing very large amounts of network traffic. ### NAPI polling mechanism. The design of NAPI allows the network driver to go into a polling mode, buffering the packets it receives into a ring-buffer, and raises a soft interrupt to start a NAPI polling cycle instead of being hard-interrupted for every packet. Linux kernel through NAPI will poll data from the buffer until the netdev_budget_usecs times out or the number of packets reaches the netdev_budget limit. - netdev_budget_usecs variable defines the maximum number of microseconds in one NAPI polling cycle. - netdev_budget variable defines the maximum number of packets taken from all interfaces in one polling cycle. ### Troubleshoot the alert - Increase the netdev_budget value. 1. Check your current value. ``` root@netdata~ $ sysctl net.core.netdev_budget net.core.netdev_budget = 300 ``` 2. Try to increase it gradually with increments of 100. ``` root@netdata~ $ sysctl -w net.core.netdev_budget=400 ``` 3. Verify the change and test it with the same workload that triggered the alarm originally. If the problem still exists, try to increment it again. ``` root@netdata~ $ sysctl net.core.netdev_budget net.core.netdev_budget = 400 ``` 4. If this change works for your system, you could make it permanently. Bump this `net.core.netdev_budget=<desired_value>` entry under `/etc/sysctl.conf` 5. Reload the sysctl settings. ``` root@netdata~ $ sysctl -p ```