Release v1.33.0

Happy New Year to everyone in the Netdata community. After one of our biggest releases ever, we have re-energized over the holidays, and are ready to continue helping more people troubleshoot their infrastructure. Hopefully, you've already heard about the improvements we made to the kickstart script. With this release, we're adding even more features:
- Netdata is now distributed as pre-built packages on many Linux distributions
- Stream compression (tech preview)
- eBPF CO-RE support

❗We're also keeping our codebase healthy by removing features that are end of life. Read the deprecation notice to check if you are affected.

If you love Netdata and haven't yet considered giving us a Github star, we would appreciate for you to do so!

Netdata open-source Agent growth

The open-source Netdata Agent, the best OSS node monitoring and troubleshooting solution, currently has:

• 1,300,000 unique Netdata nodes live!
• An amazing adoption rate, with 3,300 new nodes per day!
• 280,000 Docker pulls per day with 375 million total, according to DockerHub!

Community news

Netdata is supported both by an active community of global contributors and the Netdata staff.

Get involved:
- Build with us on Netdata.
- Join us on Discord, and say hello.
- Contribute your knowledge with open GitHub issues.
- Launching soon, be on the lookout for our new monthly community awards

Release highlights

Netdata is now distributed as pre-built packages on many Linux distributions

We recently released a completely new version of our one-line installer code. This new kickstart script uses DEB or RPM packages provided by Netdata wherever available. These packages are tightly integrated with the package management system of the distribution, providing the best installation experience in a reliable and fast way.

Already over 70% of our new installations use DEB or RPM packages! The updated kickstart script has several advantages over the old one:

• It’s more advanced because it automatically selects the best supported installation method for your system. However, you can still explicitly ask for a specific type of installation method.
• It’s more convenient as it requires no manual installation of packages on a majority of systems.
• It’s more resource efficient on most systems, meaning less impact on your running workloads (and much faster installs on idle systems).

📄 Find the updated install documentation on our official docs site.

If you were using the old kickstart.sh script through a custom script or orchestration tool, you may need to update the options being passed to get it to behave like it used to (this will usually just involve adding --build-only to the options).

Other installation types do not need to make any changes because of this.

Stream compression (tech preview)

The Agent's streaming mechanism now supports stream compression. Streaming thousands of metrics between Netdata Agents increases your data availability and provides a more robust mechanism to monitor your metrics and troubleshoot problems.

Stream compression allows you to:
- Save up to 70% of bandwidth by reducing the size of transmitted metrics between Netdata Agents.
- Therefore, reduce costs over metered data connections by up to 70%.
- Take advantage of low-speed connections.

Stream compression uses the lossless "LZ4 - Extreme fast compression" library. It achieves compression speeds up to 800Mbps, decompression speeds up to 4500Mbps with an average compression ratio between 2.0 and 3.0. Because this is a technical preview and we are still working to make it amazing, stream compression will be disabled by default.

📄 Learn how to enable streaming between nodes.
📄 If you already stream between nodes, learn how to enable streaming compression

Note: Stream compression only works if all participating Netdata Agents are hosted on an OS which supports the library version lz4 v1.9.0+. If a Netdata Agent does not detect the lz4 v1.9.0+ library version, it will disable stream compression.

eBPF CO-RE support

In v1.32 we added some major improvements to our eBPF support. For this release, we’re taking the next step by gradually introducing BPF CO-RE support!

Today, the distribution of eBPF programs is a challenging. Trying to compile an eBPF program with so many different Linux kernels is extremely complex. We want to make eBPF widely available to everyone without worrying about compatibility. And here is where eBPF CO-RE (Compile Once, Run Everywhere), part of libbpf, comes to the rescue!

CO-RE is a modern approach to writing portable BPF applications that can run on multiple kernel versions and configurations without modifications and runtime source code compilation on the target machine. We now have the opportunity to focus on what matters, add more features, and improve performance of our eBPF offering!

Furthermore, in this release we also introduce two new eBPF charts:
* Threads info: Displays the total number of active eBPF threads and the number of all eBPF threads.
* Load info: Measures the number of eBPF threads running on legacy code or CO-RE.

Acknowledgments

We would like to thank our dedicated, talented contributors that make up this amazing community. The time and expertise that you volunteer is essential to our success. We thank you and look forward to continue to grow together to build a remarkable product.

• @NikolayS for various improvements of python.d/postgres collector.
• @Saruspete for fixing handling of port_rcv_data and port_xmit_data counters in proc/infiniband collector.
• @ardabbour for fix errors in exporting walkthrough.
• @avstrakhov for adding LZ4 streaming data compression.
• @boxjan for fixing permissions of plugins for static builds.
• @candrews for adding a note that Netdata is available on Gentoo.
• @cmd-ntrf for fixing claim node examples in kickstart(-64) documentation.
• @jsoref for fixing spelling.
• @laned130 for adding a missing expression operator to the health configuration reference.
• @lokerhp for fixing a typo in the dashboard_info.js.
• @neotf for adding memory usage chart to python.d/spigotmc collector.
• @pbouchez for adding bar1 memory usage chart to python.d/nvidia_smi collector.
• @scatenag for fixing collecting user statistics for LDAP users in python.d/nvidia_smi collector.
• @sourcecodes2 for adding channels support to PushBullet notification method.
• @bompus for fixing collecting replica set stats in go.d/mongodb collector.

Collectors

Improvements

• Prefer python3 if available (python.d) (#12001, @ilyam8)
• Add bar1 memory usage chart (python.d/nvidia_smi) (#11956, @pbouchez)
• Add a note that Netfilter's "new" and "ignore" counters are removed in the latest kernel (#11950, @ilyam8)
• Consider mat. views as tables in table size/count chart (python.d/postgres) (#11816, @NikolayS)
• Use block_size instead of 8*1024 (python.d/postgres) (#11815, @NikolayS)

Bug fixes

• Fix handling of port_rcv_data and port_xmit_data counters (proc/infiniband)(#11994, @Saruspete)
• Fix handling of decoding errors in ExecutableService (python.d) (#11979, @ilyam8)
• Fix lack of sufficient system capabilities (perf.plugin) (#11958, @vlvkobal)
• Fix Netfilter accounting charts priority (nfacct.plugin) (#11952, @ilyam8)
• Fix lack of sufficient system capabilities (nfacct.plugin) (#11951, @ilyam8)
• Fix collecting user statistics for LDAP users (python.d/nvidia_smi) (#11858, @scatenag)
• Fix tps decode, and add memory usage chart (python.d/spigotmc) (#11797, @neotf)
• Fix collecting replica set stats (go.d/mongodb) (#639, @bompus)

eBPF

Improvements

• Add ebpf.plugin informational charts and various optimizations (#11992, @thiagoftsm)
• Update libbpf library to v0.6.1 (#11865, @thiagoftsm)

Bug fixes

• Fix disabling specific ebpf collectors (#12014, @thiagoftsm)
• Fix cachestat on kernel 5.15.x (#11833, @thiagoftsm)

Health

• Add sending notifications to channels support to PushBullet (#11850, @sourcecodes2)

Streaming

• Add LZ4 streaming data compression (#11821, @avstrakhov)

Documentation

• Fix formatting in the streaming doc (#12026, @kickoke)
• Fix a typo in the python.d/mongodb readme (#12024, @cboydstun)
• Add a note that streaming compression is disabled by default (#12019, @odynik)
• Refine the idlejitter.plugin docs (#12012, @kickoke)
• Delete unused collectors quickstart guide (#12000, @kickoke)
• Delete duplicate getting started doc (#11978, @kickoke)
• Refine the python example for clarity (#11989, @kickoke)
• Add alternative install command for macOS (#11997, @Ferroin)
• Refine the bash example for clarity (#11990, @kickoke)
• Fix the statsd.plugin readme formatting (#11943, @kickoke)
• Update SNMPv3 documentation (#11959, @kickoke)
• Improve PagerDuty notification doc (#11147, @joelhans)
• Fix spelling (#10976, @jsoref)
• Fix tables side borders (#11923, @ilyam8)
• Fix claim node examples in kickstart(-64) documentation (#11242, @cmd-ntrf)
• Fix the title of exporting reference doc (#11252, @joelhans)
• Add a note with a link to guide for using on Pi (#11605, @andrewm4894)
• Add "==" to the list of health expression operators (#11905, @laned130)
• Fix errors in exporting walkthrough (#11902, @ardabbour)
• Fix unresolved file references (#11903, @ilyam8)
• Add missing dependencies for New Cloud Architecture (#11373, @underhood)
• Indicate availability on Gentoo (#7675, @candrews)

Packaging / Installation

• Fix cleanup from a failed DEB install (#12006, @Ferroin)
• Update go.d.plugin version to v0.31.2 (#12005, @ilyam8)
• Fix handling of static archive selection for installs. (#12004, @Ferroin)
• Fix install prefix handling for claiming code in new kickstart script (#11999, @Ferroin)
• Fix the updater script checksum validation for static builds (#11986, @ilyam8)
• Fix retrieving service commands without failure (#11947, @maneamarius)
• Fix getting the latest tag in the updater script (#11908, @maneamarius)
• Fix handling of agent restart on update. (#11887, @Ferroin)
• Fix permissions of plugins that may be built (#11877, @boxjan)
• Fix the code that checks for available updates. (#11870, @Ferroin)
• Initial release of new kickstart script (#11764, @Ferroin)
• Add support to updater for updating native DEB/RPM installs with our official packages (#11753, @Ferroin)

Other notable changes

Improvements

• Add install type info to -W buildinfo output. (#12010, @Ferroin)
• Add support for NVME disks with blkext driver (#12007, @ralphm)
• Perform a host metadata update on child reconnection (#11965, @stelfrag)
• Send ML feature information with UpdateNodeInfo (#11913, @vkalintiris)
• Use absolute features when doing training/prediction. (#11876, @vkalintiris)
• Send the cloud protocol used to posthog (#11842, @MrZammler)
• Remove ACLK Legacy (#11841, @underhood)

Bug fixes

• Fix access to freed memory in ACLK (#12015, @underhood)
• Fix a typo in the dashboard_info.js spigot part (#12008, @lokerhp)
• Fix queue removed alerts (#11996, @MrZammler)
• Fix coverity 374746 (#11973, @MrZammler)
• Fix ACLK chart description (#11970, @underhood)
• Fix a broken link in dashboard_info.js (#11948, @Ancairon)
• Fix an error in configure.ac (#11937, @underhood)
• Fix handling of the "-url" parameter in the claiming script (#11919, @ilyam8)
• Fix time_t format (#11897, @vlvkobal)
• Fix compiling with AWS Kinesis support (#11867, @vlvkobal)
• Fix cmake build (#11862, @vlvkobal)
• Fix compilation warnings (#11846, @vlvkobal)

Code organization

• Remove internal dbengine header from spawn/spawn_client.c (#12009, @vkalintiris)
• Better handle creation of UUID for claiming (#11974, @Ferroin)
• Use libnetdata/required_dummies.h in collectors. (#11971, @vkalintiris)
• Do not use dbengine headers when dbengine is disabled. (#11967, @vkalintiris)
• Use libnetdata/required_dummies.h in collectors. (#11971, @vkalintiris)
• Do not use dbengine headers when dbengine is disabled. (#11967, @vkalintiris)
• Perform a host metadata update on child reconnection (#11965, @stelfrag)
• Remove bitfields from rrdhost. (#11964, @vkalintiris)
• Update libmongoc CMake config (#11962, @vlvkobal)
• Find host and pass host->health_enabled to cloud AlarmLogHealth message (#11960, @MrZammler)
• Compute platform-specific list of static_threads at runtime. (#11955, @vkalintiris)
• Blocking publish and in flight buffer regrowth (#11932, @underhood)
• Try to find worker config thread from inactive threads for new architecture (#11928, @MrZammler)
• Handle re-claim while the agent is running in new architecture (#11924, @MrZammler)
• Include libatomic to allow protobuf to resolve __atomic functions (#11917, @MrZammler)
• Provide runtime ml info from a new endpoint (#11886, @vkalintiris)
• Update dependencies for the pubsub exporting connector (#11872, @vlvkobal)
• Remove ACLK-NG 'cmd' switch by message type (#11866, @underhood)
• Optimize rx msg name resolution (#11811, @underhood)
• Add localhost hostname to the edit_command (#11793, @MrZammler)

Deprecation notice

The following items will be removed in our next release:
- backends subsystem. Has been replaced by the exporting engine.
- node.d/fronius collector. Will be moved to the netdata/community repository.
- node.d/sma_webbox collector. Will be moved to the netdata/community repository.
- node.d/stiebeleltron collector. Will be moved to the netdata/community repository.
- node.d/named collector. Has been replaced by go.d/bind. Will be moved to the netdata/community repository.

Deprecated in this release

Following our previous deprecation notice legacy ACLK support is officially removed in this release. See more information in our last release notes (v1.32).

Support options

As we grow, we stay committed to providing the best support ever seen from an open-source solution. Should you encounter an issue with any of the changes made in this release or any feature in the Netdata agent, feel free to contact us by one of the following channels:

• Github: You can use our Github repo to report bugs.
• Community forum: You can visit our community forum for questions and training.
• Discord: You can jump into our Discord for interactive, synchronous help and discussion. More than 800 engineers are already using it! Join us!