The file system ZFS It is one of the best for maintaining the integrity of files on a server, something essential to avoid data loss. In addition, the ZFS file system incorporates other really important features for a server or a NAS, such as Copy-on-Write to later take snapshots natively, real-time compression to save a lot of storage space, and even deduplication to avoid saving the same information in the file system, and thus also save a lot of storage. Now they have improved this last feature, do you want to know what new features will be available soon?
The deduplication process allows us to eliminate copies or duplicates of data or information. This technique is used in databases, storage systems and even email servers. Activating this feature allows us to save storage space locally, but it will also save us from having to send duplicate information to the cloud as a backup, thus saving bandwidth and backup copies will be made much faster.
New Fast Dedup feature in ZFS
The iXsystems development team and Klara have developed and donated the functionality of Fast Dedup to the OpenZFS file system, so very soon any operating system will be able to use this technology. Right now the source code has been made available to the community, for review, additional testing, and for integration into future versions of the OpenZFS file system. Of course, this new feature will be available in the upcoming versions of TrueNAS SCALE and TrueNAS CORE.
Fast Dedup is a very important hotfix that improves the original deduplication capability of OpenZFS. The main problem with traditional deduplication with ZFS is that it eliminates the need to maintain hash tables in RAM continuously, with the aim of avoiding the loss of performance in real reading and writing. This makes the consumption of RAM memory on the server or NAS really high. For example, the manufacturer QNAP in its QuTS hero operating system does not allow deduplication to be activated if the NAS server does not have 16GB of RAM or more, since this important functionality makes intensive use of RAM.
With the introduction of Fast Dedup, the original deduplication has been improved in the following areas:
- The metadata size is dynamically sized to fit in RAM or dedicated SSDs to avoid a performance penalty.
- The metadata structure has been completely redesigned to allow efficient updates by adding records, thereby improving performance for large updates such as deletions.
- The deduplication table will prioritize data that can be easily deduplicated, and remove blocks that do not show deduplication potential.
- Improvements in this metadata along with properly configured storage, including SSDs to store the metadata, will improve sustained read and write performance.
It is expected that these improvements that have been incorporated into OpenZFS will be available for the next OpenZFS 2.3 version. However, the alpha versions of TrueNAS will incorporate this functionality in the beta phase, to thoroughly refine the operation of this technology that is so important for saving space in the NAS. This version 2.3 will be available at the end of this year 2024, or beginning of 2025, so it will still take a long time to see it definitively in a stable version. However, we can use TrueNAS SCALE starting in April with this feature, but never in production to avoid data loss issues.
-
-
-
-
-
-
-
-
