The amount of data managed by organisations continues to grow relentlessly.
Driven by the high costs of maintaining multiple local storage systems, there
is a well established trend towards storage consolidation using multi-tier Virtualised Storage Systems (VSSs). At the same time, storage infrastructures
are increasingly subject to stringent Quality of Service (QoS) demands.
Within a VSS, it is challenging to match desired QoS with delivered QoS,
considering the latter can vary dramatically both across and within tiers.
Manual efforts to achieve this match require extensive and ongoing human
intervention. Automated efforts are based on workload analysis, which ignores
the business importance of infrequently accessed data.
This thesis presents our design, implementation and evaluation of data
maintenance strategies in an enhanced version of the popular Linux Extended
3 Filesystem which features support for the elegant specification
of QoS metadata while maintaining compatibility with stock kernels. Users
and applications specify QoS requirements using a chmod-like interface. System
administrators are provided with a character device kernel interface
that allows for profiling of the QoS delivered by the underlying storage. We
propose a novel score-based metric, together with associated visualisation
resources, to evaluate the degree of QoS matching achieved by any given
data layout. We also design and implement new inode and datablock allocation
and migration strategies which exploit this metric in seeking to match
the QoS attributes set by users and/or applications on files and directories
with the QoS actually delivered by each of the filesystem’s block groups.
To create realistic test filesystems we have included QoS metadata support
in the Impressions benchmarking framework. The effectiveness of the
resulting data layout in terms of QoS matching is evaluated using a special
kernel module that is capable of inspecting detailed filesystem data on-the-fly.
We show that our implementations of the proposed inode and datablock
allocation strategies are capable of dramatically improving data placement
with respect to QoS requirements when compared to the default allocators.