RBD Persistent Read-only Cache
Immutable Object Cache Daemon
Introduction and Generic Settings
The ceph-immutable-object-cache
daemon is responsible for caching parent
image content within its local caching directory. For better performance it's
recommended to use SSDs as the underlying storage.
The key components of the daemon are:
Domain socket based IPC: The daemon will listen on a local domain socket on start up and wait for connections from librbd clients.
LRU based promotion/demotion policy: The daemon will maintain in-memory statistics of cache-hits on each cache file. It will demote the cold cache if capacity reaches to the configured threshold.
File-based caching store: The daemon will maintain a simple file based cache store. On promotion the RADOS objects will be fetched from RADOS cluster and stored in the local caching directory.
On opening each cloned rbd image, librbd
will try to connect to the
cache daemon through its Unix domain socket. Once successfully connected,
librbd
will coordinate with the daemon on the subsequent reads.
If there's a read that's not cached, the daemon will promote the RADOS object
to local caching directory, so the next read on that object will be serviced
from cache. The daemon also maintains simple LRU statistics so that under
capacity pressure it will evict cold cache files as needed.
Here are some important cache configuration settings:
immutable_object_cache_sock
- Description
The path to the domain socket used for communication between librbd clients and the ceph-immutable-object-cache daemon.
- Type
String
- Required
No
- Default
/var/run/ceph/immutable_object_cache_sock
immutable_object_cache_path
- Description
The immutable object cache data directory.
- Type
String
- Required
No
- Default
/tmp/ceph_immutable_object_cache
immutable_object_cache_max_size
- Description
The max size for immutable cache.
- Type
Size
- Required
No
- Default
1G
immutable_object_cache_watermark
- Description
The high-water mark for the cache. The value is between (0, 1). If the cache size reaches this threshold the daemon will start to delete cold cache based on LRU statistics.
- Type
Float
- Required
No
- Default
0.9
The ceph-immutable-object-cache
daemon is available within the optional
ceph-immutable-object-cache
distribution package.
重要
ceph-immutable-object-cache
daemon requires the ability to
connect RADOS clusters.
Running the Immutable Object Cache Daemon
ceph-immutable-object-cache
daemon should use a unique Ceph user ID.
To create a Ceph user, with ceph
specify the auth get-or-create
command, user name, monitor caps, and OSD caps:
ceph auth get-or-create client.ceph-immutable-object-cache.{unique id} mon 'allow r' osd 'profile rbd-read-only'
The ceph-immutable-object-cache
daemon can be managed by systemd
by specifying the user
ID as the daemon instance:
systemctl enable ceph-immutable-object-cache@ceph-immutable-object-cache.{unique id}
The ceph-immutable-object-cache
can also be run in foreground by ceph-immutable-object-cache
command:
ceph-immutable-object-cache -f --log-file={log_path}
QOS Settings
The immutable object cache supports throttling, controlled by the following settings:
immutable_object_cache_qos_schedule_tick_min
- Description
Minimum schedule tick for immutable object cache.
- Type
Milliseconds
- Required
No
- Default
50
immutable_object_cache_qos_iops_limit
- Description
The desired immutable object cache IO operations limit per second.
- Type
Unsigned Integer
- Required
No
- Default
0
immutable_object_cache_qos_iops_burst
- Description
The desired burst limit of immutable object cache IO operations.
- Type
Unsigned Integer
- Required
No
- Default
0
immutable_object_cache_qos_iops_burst_seconds
- Description
The desired burst duration in seconds of immutable object cache IO operations.
- Type
Seconds
- Required
No
- Default
1
immutable_object_cache_qos_bps_limit
- Description
The desired immutable object cache IO bytes limit per second.
- Type
Unsigned Integer
- Required
No
- Default
0
immutable_object_cache_qos_bps_burst
- Description
The desired burst limit of immutable object cache IO bytes.
- Type
Unsigned Integer
- Required
No
- Default
0
immutable_object_cache_qos_bps_burst_seconds
- Description
The desired burst duration in seconds of immutable object cache IO bytes.
- Type
Seconds
- Required
No
- Default
1