Dotcache

Why Dotcache?

Several applications store volatile, yet persistent information in SQLite databases (or plain files) in a user's home directory. With home directories on a file server, updates to these databases, involving a large number of small synchronous write operations, put a severe pressure on the server. Also, there are locking issues with SQLite and NLM. With home directories on a local hard disc, constant updates prevent disc spindown. Dotcache addresses this problem by redirecting the original objects to a local non-persistent storage, while keeping a regularily synchronized backup on persistent storage. It can deal with the same user logging in to several machines at once, sharing the same (NFS-mounted) home directory.

How it works

Dotcache (as invoked by the pam_dotcache helper as described below) handles (graphical login) sessions in which it manages items (files or directories) originally located in the user's home directory. For every item, it uses three distinct locations: the original one, a backup one (also inside the home directory) and a cache one on local, possibly non-persistent, storage.

Being able to open several parallel sessions on diffent clients implies that only one of these session's local changes to the cached item can be copied back to the original (lifting this limitation would require a synchronization between different clients). Such a session is called a primary one while all others are called secondary ones. A secondary sessions' changes to objects managed by Dotcache are not visible to other sessions and are lost on closing the secondary session.

More details

Opening an initial session (which becomes a primary one) moves the original to the backup location (under an encoded name where the full pathname may serve as a filename), copies it to the cache and installs a symlink pointing there in the original location. It then spawns a background syncer that will periodically sync the cache to the backup. Opening a session while another one is already active will re-use the existing backup and the links pointing from the original location to the cache and simply copy the backuped items to the local cache. If one of the other active sessions is a primary one, the newly created session will be secondary and not synchronize the cache to the backup, neither periodically nor when closing. Instead of a background syncer, it will spawn a background pinger to periodically mark the session active. Closing a primary session stops the background synching and explicitly synchronizes each item's backup location with the cache. Closing a secondary session will only stop the background pinger (and may also clean the cache). Closing the final session (no matter whether primary or secondary) removes the symlink which has been installed replacing the original item and then moves the item back from the backup location to the original one. After that, the original looks as if it had never been managed by Dotcache.

Since there is no communication between different clients, some measures must be taken in order to recognize a defunct, yet still registered, session. This may happen if, i. e. a client crashes with a Dotcache session open. Such sessions are called (and, if identified, declared as) stale. So, in more detail, opening a session first looks for stale local sessions, i. e. sessions registered as running on the local host, but either not registered locally or where the (locally) registered syncer or pinger process is not actually running. This will catch most cases of a client crash without needing to wait for a timeout to elapse. It will then try to identify non-local stale sessions by examining a timestamp periodically updated by the syncher or pinger process and remove them from the list of active sessions. It then declares itself primary if no other primary session is active and secondary otherwise. It will also notice if any other sessions are active or were registered but declared stale by the process above. After registering itself globally and locally (only one session per user and display/tty on a client is allowed since it would be hard to tell two with identical properties apart), for each configured item, depending on the exact combination of this session being the only one/the only non-stale one/not the only one, the type of object found in the original and backup locations, either one of the steps described above will be taken, recovery be tried or Dotcache will fail. The initial session will then register the list of managed objects. After that, the syncer/pinger process is started. Closing the final session will make a similar decision based on the type of object present in the backup and original locations and either follow the steps outlined above, recover from errors if possible or fail.

A helper, pam_dotcache, faciliates calling Dotcache (via pam_exec) on session opening/closing. It can also be used to restrict dotcache to individual users. Currently, it will only invoke dotcache proper for a graphical login.

Implementation and dependencies

Both dotcache and the pam_dotcache helper are written in POSIX Shell. In addition to standard POSIX utilites, a find utility supporting the -mmin option and rsync are required.

Availability

Dotcache is available in both tar.gz and dpkg form under a 2-clause-BSD-style License.