Synchronization

Didget Management is much more than just managing lots of Didgets within a given Chamber. It is about managing all the Didgets within a given user's Domain. Each Chamber within the global Didget Realm is a member of one and only one Domain. Since each Chamber within a user's Domain is probably located on a completely separate storage device there is a need to be able to manage the data across those devices.

Unlike file systems, the Didget Manager can perform operations against a set of Didgets without explicit commands from a running application. Policy Didgets created by the user can direct the Didget Manager to perform those operations automatically when certain events occur or when a specified amount of time has passed. Tasks like backup, replication, and synchronization can all be controlled using Policy Didgets.

One of the biggest challenges for existing applications that must try to synchronize data between two separate file system volumes today is in determining exactly which files are the same and which are different. If each volume has a large number of files, this task can also take a very long time. Even if two files have the same name, extra metadata and even the full contents of the data stream must be checked to make sure there are no differences between them. The challenge is even harder if most of the files are the same, but located in different folders on each system.

For example, suppose an application wanted to make sure two separate volumes both had the exact same copies of all photographs stored within them. It would need to first find every photograph in each volume and then compare it with each photograph in the other volume. If Volume A had some photographs that Volume B did not (or vice versa), then it would need to copy them. What should it do if all the pictures on Volume A were located under a /photos file folder hierarchy and all the pictures on Volume B were located under a /pictures folder? Should it synchronize by trying to replicate the folder structures or instead try to copy files to existing folders?

Synchronization between any two Chambers in the Didget Realm is almost trivial. The Didget Managers can quickly compare the two Chambers and find all the differences between them. The event counters and Marker Didgets discussed in an earlier post are tools the Didget Manager uses to figure out what has changed and what order things have happened. Didgets can be copied between two Chambers without needing to worry about folder structures.

For example, two Chambers that each have a million Didgets in them can be compared in just a few seconds and a complete list of all new or modified Didgets since the last synchronization event can be generated. Following the synchronization policy (or policies), the Didget Manager can copy any changes between the two Chambers so that they are completely in sync with each other.