Understanding the Cloud Drive Mapper (CDM) Client

File and folder structure enumeration when browsing

CDM uses a real-time browse algorithm to browse cloud storage quickly and efficiently without requiring the cloud-based file and folder structure to be synced locally. This has several significant advantages over the sync model used by the OneDrive sync client and most other cloud storage sync clients:

  • CDM maps drives nearly instantly at startup and makes the entire drive's contents available immediately, even on a device or environment the user has never accessed.

  • CDM has no file limits (like the 300,000 OneDrive sync client limit) or performance issues related to large drives. Since CDM doesn't sync the whole file structure, it doesn't matter how large it is.

  • CDM has no inbound sync queue, so it almost instantly reflects changes happening in the cloud locally. Sync systems have to honor the queue because a change earlier in the queue can affect changes later in the queue. Consequently, because there's no way for a change to jump the queue, it could take a long time to be reflected locally.

  • Unlike the sync client, CDM has a very light footprint on the local device. The sync client stores substantial file structure metadata, which consumes more CPU and memory.

Unlike WebDAV and other direct-access methods, the folder-view (not the contents) becomes temporarily synced when users browse inside a particular folder with CDM. This gives CDM much more resilience to connection failures, API throttling, and outages (including total outage of the cloud storage service itself). This is because the drive and the active folders within the drive are still usable. This hybrid of real-time browse and transient active sync is unlike any other cloud storage product anywhere. It brings the absolute best-of-both-worlds capabilities to accessing cloud storage.

Opening and local copying (aka downloading) files

Above, we discuss how CDM allows users to browse their files and folders quickly and efficiently. The files they see are essentially just local shadows of the actual files hosted in the cloud. When a user selects a file to open from File Explorer or via a desktop application's Open dialog box, CDM goes to the cloud API in real-time to hydrate the file. This involves seamlessly downloading the file in the background and making it available to the local operating system and the desktop application configured to open it.

The file data technically resides in the cache within the user's local AppData folder for the time the file is open. Whenever the user saves the file, the local cache is updated, and subsequently, the file is updated in the cloud, too.

When the user closes the file, CDM receives a close-handle. CDM then has options over what it does next - these choices are configurable within the CDM Cleanup policy. The local cache data is deleted within 2 minutes by default. But it is possible to modify this setting to have the local cache data removed earlier or later than that point to promote transience or persistence, depending on whatever is most desired by the customer.

Autosave and co-authoring

CDM fully supports Office document co-authoring and autosave. The section above describes the process of a file being downloaded/opened. The process slightly differs with the Office files .docx, .xlsx, and .pptx. CDM will still hydrate the file. However, when the Office application opens the file, it detects from a property within the file that it is hosted with Microsoft 365 and switches it into online mode. This is what allows the autosave and co-authoring features to work. As such, in online mode, when the user updates a file, it doesn't get updated in the local cache to then subsequently be updated in the cloud by CDM. Instead, it bypasses the cache (and CDM) and is updated directly in the cloud.

Saving and uploading files

When you save a file, create a new file, or copy an existing local file into a CDM drive, CDM will first copy the updated or new data into its local cache. From its local cache, it then uploads the file in the background to the cloud.

WebDAV does something similar, except that copying to the cache and uploading to the cloud are all captured within the Windows Save As/Copying dialog boxes. As such, WebDAV handles this process in the foreground rather than the background. While WebDAV does not do this particularly well because the progress status is often inaccurate - particularly with larger files - it is, in principle, a helpful indicator of the status of the file.

Sync tools like the OneDrive sync client do not give a particularly good indication of the upload status of a file. This is usually not a problem on a user's own device. However, where there is an element of non-persistence, such as with a VDI, multi-user workstation, or where the user has to abruptly end their session (such as a student leaving a classroom at the ring of a bell), this lack of user awareness can lead to problems - including in a worst case scenario files not being uploaded at all and the permanent loss of data.

There are some advantages to the sync client model because it can more intelligently control the uploads. It can batch and scale the uploads to optimize upload performance. It can resume temporary losses of internet connectivity. WebDAV, on the other hand, for its advantages of upload visibility, has several significant fragilities, inefficiencies and poor performance, especially when uploading large files.

Again, CDM takes a hybrid approach here. It combines the sync client's background upload method with a front-end UI file progress display similar to WebDAV's. So the uploads are as quick and efficient as possible, but the user has complete visibility of the state and progress of the upload. Indeed, CDM's upload status is also more accurate than that of WebDAV.

OneDrive and SharePoint versioning

When using CDM with OneDrive or SharePoint Online, if a user opens an existing file, updates it, and saves it via a CDM drive, CDM will save it as a new version. However, the file's previous version(s) are still available. This means that if the user wants to discard the recent changes, it's still possible to access and restore the previous version. So if a user accidentally made changes to the wrong file and only realized after they'd saved the changes and closed the file, they can still undo the mistake by restoring the previous version.

The same mechanism also protects users against possible file corruption. In a worst-case scenario, where something managed to corrupt a file either while it's open in an application or in the process of being uploaded, the user can undo the corruption by restoring the previous uncorrupted version.

File deletions

When a user deletes an item from a CDM-mapped drive, CDM sends a message to the cloud storage API to delete the item in the cloud. At this point, it disappears from the local drive. What happens to the file depends on the cloud storage provider. In OneDrive and SharePoint Online, the file is not deleted but moved into a recycle bin. It is still retrievable from the recycle bin for 93 days by default, although the duration is configurable. This means that if a user unintentionally deletes a file from a CDM-mapped drive, they can restore it from the recycle bin. At this point, the file will reappear in the user's drive.

Pausing active uploads

In the CDM client UI, accessed from the system tray, you'll see the Pause button that stops active uploads, freeing up your internet bandwidth. This feature is just for emergencies, such as attending an online meeting and having issues with the connection because of a large upload. The Pause button temporarily stops uploads, increasing your bandwidth for the meeting instead.

However, CDM will not necessarily be able to resume from where it left off with the upload and may need to start it again. Generally, the longer the time between the local file being in sync with the cloud file, the more opportunities there are for issues. You may also forget that the uploads are paused and wonder why your files are not appearing in the cloud. So, while we felt it was a critical just in case feature to prevent CDM causing issues with other processes, we recommend only using the pause functionality sparingly.