Content indexing of files in virtual disk block-level backup copies

Abstract

A streamlined approach analyzes block-level backups of VM virtual disks and creates both coarse and fine indexes of backed up VM data files in the block-level backups. The indexes (collectively the “content index”) enable granular searching by filename, by file attributes (metadata), and/or by file contents, and further enable granular live browsing of backed up VM files. Thus, by using the illustrative data storage management system, ordinary block-level backups of virtual disks are “opened to view” through indexing. Any block-level copies can be indexed according to the illustrative embodiments, including file system block-level copies. The indexing occurs offline in an illustrative data storage management system, after VM virtual disks are backed up into block-level backup copies, and therefore the indexing does not cut into the source VM's performance. The disclosed approach is widely applicable to VMs executing in cloud computing environments and/or in non-cloud data centers. The illustrative content indexing is accomplished without restoring the VM data files being indexed to a staging location.

What is claimed is: 
     
         1 . A method for granular indexing of backup copies of virtual machine data files using one or more pseudo-disk drivers, the method comprising:
 by a storage manager, initiating indexing of a first representation of a virtual machine disk at a first point in time, wherein the first representation is based on a first plurality of block-level backup copies of first virtual machine data files in the virtual machine disk at or before the first point in time, and wherein the backup copies were generated by a media agent;   by the storage manager, cause a first pseudo-disk driver to be activated at a first computing device, wherein the first pseudo-disk driver corresponds to the first representation and is in communication with the media agent, wherein the first computing device comprises one or more hardware processors, and wherein the storage manager comprises one or more hardware processors;   by the first pseudo-disk driver, cause a first pseudo-disk to be mounted at the first computing device as a data storage volume for the first representation, wherein the first pseudo-disk is implemented in cache memory at the first computing device;   by a file manager application executing at the first computing device, issuing read requests for metadata in the first representation to the pseudo-disk driver, wherein the pseudo-disk driver transmits a read request to the media agent if the read request cannot be served from the pseudo-disk;   by the media agent, responding to each received read request by recalling metadata from the first plurality of block-level backup copies without restoring each of the first virtual machine data files in its entirety to the first computing device; and   by the first computing device, generating a first index that comprises metadata about the first virtual machine data files in the virtual machine disk as of the first point in time according to the first representation of the virtual machine disk, and wherein the metadata comprises filenames and file attributes.   
     
     
         2 . The method of  claim 1 , wherein the first index enables searching for individual files among the first virtual machine data files based on the first plurality of block-level backup copies. 
     
     
         3 . The method of  claim 1 , wherein the first representation is a logical view of the first virtual machine data files in the virtual machine disk at the first point in time. 
     
     
         4 . The method of  claim 1 , wherein the first index is based on metadata identified by the media agent as current as of the first point in time and stored at the pseudo-disk. 
     
     
         5 . The method of  claim 1 , wherein based on metadata identified by the media agent as current as of the first point in time, the first index tracks the first virtual machine data files in the virtual machine disk as of the first point in time. 
     
     
         6 . The method of  claim 1 , wherein the media agent processes the metadata recalled from the first plurality of block-level backup copies to identify metadata about the first virtual machine data files that is current as of the first point in time;
 wherein the media agent's response to each received read request comprises metadata identified as current as of the first point in time;   wherein the pseudo-disk driver stores responses from the media agent to the pseudo-disk; and   wherein the first index is based on the metadata identified as current as of the first point in time and stored at the pseudo-disk.   
     
     
         7 . The method of  claim 1  further comprising:
 by the storage manager, cause a second pseudo-disk driver to be activated at the first computing device, wherein the second pseudo-disk driver corresponds to a second representation of the virtual machine disk at a second point in time, wherein the second representation is based on a second plurality of block-level backup copies of second virtual machine data files in the virtual machine disk at or before the second point in time, and wherein the second plurality of block-level backup copies were generated by the media agent; 
 by the second pseudo-disk driver, cause a second pseudo-disk to be mounted at the first computing device as a data storage volume for the second representation, wherein the second pseudo-disk is implemented in cache memory at the first computing device, and wherein the second pseudo-disk driver is in communication with the media agent; 
 by the file manager application, issuing second read requests for metadata in the second representation to the second pseudo-disk driver, wherein the pseudo-disk driver transmits a second read request to the media agent if the second read request cannot be served from the pseudo-disk; 
 by the media agent, recalling metadata from the second plurality of block-level backup copies without restoring each of the second virtual machine data files in its entirety to the first computing device; and 
 by the first computing device, generating a second index that comprises metadata about the second virtual machine data files as of the second point in time according to the second representation of the virtual machine disk. 
 
     
     
         8 . The method of  claim 7 , wherein the first index and the second index track file metadata for the virtual machine disk at the first point in time and at the second point in time, respectively; and
 wherein at least one data file among the first virtual machine data files is also among the second virtual machine data files.   
     
     
         9 . The method of  claim 1  further comprising:
 by the storage manager, cause a second pseudo-disk driver to be activated at the first computing device, wherein the second pseudo-disk driver corresponds to a second representation of the virtual machine disk at a second point in time, wherein the second representation is based on a second plurality of block-level backup copies of second virtual machine data files in the virtual machine disk at or before the second point in time, and wherein the second plurality of block-level backup copies were generated by the media agent; 
 by the second pseudo-disk driver, cause a second pseudo-disk to be mounted at the first computing device as a data storage volume for the second representation, wherein the second pseudo-disk is implemented in cache memory at the first computing device, and wherein the second pseudo-disk driver is in communication with the media agent; 
 by the file manager application, issuing second read requests for metadata in the second representation to the second pseudo-disk driver, which transmits the second read requests to the media agent; 
 by the media agent, recalling metadata from the second plurality of block-level backup copies without restoring each of the second virtual machine data files in its entirety to the first computing device; and 
 by the first computing device, adding to the first index metadata about the second virtual machine data files as of the second point in time according to the second representation of the virtual machine disk, wherein the first index comprises file metadata for the virtual machine disk at the first point in time and at the second point in time. 
 
     
     
         10 . The method of  claim 1  further comprising:
 by the first computing device, selecting one of the first virtual machine data files tracked by the first index; 
 by the media agent, restoring to the pseudo-disk from the first plurality of block-level backup copies, individual portions of the one first virtual machine data file that the media agent determines to be current as of the first point in time, wherein the individual portions are restored one by one without restoring the one first virtual machine data file in its entirety to the first computing device; 
 by the first computing device, applying content criteria to each individual portion in the pseudo-disk to identify content that matches the content criteria in the one first virtual machine data file at the first point in time; and 
 by the first computing device, generating a second index that tracks the identified content that matches the content criteria, wherein the second index enables searching for content among the first virtual machine data files in the virtual machine disk as of the first point in time, based on the first plurality of block-level backup copies. 
 
     
     
         11 . The method of  claim 1  further comprising:
 by the first computing device, causing the first pseudo-disk to be discarded after the first index is generated with respect to the first point in time. 
 
     
     
         12 . The method of  claim 1 , wherein the pseudo-disk driver stores data included in write commands issued by the file manager application without making changes to the first plurality of block-level backup copies; and
 by the first computing device, after the first index is generated with respect to the first point in time, causing the first pseudo-disk to be discarded, including the data included in the write commands issued by the file manager application.   
     
     
         13 . A method for granular indexing backup copies of virtual machine data files using one or more pseudo-disk drivers, the method comprising:
 activating a first pseudo-disk driver at a first computing device, wherein the first pseudo-disk driver corresponds to a first logical view of first virtual machine data files in a virtual machine disk at a first point in time, wherein the first logical view is based on a first plurality of block-level backup copies of the first virtual machine data files at or before the first point in time, and wherein the backup copies were generated by a media agent, and wherein the first computing device comprises one or more hardware processors;   by the first pseudo-disk driver, cause a first pseudo-disk to be mounted at the first computing device as a first data storage volume implemented in cache memory at the first computing device;   by the pseudo-disk driver, receiving from a file manager application executing at the first computing device read requests for metadata in the first data storage volume, wherein the pseudo-disk driver transmits the read requests to the media agent if the read requests cannot be served from the pseudo-disk;   by the media agent, responding to received read requests by recalling metadata from the first plurality of block-level backup copies without restoring each of the first virtual machine data files in its entirety to the first computing device; and   by the first computing device, generating a first index that comprises metadata about the first virtual machine data files as of the first point in time, wherein the first index enables searching for individual files among the first virtual machine data files based on the first plurality of block-level backup copies.   
     
     
         14 . The method of  claim 13  further comprising,
 by the first computing device, selecting one of the first virtual machine data files tracked by the first index; 
 by the media agent, restoring to the pseudo-disk from the first plurality of block-level backup copies, individual portions of the one first virtual machine data file that the media agent determines to be current as of the first point in time, wherein the individual portions are restored one by one without restoring the one first virtual machine data file in its entirety to the first computing device; 
 by the first computing device, applying content criteria to each individual portion in the pseudo-disk to identify content that matches the content criteria in the one first virtual machine data file at the first point in time; and 
 by the first computing device, generating a second index that tracks the identified content, wherein the second index enables searching for content among the first virtual machine data files in the virtual machine disk based on the first plurality of block-level backup copies. 
 
     
     
         15 . The method of  claim 13  further comprising:
 by a storage manager, initiating indexing of the virtual machine disk at the first point in time, by causing the activating of the first pseudo-disk driver at the first computing device, wherein the storage manager comprises one or more hardware processors. 
 
     
     
         16 . The method of  claim 13 , wherein upon receiving a read request the pseudo-disk driver serves a response from the pseudo-disk if responsive data is available therein, and otherwise transmits the read request to the media agent for retrieval from a backup copy if the responsive data is not available at the media agent. 
     
     
         17 . A data storage management system comprising:
 a first computing device comprising one or more hardware processors and computer memory for executing program instructions;   a second computing device comprising one or more hardware processors and computer memory for executing program instructions;   wherein the first computing device is configured to:
 activate a first pseudo-disk driver that executes at the first computing device, wherein the first pseudo-disk driver corresponds to a first logical view of first virtual machine data files in a virtual machine disk at a first point in time, wherein the first logical view is based on a first plurality of block-level backup copies of the first virtual machine data files at or before the first point in time, and wherein the backup copies were generated by a media agent that executes at the second computing device; 
 by the first pseudo-disk driver, cause a first pseudo-disk to be mounted at the first computing device as a first data storage volume implemented in cache memory at the first computing device; 
 by the pseudo-disk driver, receive from a file manager application executing at the first computing device read requests for metadata in the first data storage volume, wherein the pseudo-disk driver transmits the read requests to the media agent if the read requests cannot be served from the pseudo-disk; 
   wherein the second computing device is configured to: by the media agent, respond to received read requests by recalling metadata from the first plurality of block-level backup copies without restoring each of the first virtual machine data files in its entirety to the first computing device; and   wherein the first computing device is further configured to: generate a first index that comprises file metadata for the first virtual machine data files in the virtual machine disk at the first point in time, wherein the first index enables searching for individual files among the first virtual machine data files based on the first plurality of block-level backup copies, and wherein the file metadata comprises file names and file attributes.   
     
     
         18 . The system of  claim 17 , wherein the first computing device is further configured to: select one of the first virtual machine data files tracked by the first index;
 wherein the second computing device is further configured to: by the media agent, restore to the pseudo-disk from the first plurality of block-level backup copies, individual portions of the one first virtual machine data file that the media agent determines to be current as of the first point in time, wherein the individual portions are restored one by one without restoring the one first virtual machine data file in its entirety to the first computing device; and   wherein the first computing device is further configured to:   apply content criteria to each individual portion in the pseudo-disk to identify content that matches the content criteria in the one first virtual machine data file at the first point in time, and   generate a second index that tracks the identified content, wherein the second index enables searching for content among the first virtual machine data files in the virtual machine disk based on the first plurality of block-level backup copies.   
     
     
         19 . The system of  claim 17 , wherein the second computing device is further configured to:
 by the media agent, process the metadata recalled from the first plurality of block-level backup copies to identify metadata about the first virtual machine data files that is current as of the first point in time,   wherein the media agent's response to each received read request comprises metadata identified as current as of the first point in time,   wherein the pseudo-disk driver stores responses from the media agent to the pseudo-disk, and   wherein the first index is based on the metadata identified as current as of the first point in time and stored at the pseudo-disk.   
     
     
         20 . The system of  claim 17 , wherein upon receiving a read request the pseudo-disk driver is configured to serve a response from the pseudo-disk if responsive data is available therein, and otherwise to transmit the read request to the media agent for retrieval from a backup copy if the responsive data is not available at the media agent.