Method and system for managing metadata in a storage environment

Abstract

Method and system is provided for managing metadata for a plurality of data containers that are stored at one or more storage volumes in a storage system. The metadata is collected from one or more storage volumes and then provided to a catalog module. The catalog module pre-processes the metadata and then generates a searchable data structure. The searchable data structure may then be used to respond to a user request for information regarding the storage system.

1 - 36 . (canceled) 
     
     
         37 . A machine implemented method, comprising:
 configuring a storage volume as a catalog volume for storing metadata associated with a plurality of data containers stored at a plurality of storage volumes managed by a plurality of storage system nodes of a storage system, each node executing a storage operating system for reading and writing the plurality of data containers at storage space associated with the plurality of storage volumes;   collecting metadata for the plurality of data containers; wherein after a first snapshot is generated for the plurality of data storage volume, metadata is collected only for data containers that were changed after the first snapshot was generated; and   storing pre-processed metadata in a searchable data structure for the plurality of data containers; wherein the searchable data structure stores metadata for a plurality of directory entries and for non-directory based data containers stored at the plurality of storage volumes;   and wherein each entry for the non-directory based data containers stores a reference to a parent directory entry that stores a storage path for a corresponding non-directory based data container such that the storage path for each data container can be obtained using the reference to the parent directory entry without having to store individual storage paths for each non-directory based data containers.   
     
     
         38 . The method of  claim 37 , wherein the searchable data structure includes a first searchable segment that stores attributes for the plurality of directory entries including a unique directory identifier, a directory name, a size of a directory, a permission associated with a directory, a user identifier identifying a user that uses a directory, a group identifier identifying a group associated with a directory, an access time when a directory was accessed, a creation time when a directory was created, a modification time when a directory was modified and an indicator indicating if a directory was created, modified or deleted. 
     
     
         39 . The method of  claim 38 , wherein the searchable data structure includes a second searchable segment that stores attributes for the plurality of data containers, including a unique identifier for identifying a data container, an identifier that associates a data container to a directory entry, a data container name, a size of a data container, a permission associated with a data container, a user identifier identifying a user for a data container, a group identifier identifying a group associated with a data container, an access time when a data container was accessed, a creation time when a data container was created, a modification time when a data container was modified and an indicator indicating if a data container was created, modified or deleted. 
     
     
         40 . The method of  claim 37 , wherein the searchable data structure includes a snapshot identifier for identifying a snapshot when the plurality of data containers were replicated, a time stamp when the snapshot was taken and a snapshot version indicator indicating the snapshot when a change in status for the data container was discovered. 
     
     
         41 . The method of  claim 39 , wherein in response to a user request for data container information, the first searchable segment and the second searchable segment are used to obtain the requested information. 
     
     
         42 . The method of  claim 37 , wherein the searchable data structure stores metadata associated with an active file system. 
     
     
         43 . The method of  claim 37 , wherein the metadata is collected based on an event, a schedule or a user request. 
     
     
         44 . A non-transitory, machine readable storage medium storing executable instructions, which when executed by a machine, causes the machine to perform a method, the method comprising:
 configuring a storage volume as a catalog volume for storing metadata associated with a plurality of data containers stored at a plurality of storage volumes managed by a plurality of storage system nodes of a storage system, each node executing a storage operating system for reading and writing the plurality of data containers at storage space associated with the plurality of storage volumes;   collecting metadata for the plurality of data containers; wherein after a first snapshot is generated for the plurality of data storage volume, metadata is collected only for data containers that were changed after the first snapshot was generated; and   storing pre-processed metadata in a searchable data structure for the plurality of data containers; wherein the searchable data structure stores metadata for a plurality of directory entries and for non-directory based data containers stored at the plurality of storage volumes;   and wherein each entry for the non-directory based data containers stores a reference to a parent directory entry that stores a storage path for a corresponding non-directory based data container such that the storage path for each data container can be obtained using the reference to the parent directory entry without having to store individual storage paths for each non-directory based data containers.   
     
     
         45 . The storage medium of  claim 44 , wherein the searchable data structure includes a first searchable segment that stores attributes for the plurality of directory entries including a unique directory identifier, a directory name, a size of a directory, a permission associated with a directory, a user identifier identifying a user that uses a directory, a group identifier identifying a group associated with a directory, an access time when a directory was accessed, a creation time when a directory was created, a modification time when a directory was modified and an indicator indicating if a directory was created, modified or deleted. 
     
     
         46 . The storage medium of  claim 45 , wherein the searchable data structure includes a second searchable segment that stores attributes for the plurality of data containers, including a unique identifier for identifying a data container, an identifier that associates a data container to a directory entry, a data container name, a size of a data container, a permission associated with a data container, a user identifier identifying a user for a data container, a group identifier identifying a group associated with a data container, an access time when a data container was accessed, a creation time when a data container was created, a modification time when a data container was modified and an indicator indicating if a data container was created, modified or deleted. 
     
     
         47 . The storage medium of  claim 44 , wherein the searchable data structure includes a snapshot identifier for identifying a snapshot when the plurality of data containers were replicated and a snapshot version indicator indicating the snapshot when a change in status for the data container was discovered. 
     
     
         48 . The storage medium of  claim 46 , wherein in response to a user request for data container information, the first searchable segment and the second searchable segment are used to obtain the requested information. 
     
     
         49 . The storage medium of  claim 44 , wherein the searchable data structure stores metadata associated with an active file system. 
     
     
         50 . The storage medium of  claim 44 , wherein the metadata is collected based on an event, a schedule or a user request. 
     
     
         51 . A system, comprising:
 a plurality of storage volumes managed by a plurality of storage system nodes of a storage system, each node having a processor for executing a storage operating system for reading and writing a plurality of data containers at storage space associated with the plurality of storage volumes; wherein a storage volume is configured as a catalog volume for storing metadata associated with the plurality of data containers; and   a processor executing instructions out of memory for:   collecting metadata for the plurality of data containers; wherein after a first snapshot is generated for the plurality of data storage volume, metadata is collected only for data containers that were changed after the first snapshot was generated; and   storing pre-processed metadata in a searchable data structure for the plurality of data containers;   wherein the searchable data structure stores metadata for a plurality of directory entries and for non-directory based data containers stored at the plurality of storage volumes;   and wherein each entry for the non-directory based data containers stores a reference to a parent directory entry that stores a storage path for a corresponding non-directory based data container such that the storage path for each data container can be obtained using the reference to the parent directory entry without having to store individual storage paths for each non-directory based data containers.   
     
     
         52 . The system of  claim 51 , wherein the searchable data structure includes a first searchable segment that stores attributes for the plurality of directory entries including a unique directory identifier, a directory name, a size of a directory, a permission associated with a directory, a user identifier identifying a user that uses a directory, a group identifier identifying a group associated with a directory, an access time when a directory was accessed, a creation time when a directory was created, a modification time when a directory was modified and an indicator indicating if a directory was created, modified or deleted. 
     
     
         53 . The system of  claim 52 , wherein the searchable data structure includes a second searchable segment that stores attributes for the plurality of data containers, including a unique identifier for identifying a data container, an identifier that associates a data container to a directory entry, a data container name, a size of a data container, a permission associated with a data container, a user identifier identifying a user for a data container, a group identifier identifying a group associated with a data container, an access time when a data container was accessed, a creation time when a data container was created, a modification time when a data container was modified and an indicator indicating if a data container was created, modified or deleted. 
     
     
         54 . The system of  claim 51 , wherein the searchable data structure includes a snapshot identifier for identifying a snapshot when the plurality of data containers were replicated and a snapshot version indicator indicating the snapshot when a change in status for the data container was discovered. 
     
     
         55 . The system of  claim 53 , wherein in response to a user request for data container information, the first searchable segment and the second searchable segment are used to obtain the requested information. 
     
     
         56 . The system of  claim 51 , wherein the searchable data structure stores metadata associated with an active file system. 
     
     
         57 . The system of  claim 51 , wherein the metadata is collected based on an event, a schedule or a user request.