System and method of querying objects on demand

Abstract

An illustrative embodiment disclosed herein is an apparatus including a processor having programmed instructions that identify a temporary bucket linked to one or more objects of a main bucket, detect that an object is uploaded to the main bucket, determine whether the object has an object attribute satisfying an object attribute relationship, and responsive to determining that the object has the object attribute that satisfies the object attribute relationship, add, to the temporary bucket, a link to the object.

What is claimed: 
     
         1 . An apparatus comprising a processor and a memory, the memory comprising programmed instructions that, when executed by the processor, cause the apparatus to:
 receive, from a client, a first structured query language (SQL) query;   execute the first SQL query against a metadata server that stores metadata of a bucket, the first SQL query executed to create a temporary bucket that references a first list of objects included in the bucket, the first list of objects having metadata satisfying a metadata relationship specified in the first SQL query;   receive, from the client, a second SQL query;   execute the second SQL query against the temporary bucket to filter the first list of objects and generate a second list of objects having a fewer number of objects than the first list of objects; and   provide, to the client, the second list of objects.   
     
     
         2 . The apparatus of  claim 1 , wherein the first SQL query and the second SQL query are S3 application programming interface (API) calls. 
     
     
         3 . The apparatus of  claim 1 , wherein the bucket resides in an S3-compliant object storage system. 
     
     
         4 . The apparatus of  claim 1 , the memory comprising the programmed instructions that, when executed by the processor, further cause the apparatus to:
 execute the first SQL query to create the temporary bucket that includes a symbolic link to the first list of objects included in the bucket.   
     
     
         5 . The apparatus of  claim 1 , the memory comprising the programmed instructions that, when executed by the processor, further cause the apparatus to:
 determine that an object that is uploaded to the bucket and that the object has metadata satisfying the metadata relationship; and   responsive to determining that the uploaded object has metadata satisfying the metadata relationship, add, to the temporary bucket, a second reference to the object.   
     
     
         6 . The apparatus of  claim 1 , wherein each object of the first list of objects includes immutable data. 
     
     
         7 . The apparatus of  claim 1 , the memory comprising the programmed instructions that, when executed by the processor, further cause the apparatus to:
 execute the first SQL query to generate the first list of objects having an object prefix matching a prefix specified in the first SQL query.   
     
     
         8 . A non-transitory computer readable storage medium comprising instructions stored thereon that, when executed by a processor, cause the processor to:
 receive, from a client, a first structured query language (SQL) query;   execute the first SQL query against a metadata server that stores metadata of a bucket, the first SQL query executed to create a temporary bucket that references a first list of objects included in the bucket, the first list of objects having metadata satisfying a metadata relationship specified in the first SQL query;   receive, from the client, a second SQL query;   execute the second SQL query against the temporary bucket to filter the first list of objects and generate a second list of objects having a fewer number of objects than the first list of objects; and   provide, to the client, the second list of objects.   
     
     
         9 . The medium of  claim 8 , wherein the first SQL query and the second SQL query are S3 application programming interface (API) calls. 
     
     
         10 . The medium of  claim 8 , wherein the bucket resides in an S3-compliant object storage system. 
     
     
         11 . The medium of  claim 8 , comprising the instructions stored thereon that, when executed by the processor, further cause the processor to:
 execute the first SQL query to create the temporary bucket that includes a symbolic link to the first list of objects included in the bucket.   
     
     
         12 . The medium of  claim 8 , comprising the instructions stored thereon that, when executed by the processor, further cause the processor to:
 determine that an object that is uploaded to the bucket and that the object has metadata satisfying the metadata relationship; and   responsive to determining that the uploaded object has metadata satisfying the metadata relationship, add, to the temporary bucket, a second reference to the object.   
     
     
         13 . The medium of  claim 8 , wherein each object of the first list of objects includes immutable data. 
     
     
         14 . The medium of  claim 8 , comprising the instructions stored thereon that, when executed by the processor, further cause the processor to:
 execute the first SQL query to generate the first list of objects having an object prefix matching a prefix specified in the first SQL query.   
     
     
         15 . A computer-implemented method comprising:
 receiving, by a processor, from a client, a first structured query language (SQL) query;   executing, by the processor, the first SQL query against a metadata server that stores metadata of a bucket, the first SQL query executed to create a temporary bucket that references a first list of objects included in the bucket, the first list of objects having metadata satisfying a metadata relationship specified in the first SQL query;   receiving, by the processor, from the client, a second SQL query;   executing, by the processor, the second SQL query against the temporary bucket to filter the first list of objects and generate a second list of objects having a fewer number of objects than the first list of objects; and   providing, by the processor, to the client, the second list of objects.   
     
     
         16 . The method of  claim 15 , wherein the first SQL query and the second SQL query are S3 application programming interface (API) calls. 
     
     
         17 . The method of  claim 15 , wherein the bucket resides in an S3-compliant object storage system. 
     
     
         18 . The method of  claim 15 , further comprising:
 executing, by the processor, the first SQL query to create the temporary bucket that includes a symbolic link to the first list of objects included in the bucket.   
     
     
         19 . The method of  claim 15 , further comprising:
 determining, by the processor, that an object that is uploaded to the bucket and that the object has metadata satisfying the metadata relationship; and   responsive to determining that the uploaded object has metadata satisfying the metadata relationship, adding, by the processor, to the temporary bucket, a second reference to the object.   
     
     
         20 . The method of  claim 15 , wherein each object of the first list of objects includes immutable data.