System and method to reduce bandwidth requirement for visibility event packet streaming using a predicted maximal view frustum and predicted maximal viewpoint extent, each computed at runtime

Abstract

There is provided a method of predictive prefetching and transmitting from a server to a client device at least one partial visibility event packet and/or deferred visibility event packet including renderable graphics information occluded from a first viewcell and not occluded from a second viewcell, including otherwise renderable graphics information in a client view frustum not previously transmitted to the client device; determining an estimated maximal client view frustum; calculating a subset comprising renderable graphics information that is included in the estimated maximal client view frustum; determining whether the calculated subset has previously been transmitted to the client device by comparing the calculated subset to the stored renderable graphics information previously transmitted; and transmitting the at least one partial visibility event packet and/or deferred visibility event packet to the client device if said packet has not been previously transmitted to the client device.

I claim: 
     
         1 . A method, conducted on a server, of predictive prefetching and transmitting at least one partial visibility event packet from the server to a client device, the at least one partial visibility event packet being a subset of a complete visibility event packet, the complete visibility event packet including renderable graphics information occluded from a first viewcell and not occluded from a second viewcell among a plurality of viewcells, the method comprising:
 a) storing, using storage circuitry, information representing a current client view frustum;   b) storing, using the storage circuitry, renderable graphics information previously transmitted to the client device;   c) determining, using a processor, from the information representing the current client view frustum, an estimated maximal client view frustum comprising a volume of space intersected by a maximum possible movement of the current client view frustum during a delay period of time at least equal to a round-trip-time of data communication between the server and the client device;   d) calculating, using the processor, the subset of the complete visibility event packet comprising renderable graphics information that is included in the estimated maximal client view frustum;   e) determining, using the processor, whether the calculated subset has previously been transmitted to the client device by comparing the calculated subset to the stored renderable graphics information previously transmitted to the client device; and   f) transmitting, to the client device, the at least one partial visibility event packet comprising the calculated subset of the complete visibility event packet, if the calculated subset has not been previously transmitted to the client device.   
     
     
         2 . The method according to  claim 1 , wherein the information representing the client view frustum includes a current client view direction vector. 
     
     
         3 . The method according to  claim 2 , wherein the information representing the client view frustum includes a current client viewpoint location. 
     
     
         4 . The method according to  claim 3 , further comprising determining the estimated maximal client view frustum by:
 c1) determining, using the processor, a conservative maximal viewpoint extent comprising a representation of a maximal possible subregion of the second viewcell occupied by a client viewpoint during the delay period;   c2) determining, using the processor, a conservative maximal view direction vector rotation comprising a representation of a maximal rotational extent of the client view direction vector during the delay period; and   c3) calculating, using the processor, the estimated maximal client view frustum from the conservative maximal viewpoint extent and the conservative maximal view direction vector rotation.   
     
     
         5 . The method according to  claim 4 , wherein the information representing the client view frustum includes a current velocity of the current client viewpoint. 
     
     
         6 . The method according to  claim 4 , wherein the information representing the client view frustum includes a current acceleration of the current client viewpoint. 
     
     
         7 . The method according to  claim 5 , wherein the determining the conservative maximal viewpoint extent is based on the current client viewpoint location, the current velocity of the client viewpoint, and a predetermined maximal value for viewpoint velocity. 
     
     
         8 . The method according to  claim 4 , wherein the information representing the client view frustum includes a current angular velocity of the current client view direction vector. 
     
     
         9 . The method according to  claim 4 , wherein the information representing the client view frustum includes a current angular acceleration of the current view direction vector. 
     
     
         10 . The method according to  claim 9 , wherein the conservative maximal view direction vector rotation is based on the current client view direction vector, the current angular velocity of the current client view direction vector, and a predetermined maximal value for the client view direction vector angular velocity. 
     
     
         11 . The method according to  claim 4 , further comprising:
 g) storing, using the storage circuitry, a complete data set for each viewcell of the plurality of viewcells that is predictively penetrated by the client viewpoint, as determined by the conservative maximal viewpoint extent, the complete data set representing a complete potentially visible set for said each viewcell of the plurality of viewcells;   h) storing, using the storage circuitry, an incomplete data set for said each viewcell of the plurality of viewcells that is predictively penetrated by the client viewpoint, as determined by the conservative maximal viewpoint extent, the incomplete data set representing, for said each viewcell, a subset of the calculated subset actually transmitted to the client device in step f);   i) calculating, using the processor, at least one deferred visibility event packet for said each viewcell of the plurality of viewcells that is predictively penetrated by the client viewpoint, by comparing the complete data set to the incomplete data set, the at least one deferred visibility event packet including, for said each viewcell, a subset of the complete potentially visible set not previously transmitted to the client device;   j) determining, using the processor, for said each viewcell of the plurality of viewcells that is predictively penetrated by the client viewpoint, whether renderable graphics information of the at least one deferred visibility event packet intersects the estimated maximal client view frustum; and   k) transmitting, to the client device, the at least one deferred visibility event packet when the renderable graphics information of the at least one deferred visibility event packet intersects the estimated maximal client view frustum.   
     
     
         12 . A method, conducted on a client device, of receiving at least one partial visibility event packet from a server, the at least one partial visibility event packet being a subset of a complete visibility event packet, the complete visibility event packet including renderable graphics information occluded from a first viewcell and not occluded from a second viewcell among a plurality of viewcells, the method comprising:
 a) transmitting to the server, using a processor, client view information representing a current client view frustum;   b) transmitting to the server, using the processor, data comprising at least one of a current client view direction vector and a current client viewpoint location; and   c) receiving, from the server, the at least one partial visibility event packet comprising renderable graphics information that intersects a maximal client view frustum,   wherein the maximal client view frustum comprises a volume of space intersected by a maximum possible movement of the current client view frustum, based on said at least one of the current client view direction vector and the current client viewpoint location, during a delay period of time at least equal to a round-trip-time of data communication between the server and the client device.   
     
     
         13 . The method according to  claim 12 , further comprising:
 d) receiving, from the server, at least one deferred visibility event packet for said each viewcell of the plurality of viewcells that is predictively penetrated by the client viewpoint, the at least one deferred visibility event packet including, for said each viewcell, a subset of the complete potentially visible set not previously received from the server,   wherein the subset includes renderable graphics information of the at least one deferred visibility event packet that intersects the maximal client view frustum.   
     
     
         14 . A method, conducted on a client device, of receiving at least one partial visibility event packet from a server, the at least one partial visibility event packet being a subset of a complete visibility event packet, the complete visibility event packet including renderable graphics information occluded from a first viewcell and not occluded from a second viewcell among a plurality of viewcells, the method comprising:
 a) determining, using a processor, client view information representing a current client view frustum and at least one of a current client view direction vector and a current client viewpoint location;   b) determining, from the client view information, a maximal view frustum comprising a volume of space intersected by a maximum possible movement of the current client view frustum, based on said at least one of the current client view direction vector and the current client viewpoint location, during a delay period of time at least equal to a round-trip-time of data communication between the server and the client device;   c) transmitting to the server, using the processor, data representing the maximal view frustum; and   d) receiving, from the server, the at least one partial visibility event packet comprising renderable graphics information that intersects the maximal client view frustum.   
     
     
         15 . A method, conducted on a server, of predictive prefetching and transmitting at least one partial visibility event packet from the server to a client device, the at least one partial visibility event packet being a subset of a complete visibility event packet, the complete visibility event packet including renderable graphics information occluded from a first viewcell and not occluded from a second viewcell among a plurality of viewcells, the method comprising:
 a) receiving, from the client device, client view information representing a current client view frustum and at least one of a current client view direction vector and a current client viewpoint location;   b) determining, using a processor, from the client view information, an estimated maximal client view frustum comprising a volume of space intersected by a maximum possible movement of the current client view frustum during a delay period of time at least equal to a round-trip-time of data communication between the server and the client device;   c) calculating, using the processor, the subset of the complete visibility event packet comprising renderable graphics information that is included in the estimated maximal client view frustum; and   d) transmitting, at a first time, to the client device, a reduced level-of-detail representation of the at least one partial visibility event packet comprising the calculated subset of the complete visibility event packet that intersects the estimated maximal view frustum.   
     
     
         16 . The method according to  claim 15 , wherein the first time is a time or a time period when only the reduced level-of-detail representation of the renderable graphics information can be visually perceived. 
     
     
         17 . The method according to  claim 15 , further comprising:
 e) transmitting, at a second time later than the first time, to the client device, the at least one partial visibility event packet comprising the calculated subset as a high level-of-detail representation of the subset of the renderable graphics information.   
     
     
         18 . The method according to  claim 17 , wherein the second time is another time or another time period when the high level-of-detail representation of the renderable graphics information can be visually perceived. 
     
     
         19 . The method according to  claim 17 , wherein a level-of-detail of the high level-of-detail representation is proportional to human visual acuity as a function of visual exposure time. 
     
     
         20 . The method according to  claim 17 , wherein a level-of-detail of the high level-of-detail representation is inversely proportional to human visual acuity as a function of image-plane velocity of renderable surfaces included in the at least one visibility event packet. 
     
     
         21 . The method according to  claim 17 , further comprising:
 f) transmitting, at a third time, to the client device, at least one deferred visibility event packet for said each viewcell of the plurality of viewcells that is predictively penetrated by the client viewpoint, the at least one deferred visibility event packet including, for said each viewcell, a subset of the complete visibility event packet that intersects the estimated maximal view frustum and that has not been previously transmitted to the client device,   wherein the complete visibility event packet comprises an additive combination of the at least one partial visibility event packet and the at least one deferred visibility event packet.   
     
     
         22 . The method according to  claim 15 , further comprising determining the estimated maximal client view frustum by:
 b1) determining, using the processor, a conservative maximal viewpoint extent comprising a representation of a maximal possible subregion of the second viewcell occupied by a client viewpoint during the delay period;   b2) determining, using the processor, a conservative maximal view direction vector rotation comprising a representation of a maximal rotational extent of a client view direction vector during the delay period; and   b3) calculating, using the processor, the estimated maximal client view frustum from the conservative maximal viewpoint extent and the conservative maximal view direction vector rotation.   
     
     
         23 . The method according to  claim 22 , further comprising:
 e) storing, using storage circuitry, a complete data set for each viewcell of the plurality of viewcells that is predictively penetrated by the client viewpoint, as determined by the conservative maximal viewpoint extent, the complete data set representing a complete potentially visible set for said each viewcell of the plurality of viewcells;   f) storing, using the storage circuitry, an incomplete data set for said each viewcell of the plurality of viewcells that is predictively penetrated by the client viewpoint, as determined by the conservative maximal viewpoint extent, the incomplete data set representing, for said each viewcell, a subset of the calculated subset actually transmitted to the client device in step d);   g) calculating, using the processor, at least one deferred visibility event packet for said each viewcell of the plurality of viewcells that is predictively penetrated by the client viewpoint, by comparing the complete data set to the incomplete data set, the at least one deferred visibility event packet including, for said each viewcell, a subset of the complete potentially visible set not previously transmitted to the client device;   h) determining, using the processor, for said each viewcell of the plurality of viewcells that is predictively penetrated by the client viewpoint, whether renderable graphics information of the at least one deferred visibility event packet intersects the estimated maximal client view frustum; and   i) transmitting, to the client device, the at least one deferred visibility event packet when the renderable graphics information of the at least one deferred visibility event packet intersects the estimated maximal client view frustum.   
     
     
         24 . The method according to  claim 23 , further comprising:
 j) transmitting, at a third time, to the client device, a reduced level-of-detail representation of the at least one deferred visibility event packet comprising the calculated subset of the complete visibility event packet that intersects the estimated maximal view frustum.   
     
     
         25 . The method according to  claim 24 , wherein the third time is a time or a time period when only the reduced level-of-detail representation of the renderable graphics information can be visually perceived. 
     
     
         26 . The method according to  claim 24 , further comprising:
 k) transmitting, at a fourth time later than the third time, to the client device, the at least one deferred visibility event packet comprising the calculated subset as a high level-of-detail representation of the subset of the renderable graphics information.   
     
     
         27 . The method according to  claim 26 , wherein the fourth time is another time or another time period when the high level-of-detail representation of the renderable graphics information can be visually perceived. 
     
     
         28 . The method according to  claim 26 , wherein a level-of-detail of the high level-of-detail representation is proportional to human visual acuity as a function of visual exposure time. 
     
     
         29 . The method according to  claim 26 , wherein a level-of-detail of the high level-of-detail representation is inversely proportional to human visual acuity as a function of image-plane velocity of renderable surfaces included in the at least one visibility event packet.