Systems and methods for balancing haptics and graphics rendering processing with content adaptation

Abstract

Systems and methods are presented herein for balancing rendering of haptics and graphics of a version of media content from a server at a client device. At a server, a request is received for a media asset for interaction on a haptic enabled device. Based on the request, haptic feedback settings of the haptic enabled device associated with the request are determined. The server transmits to the haptic enabled device a version of the media asset comprising haptic feedback rendering criteria compatible with the haptic enabled device. The version of the media asset is comprised of graphics and haptics rendering criteria transmittable over a current network connection strength between the server and the haptic enabled device and also incorporates user interaction data. The haptic enabled device renders the version of the media asset based on the haptic feedback settings.

1 . (canceled) 
     
     
         2 . A method comprising:
 receiving performance requirements associated with a plurality of assets to be displayed at an extended reality (XR) device;   displaying, at the XR device, the plurality of assets, wherein a particular asset of the plurality of assets corresponds to a first haptics fidelity;   identifying first performance metrics associated with the displaying of the plurality of assets;   based at least in part on determining that the first performance metrics do not meet the performance requirements:
 determining that user interaction data corresponding to the XR device does not indicate any tactile interaction with the particular asset; 
 selecting an alternate version of the particular asset, wherein the alternate version of the particular asset corresponds to a second haptics fidelity lower than the first haptics fidelity; and 
 displaying, at the XR device, an adjusted plurality of assets comprising the alternate version of the particular asset, such that second performance metrics associated with the displaying of the adjusted plurality of assets meet the performance requirements. 
   
     
     
         3 . The method of  claim 2 , further comprising determining that the user interaction data corresponding to the XR device indicates a visual interaction with the particular asset, wherein selecting the alternate version of the particular asset is based on the indicated visual interaction. 
     
     
         4 . The method of  claim 2 , wherein the particular asset is a first particular asset of the plurality of assets, wherein a second particular second asset of the plurality of assets corresponds to a first graphics fidelity, and wherein the method further comprises:
 based at least in part on determining that the first performance metrics do not meet the performance requirements:
 determining that the user interaction data corresponding to the XR device does not indicate a visual interaction with the second particular asset; and 
 selecting an alternate version of the second particular asset, wherein the alternate version of the second particular asset corresponds to a second graphics fidelity lower than the first graphics fidelity, and wherein the adjusted plurality of assets further comprises the alternate version of the second particular asset. 
   
     
     
         5 . The method of  claim 4 , further comprising determining that the user interaction data corresponding to the XR device indicates a tactile interaction with the second particular asset, wherein selecting the alternate version of the second particular asset is based on the indicated tactile interaction. 
     
     
         6 . The method of  claim 4 , wherein displaying the plurality of assets comprises:
 identifying graphics capabilities of the XR device; and   selecting a version of the second particular asset corresponding to the first graphics fidelity, wherein the selected version comprises a highest graphics fidelity version of the particular asset suitable for the identified graphics capabilities of the XR device.   
     
     
         7 . The method of  claim 2 , further comprising receiving metadata associated with the plurality of assets, wherein the metadata indicates a plurality of alternate versions of the particular asset, and wherein each alternate version of the plurality of alternate versions corresponds to at least one of a distinct haptics fidelity or a distinct graphics fidelity. 
     
     
         8 . The method of  claim 2 , wherein displaying the plurality of assets comprises:
 identifying haptics capabilities of the XR device; and   selecting a version of the particular asset corresponding to the first haptics fidelity, wherein the selected version comprises a highest haptics fidelity version of the particular asset suitable for the identified haptics capabilities of the XR device.   
     
     
         9 . The method of  claim 2 , wherein one of the first performance metrics or the second performance metrics comprise at least one of:
 bitrate metrics;   error metrics;   rebuffering metrics;   asset availability metrics;   a startup time; or   video start failure metrics.   
     
     
         10 . The method of  claim 2 , wherein:
 the particular asset corresponding to the first haptics fidelity comprises a polygon mesh and a surface texture; and   the alternate version of the particular asset corresponding to the second haptics fidelity comprises a simple geometry and no tactile details.   
     
     
         11 . The method of  claim 2 , wherein determining that the user interaction data does not indicate any tactile interaction with the particular asset comprises determining that the particular asset is outside of a range of focus of the XR device. 
     
     
         12 . A system comprising:
 input/output (I/O) circuitry configured to:
 receive performance requirements associated with a plurality of assets to be displayed at an extended reality (XR) device; and 
   control circuitry configured to:
 display, at the XR device, the plurality of assets, wherein a particular asset of the plurality of assets corresponds to a first haptics fidelity; 
 identify first performance metrics associated with the displaying of the plurality of assets; 
 based at least in part on determining that the first performance metrics do not meet the performance requirements:
 determine that user interaction data corresponding to the XR device does not indicate any tactile interaction with the particular asset; 
 select an alternate version of the particular asset, wherein the alternate version of the particular asset corresponds to a second haptics fidelity lower than the first haptics fidelity; and 
 display, at the XR device, an adjusted plurality of assets comprising the alternate version of the particular asset, such that second performance metrics associated with the displaying of the adjusted plurality of assets meet the performance requirements. 
 
   
     
     
         13 . The system of  claim 12 , wherein the control circuitry is further configured to determine that the user interaction data corresponding to the XR device indicates a visual interaction with the particular asset, and wherein the control circuitry is configured to select the alternate version of the particular asset based on the indicated visual interaction. 
     
     
         14 . The system of  claim 12 , wherein the particular asset is a first particular asset of the plurality of assets, wherein a second particular second asset of the plurality of assets corresponds to a first graphics fidelity, and wherein the control circuitry is further configured to:
 based at least in part on determining that the first performance metrics do not meet the performance requirements:
 determine that the user interaction data corresponding to the XR device does not indicate a visual interaction with the second particular asset; and 
 select an alternate version of the second particular asset, wherein the alternate version of the second particular asset corresponds to a second graphics fidelity lower than the first graphics fidelity, and wherein the adjusted plurality of assets further comprises the alternate version of the second particular asset. 
   
     
     
         15 . The system of  claim 14 , wherein the control circuitry is configured to determine that the user interaction data corresponding to the XR device indicates a tactile interaction with the second particular asset, and wherein the control circuitry is configured to select the alternate version of the second particular asset based on the indicated tactile interaction. 
     
     
         16 . The system of  claim 14 , wherein the control circuitry is configured to display the plurality of assets by:
 identifying graphics capabilities of the XR device; and   selecting a version of the second particular asset corresponding to the first graphics fidelity, wherein the selected version comprises a highest graphics fidelity version of the particular asset suitable for the identified graphics capabilities of the XR device.   
     
     
         17 . The system of  claim 12 , wherein the I/O circuitry is configured to receive metadata associated with the plurality of assets, wherein the metadata indicates a plurality of alternate versions of the particular asset, and wherein each alternate version of the plurality of alternate versions corresponds to at least one of a distinct haptics fidelity or a distinct graphics fidelity. 
     
     
         18 . The system of  claim 12 , wherein the control circuitry is configured to display the plurality of assets by:
 identifying haptics capabilities of the XR device; and   selecting a version of the particular asset corresponding to the first haptics fidelity, wherein the selected version comprises a highest haptics fidelity version of the particular asset suitable for the identified haptics capabilities of the XR device.   
     
     
         19 . The system of  claim 12 , wherein one of the first performance metrics or the second performance metrics comprise at least one of:
 bitrate metrics;   error metrics;   rebuffering metrics;   asset availability metrics;   a startup time; or   video start failure metrics.   
     
     
         20 . The system of  claim 12 , wherein:
 the particular asset corresponding to the first haptics fidelity comprises a polygon mesh and a surface texture; and   the alternate version of the particular asset corresponding to the second haptics fidelity comprises a simple geometry and no tactile details.   
     
     
         21 . A method comprising:
 receiving performance requirements associated with a plurality of assets to be displayed at an extended reality (XR) device;   displaying, at the XR device, the plurality of assets, wherein a particular asset of the plurality of assets corresponds to a first graphics fidelity;   identifying first performance metrics associated with the displaying of the plurality of assets;   based at least in part on determining that the first performance metrics do not meet the performance requirements:
 determining that user interaction data corresponding to the XR device does not indicate any visual interaction with the particular asset; 
 selecting an alternate version of the particular asset, wherein the alternate version of the particular asset corresponds to a second graphics fidelity lower than the first graphics fidelity; and 
 displaying, at the XR device, an adjusted plurality of assets comprising the alternate version of the particular asset, such that second performance metrics associated with the displaying of the adjusted plurality of assets meet the performance requirements.