Dynamic adaptation of device interfaces in a voice-based system

Abstract

Implementations relate to dynamic adaptation of images for projection by a projector, based on one or more properties of user(s) that are in an environment with the projector. The projector can be associated with an automated assistant client of a client device. In some versions of those implementations, a pose of a user in the environment is determined and, based on the pose, a base image for projecting onto a surface is warped to generate a transformed image. The transformed image, when projected onto a surface and viewed from the pose of the user, mitigates perceived differences relative to the base image.

1 - 20 . (canceled) 
     
     
         21 . A system to generate interfaces in audio-based environments, comprising:
 a data processing system having one or more components coupled with memory, the data processing system to:
 receive an input audio signal of a user acquired via a sensor of a client device, the client device communicatively coupled with the client device; 
 parse the input audio signal to identify a request for content to present via the projector; 
 select, from a plurality of base digital components, a base digital component using the request identified from the input audio signal, the base digital component having a first set of image frames; 
 identify a distance between the user and a projection surface upon which the projector is to project; 
 determine, based on the distance between the user and the projection surface, a transformation parameter to set at least one of a rotation, a scaling, a size, or a skew of the first set of image frames of the base digital component for projection onto the projection surface; 
 generate a second set of image frames based on the first set of image frames of the base digital component and the transformation parameter; and 
 provide the second set of image frames to the projector to project onto the projection surface. 
   
     
     
         22 . The system of  claim 21 , comprising the data processing system to:
 determine, concurrent to the projection of at least one of the second set of image frames, for a change in the distance between the user and the projection surface is greater than a threshold distance; and   update, responsive to the determination, the transformation parameter based on the change in the distance.   
     
     
         23 . The system of  claim 21 , comprising the data processing system to:
 identify a plurality of distance between a corresponding plurality of users and the projector; and   determine, based on at least a subset of the plurality of distances, the transformation parameter for the first set of image frames of the base digital component.   
     
     
         24 . The system of  claim 21 , comprising the data processing system to:
 identify a pose of the user relative to the projection surface, the pose including at least one of a position or an orientation of the user; and   determine the transformation parameter based on the pose of the user relative to the projection surface.   
     
     
         25 . The system of  claim 21 , comprising the data processing system to:
 determine that a pose of the user relative to the projection surface satisfies a threshold condition; and   select, responsive to the determination that the pose of the user satisfies the threshold condition, the base digital component including one or more interactive interface elements.   
     
     
         26 . The system of  claim 21 , comprising the data processing system to select, from the plurality of base digital components, the base digital component based on the distance identified between the user and the projection surface. 
     
     
         27 . The system of  claim 21 , comprising the data processing system to:
 parse the input audio signal to identify one or more keywords;   select, from the plurality of base digital components, a second base digital component using the one or more keywords, the second base digital component including a third set of image frames;   generate a fourth set of image frames based on the third set of image frames of the second base digital component and the transformation parameter; and   provide the fourth set of image frames to the projector to project onto the projection surface with the second set of image frames.   
     
     
         28 . A system to generate interfaces in audio-based environments, comprising:
 a client device having one or more processors coupled with memory, the client device communicatively coupled with a projector, the client device to:
 receive, via a sensor, an input audio signal of a user; 
 transmit, to a data processing system, the input audio signal to cause the data processing system to identify a request for content from parsing the input audio signal and select a base digital component from a plurality of base digital components based on the request; 
 receive, from the data processing system, the base digital component including a first set of image frames; 
 identify a distance between the user and a projection surface upon which the projector is to project; 
 identify a transformation parameter to set at least one of a rotation, a scaling, a size, or a skew of the first set of image frames of the base digital component for projection onto the projection surface in accordance with the distance between the user and the projection surface; 
 generate a second set of image frames based on the first set of image frames of the base digital component and the transformation parameter; and 
 provide the second set of image frames to the projector to project onto the projection surface. 
   
     
     
         29 . The system of  claim 28 , comprising the client device to:
 determine, concurrent to the projection of at least one of the second set of image frames, for a change in the distance between the user and the projection surface is greater than a threshold distance; and   update, responsive to the determination, the transformation parameter based on the change in the distance.   
     
     
         30 . The system of  claim 28 , comprising the client device to:
 identify a plurality of distances between a corresponding plurality of users and the projector; and   identify, in accordance with at least a subset of the plurality of distances, the transformation parameter for the first set of image frames of the base digital component.   
     
     
         31 . The system of  claim 28 , comprising the client device to:
 identify a pose of the user relative to the projection surface, the pose including at least one of a position or an orientation of the user; and   identify the transformation parameter based on the pose of the user relative to the projection surface.   
     
     
         32 . The system of  claim 28 , comprising the client device to:
 determine that a pose of the user relative to the projection surface satisfies a threshold condition; and   cause the data processing system to select, responsive to the determination that the pose of the user satisfies the threshold condition, the base digital component including one or more interactive interface elements.   
     
     
         33 . The system of  claim 28 , comprising the client device to:
 determine that a pose of the user relative to the projection surface satisfies a threshold condition; and   cause the data processing system to select, responsive to the determination that the pose of the user satisfies the threshold condition, the base digital component including one or more interactive interface elements.   
     
     
         34 . The system of  claim 28 , comprising the client device to:
 transmit, to the data processing system, the input audio signal to cause the data processing system to identify one or more keywords from parsing the input audio signal and select a second base digital component from the plurality of base digital components using the one or more keywords, the second base digital component including a third set of image frames;   generate a fourth set of image frames based on the third set of image frames of the second base digital component and the transformation parameter; and   provide the fourth set of image frames to the projector to project onto the projection surface with the second set of image frames.   
     
     
         35 . A method of generating interfaces in audio-based environments, comprising:
 receiving, by a data processing system, an input audio signal of a user acquired via a sensor of a client device, the client device communicatively coupled with the client device;   parsing, by the data processing system, the input audio signal to identify a request for content to present via the projector;   selecting, by the data processing system, from a plurality of base digital components, a base digital component using the request identified from the input audio signal, the base digital component having a first set of image frames;   identifying, by the data processing system, a distance between the user and a projection surface upon which the projector is to project;   determining, by the data processing system, based on the distance between the user and the projection surface, a transformation parameter to adjust at least one of a rotation, a scaling, a size, and a skew of the first image frame of the base digital component projected onto the projection surface;   generating, by the data processing system, a second set of image frames based on the first set of image frames of the base digital component and the transformation parameter; and   providing, by the data processing system, the second set of image frames to the projector to project onto the projection surface.   
     
     
         36 . The method of  claim 35 , comprising:
 determining, by the data processing system, concurrent to the projection of at least one of the second set of image frames, for a change in the distance between the user and the projection surface is greater than a threshold distance; and   updating, by the data processing system, responsive to the determination, the transformation parameter based on the change in the distance.   
     
     
         37 . The method of  claim 35 , comprising:
 identifying, by the data processing system, a plurality of distance between a corresponding plurality of users and the projector; and   determining, by the data processing system, based on at least a subset of the plurality of distances, the transformation parameter for the first set of image frames of the base digital component.   
     
     
         38 . The method of  claim 35 , comprising
 identifying, by the data processing system, a pose of the user relative to the projection surface, the pose including at least one of a position or an orientation of the user; and   determining, by the data processing system, the transformation parameter based on the pose of the user relative to the projection surface.   
     
     
         39 . The method of  claim 35 , comprising:
 determining, by the data processing system, that a pose of the user relative to the projection surface satisfies a threshold condition; and   selecting, by the data processing system, responsive to the determination that the pose of the user satisfies the threshold condition, the base digital component including one or more interactive interface elements.   
     
     
         40 . The method of  claim 35 , comprising:
 parsing, by the data processing system, the input audio signal to identify one or more keywords; and   selecting, by the data processing system, from the plurality of base digital components, a second base digital component using the one or more keywords, the second base digital component including a third set of image frames;   generating, by the data processing system, a fourth set of image frames based on the third set of image frames of the second base digital component and the transformation parameter; and   providing, by the data processing system, the fourth set of image frames to the projector to project onto the projection surface with the second set of image frames.