System and method for providing customized connected device functionality and for operating a connected device via an alternate object

Abstract

A system and method in a mobile device for controlling a connected device is disclosed. In one embodiment, the method includes receiving an image; processing the image to identify at least a first image attribute where the first image attribute includes an image feature or an image object; accessing a database of connected devices, each connected device in the database being identified by a linked image and being associated with device metadata describing at least a device type of the connected device; comparing the first image attribute with the linked images corresponding to the connected devices stored in the database; identifying a first connected device having a linked image matching the first image attribute; and in response to the identifying, providing a user interface configured to control the first connected device, the user interface including a control function corresponding to the device type of the first connected device.

What is claimed is: 
     
         1 . A method in a mobile device for controlling a connected device, comprising:
 receiving, at the mobile device, an image;   processing, at the mobile device, the image to identify at least a first image attribute, the first image attribute comprising an image feature or an image object;   accessing, by the mobile device, a database of connected devices, each connected device in the database being identified by a linked image and being associated with device metadata describing at least a device type of the connected device;   constructing or updating a map of an environment in the database based on a geolocation of the mobile device and the first image attribute;   comparing the first image attribute with the linked images corresponding to the connected devices stored in the database;   identifying a subset of the connected devices associated with the environment;   selecting a first connected device among the subset of connected devices; and   in response to the selecting, providing, at the mobile device, a user interface configured to control the first connected device, the user interface comprising a control function corresponding to the device type of the first connected device.   
     
     
         2 . The method of  claim 1 , wherein receiving, at the mobile device, the image comprises:
 activating, at the mobile device, a control application; and   detecting, at the control application on the mobile device, the image.   
     
     
         3 . The method of  claim 1 , wherein:
 accessing the database of connected devices comprises accessing the database of connected devices including a first set of connected devices, each connected device in the first set of connected devices being identified by a linked image of the respective connected device itself; and   identifying the first connected device comprises identifying the first connected device having a linked image matching the first image attribute, the first image attribute being associated with the first connected device.   
     
     
         4 . The method of  claim 1 , wherein:
 accessing the database of connected devices comprises accessing the database of connected devices including a second set of dummy devices, each dummy device in the second set of dummy devices being linked to a respective connected device and being identified by a linked image of an object that is not the respective connected device; and   identifying the first connected device comprises identifying a first dummy device having a linked image matching the first image attribute and retrieving the linked connected device of the first dummy device as the first connected device.   
     
     
         5 . The method of  claim 1 , further comprising:
 activating, at the mobile device, a control application;   receiving, at the control application on the mobile device, an image of a connected device;   processing, at the mobile device, the image to identify at least a second image attribute, the second image attribute comprising an image feature or an image object;   receiving, at the mobile device, device metadata associated with the connected device;   registering, using the mobile device, the connected device using the second image attribute as the linked image and the device metadata; and   storing, at the database, the connected device identified by the linked image and being associated with the device metadata.   
     
     
         6 . The method of  claim 5 , further comprising:
 activating, at the mobile device, a control application;   receiving, at the control application on the mobile device, an image of a dummy device;   processing, at the mobile device, the image to identify at least a third image attribute, the third image attribute comprising an image feature or an image object;   receiving, at the mobile device, device metadata associated with the dummy device;   registering, using the mobile device, the dummy device using the third image attribute as the linked image and the device metadata;   storing, at the database, the dummy device identified by the linked image and being associated with the device metadata.   providing the list of connected devices from the database;   receiving, at the mobile device, a selection of a connected device; and   storing, at the database, linking data of the dummy device linking the selected connected device to the dummy device.   
     
     
         7 . The method of  claim 1 , wherein accessing the database of connected devices comprises:
 storing the database on the mobile device; and   accessing the database from the mobile device.   
     
     
         8 . The method of  claim 1 , wherein accessing the database of connected devices comprises:
 storing the database on a hub installed in the environment and associated with one or more connected devices in the environment; and   accessing the database on the hub through a local area network.   
     
     
         9 . The method of  claim 1 , wherein accessing the database of connected devices comprises:
 storing the database on a remote server; and   accessing the database on the remote server through a data network.   
     
     
         10 . The method of  claim 1 , wherein processing the image to identify at least a first image attribute comprises:
 processing the image to identify plurality of objects in the image; and   receiving, on the mobile device, a selection of one of the plurality of objects to be processed to identify the first image attribute.   
     
     
         11 . The method of  claim 1 , wherein processing the image to identify at least a first image attribute comprises:
 processing the image to identify a foreground object as the first image attribute or processing the image to identify a foreground object with a given background image as the first image attribute.   
     
     
         12 . The method of  claim 1 , wherein processing the image to identify at least a first image attribute comprises processing the image to identify a face image of a human as the first image attribute, and the method further comprises:
 accessing, by the mobile device, a second database of registered users, each user being identified by a linked face image and being associated with usage data for one or more connected devices;   identifying a first registered user having a linked face image matching the face image identified in the image;   in response to the identifying, providing, at the mobile device, a second user interface including a list of most frequently used connected devices associated with the first registered user; and   in response to receiving an input selecting a second connected device on the second user interface, providing, at the mobile device, the user interface configured to control the second connected device, the user interface comprising a control function corresponding to the device type of the second connected device.   
     
     
         13 . The method of  claim 1 , further comprising:
 determining, at the mobile device, the geolocation of the mobile device;   accessing, by the mobile device, the database of connected devices, each connected device being associated with a geolocation and usage data;   in response to the determining, providing, at the mobile device, a third user interface including a list of connected devices associated with the geolocation; and   in response to receiving an input selecting a third connected device on the third user interface, providing, at the mobile device, the user interface configured to control the third connected device, the user interface comprising a control function corresponding to the device type of the third connected device.   
     
     
         14 . The method of  claim 13 , wherein each connected device in the database is further associated with a user and providing the third user interface including the list of connected devices associated with the geolocation comprises:
 providing the third user interface including the list of connected devices associated with the geolocation and a given user.   
     
     
         15 . A system for controlling a connected device, comprising:
 an imaging sensing device configured to receive an image;   a processor;   a communication interface;   a display; and   a memory coupled with the processor, wherein the memory is configured to provide the processor with instructions which when executed cause the processor to:
 receive an image; 
 process the image to identify at least a first image attribute, the first image attribute comprising an image feature or an image object; 
 access a database of connected devices, each connected device in the database being identified by a linked image and being associated with device metadata describing at least a device type of the connected device; 
 construct or update a map of an environment in the database based on a geolocation of the mobile device and the first image attribute; 
 compare the first image attribute with the linked images corresponding to the connected devices stored in the database; 
 identify a subset of the connected devices associated with the environment; 
 select a first connected device among the subset of connected devices and 
 in response to selection of the first connected device, provide a user interface configured to control the first connected device, the user interface comprising a control function corresponding to the device type of the first connected device. 
   
     
     
         16 . The system recited in  claim 15 , wherein the memory is further configured to provide the processor with instructions which when executed cause the processor to:
 access the database of connected devices including a first set of connected devices, each connected device in the first set of connected devices being identified by a linked image of the respective connected device itself; and   identify the first connected device having a linked image matching the first image attribute, the first image attribute being associated with the first connected device.   
     
     
         17 . The system recited in  claim 15 , wherein the memory is further configured to provide the processor with instructions which when executed cause the processor to:
 access the database of connected devices including a second set of dummy devices, each dummy device in the second set of dummy devices being linked to a respective connected device and being identified by a linked image of an object that is not the respective connected device; and   identify a first dummy device having a linked image matching the first image attribute and retrieving the linked connected device of the first dummy device as the first connected device.   
     
     
         18 . A method for controlling a connected device, comprising:
 receiving, at a remote server, an image from a mobile device;   processing, at the remote server, the image to identify at least a first image attribute, the first image attribute comprising an image feature or an image object;   accessing, by the remote server, a database of connected devices, each connected device in the database being identified by a linked image and being associated with device metadata describing at least a device type of the connected device;   constructing or updating a map of an environment in the database based on a geolocation of the mobile device and the first image attribute;   comparing, at the remote server, the first image attribute with the linked images corresponding to the connected devices stored in the database;   identifying, at the remote server, a subset of the connected devices associated with the environment;   selecting a first connected device among the subset of connected devices; and   in response to the selecting, providing, by the remote server to the mobile device, a user interface configured to control the first connected device, the user interface comprising a control function corresponding to the device type of the first connected device.   
     
     
         19 . The method of  claim 18 , wherein:
 accessing the database of connected devices comprises accessing the database of connected devices including a first set of connected devices, each connected device in the first set of connected devices being identified by a linked image of the respective connected device itself; and   identifying the first connected device comprises identifying the first connected device having a linked image matching the first image attribute, the first image attribute being associated with the first connected device.   
     
     
         20 . The method of  claim 18 , wherein:
 accessing the database of connected devices comprises accessing the database of connected devices including a second set of dummy devices, each dummy device in the second set of dummy devices being linked to a respective connected device and being identified by a linked image of an object that is not the respective connected device; and   identifying the first connected device comprises identifying a first dummy device having a linked image matching the first image attribute and retrieving the linked connected device of the first dummy device as the first connected device.