US10110868B2ActiveUtilityA1
Image processing to determine center of balance in a digital image
Est. expiryDec 22, 2036(~10.4 yrs left)· nominal 20-yr term from priority
Inventors:Amir R. Hariri
H04N 23/63H04N 23/12H04N 23/88H04N 23/633H04N 23/611G06T 7/66H04N 23/62H04N 23/64G06F 21/32G06T 7/90G06T 2207/30201H04N 5/23293G06T 2200/04G06T 19/006H04N 9/07G06F 21/6218G06T 7/0002G06T 7/13G06F 3/04845H04N 5/23222G06T 2207/30168H04N 5/23216H04N 9/735G06T 2207/10024G06F 3/04842G06T 2210/22
45
PatentIndex Score
1
Cited by
40
References
25
Claims
Abstract
An image processing system acquires a digital image and determines notional weights for each pixel in the image. The notional weight of each pixel relates to the pixel's color value of the pixel and may include adjustments. The system uses the notional weights for each pixel to determine a center of balance (CoB) for the image. The system may then determine an eccentricity of the CoB based on a distance between the CoB and a specified location in the digital image. The system may calculate an aesthetic score for the digital image as a function of the determined eccentricity. The system may display an augmented reality overlay on the image to show the CoB and/or other determined features. The system may use the CoB, eccentricity or aesthetic value to help improve a real-time image capture process, in a security protocol, and/or to determine whether to retain an image in memory.
Claims
exact text as granted — not AI-modifiedThe invention claimed is:
1. A method of processing a digital image to assess image quality, the method comprising, by a processor:
accessing a digital image containing pixel data representing a digital image, wherein:
the pixel data comprises one or more color values for each pixel of a group of pixels, and
each color value corresponds to a color channel, such that each of the pixels is associated with a color channel that corresponds to the color value in the pixel data for the pixel;
for each of the color channels:
determining a first notional weight for each pixel that is associated with the color channel, wherein the first notional weight is the color value or a function of the color value of the pixel,
for each pixel that is associated with the color channel, determining an adjusted notional weight for the pixel by adjusting the first notional weight for the pixel by (i) an edge coefficient having a value corresponding to a proximity of the pixel to a detected edge in the digital image, and (ii) a location coefficient having a value corresponding to a proximity of the pixel to a determined point of interest in the digital image, and
determining an initial center of balance (CoB) for the color channel as a function of the adjusted notional weights of each of the pixels that are associated with the color channel;
determining a final CoB for the digital image based on the initial CoBs for each of the color channels; and
using the final CoB to cause a device to take an action.
2. The method of claim 1 , wherein:
the color channels comprise R, G and B channels in accordance with an RGB color model; and
determining the final CoB for the digital image based on the initial CoBs for each of the color channels comprises determining a weighted average of the initial CoBs for each of the color channels.
3. The method of claim 1 , wherein:
the color channels consist of a single greyscale color channel; and
determining the final CoB for the digital image comprises determining that the initial CoB for the single greyscale color channel is the final CoB.
4. The method of claim 1 , wherein the image is a two-dimensional image, and determining the initial CoB for each color channel comprises determining the initial CoB for each color channel to be:
CoB
x
,
y
=
∑
i
=
1
n
(
x
i
,
y
i
)
*
w
i
(
t
)
adj
∑
i
=
1
n
w
i
(
t
)
adj
wherein:
i represents each pixel that is associated with the color channel, and n is a number of pixels that are associated with the color channel;
(x i ,y i ) represents coordinates in the digital image of each pixel that is associated with the color channel;
w i (t) adj is the adjusted notional weight of each pixel that is associated with the color channel; and
CoB x,y represents a location (x,y) of the CoB.
5. The method of claim 1 , wherein the image is a three-dimensional image, and determining the initial CoB for each color channel comprises determining the initial CoB for each color channel to be:
CoB
x
,
y
,
z
=
∑
i
=
1
n
(
x
i
,
y
i
,
z
i
)
*
w
i
(
t
)
adj
∑
i
=
1
n
w
i
(
t
)
adj
wherein:
i represents each image element that is associated with the color channel, and n is a number of image elements that are associated with the color channel;
(x i ,y i , z i ) represents coordinates in the digital image of each pixel that is associated with the color channel;
w i (t) adj is the adjusted notional weight of each pixel that is associated with the color channel; and
CoB x,y,z represents a location (x,y,z) of the CoB.
6. The method of claim 1 , wherein, for each of the color channels:
determining the first notional weight for each pixel comprises determining the first notional weight for the pixel as w(t) 1 =255−t, wherein t is the color value for the color channel of the pixel;
the method further comprises determining a second notional weight for the color channel as w(t) 2 =t;
determining the initial CoB for the color channel comprises using the adjusted notional weights to determine a first CoB for the color channel and using additional adjusted notional weights based on the second notional weight to determine a second CoB for the color channel; and
determining the final CoB for the color channel comprises determining a weighted average of the first CoB and the second CoB for the color channel.
7. The method of claim 1 , further comprising, by a camera:
using an image sensor to acquire the digital image; and
generating an image matrix comprising the pixel data representing the digital image.
8. The method of claim 7 , wherein using the final CoB to cause a device to take an action comprises:
determining whether the final CoB is proximate to an optimal CoB in the digital image; and
if the final CoB is not proximate to the optimal CoB, cause a user interface of the camera to:
output an instruction to prompt a user of the camera to adjust a position or a zoom setting of the camera, and
acquire a second digital image after the position or the zoom setting is adjusted.
9. The method of claim 1 , wherein using the final CoB to cause a device to take an action comprises:
using the final CoB to determine an aesthetic score for the digital image; and
causing a user interface to output the aesthetic score.
10. The method of claim 9 , wherein using the final CoB to determine an aesthetic score for the digital image comprises:
determining an eccentricity of the final CoB based on a distance between the final CoB and a specified location in the digital image; and
calculating the aesthetic score as a function of the determined eccentricity.
11. The method of claim 10 , wherein the specified location comprises a point that is the center of the digital image, a point that is on a major axis of the digital image, a rule of third power point in the digital image, or a function of any or all such points.
12. The method of claim 9 , further comprising using the aesthetic score to provide, via a user interface of an electronic device, feedback for an image capture or editing process.
13. The method of claim 12 , wherein using the final CoB to cause a device to take an action further comprises one or more of the following:
causing a display device to a output visual indicator of the eccentricity;
causing a display device to a output a visual indicator of the aesthetic score; or
causing a display device to a output an augmented reality overlay showing the CoB on the digital image.
14. The method of claim 12 , wherein using the final CoB to cause a device to take an action further comprises causing a display device to a output an augmented reality overlay showing the CoB on the digital image, and the method further comprises:
displaying guidelines on the display that outputs the digital image and the augmented reality overlay; and
enabling a user of an electronic device that includes the display to use the guidelines to crop the digital image.
15. The method of claim 9 , wherein using the final CoB to cause a device to take an action further comprises using the aesthetic score to determine whether to retain the digital image in a data store by:
determining whether the aesthetic score exceeds a threshold;
determining whether the digital image contains a face of a subject;
determining whether the final CoB appears within the face of the subject; and
retaining the digital image in the data store only if the aesthetic score exceeds the threshold, the digital image contains the face of the subject and the final CoB appears within the face of the subject, otherwise deleting or not causing the digital image to be retained in the data store.
16. The method of claim 1 , wherein using the final CoB to cause a device to take an action comprises:
receiving a user identifier for a user whose face appears in the digital image; and
identifying a code that corresponds to the final CoB; and
using the user identifier and the code as credentials to permit the user to access a secure system.
17. The method of claim 1 , wherein using the final CoB to cause a device to take an action comprises:
identifying a content item;
placing the content item in the image at the CoB to yield a modified image; and
saving the modified image to a memory or causing a display device to output the modified image.
18. The method of claim 1 , wherein using the final CoB to cause a device to take an action comprises causing a display of an electronic device to display a heat map that shows each pixel of the digital image as a color having a value that corresponds to the adjusted notional weight for that pixel.
19. A method of processing a digital image to assess image quality, the method comprising, by a processor:
acquiring a digital image containing pixel data representing an image, wherein the pixel data comprises one or more color values for each pixel of a group of pixels;
determining a notional weight for each pixel as the color value or a function of the color value of the pixel;
using the notional weight for each pixel to determine a center of balance (CoB) for the digital image;
determining an eccentricity of the CoB based on a distance between the CoB and a specified location in the digital image; and
calculating an aesthetic score for the digital image as a function of the determined eccentricity; and
using the CoB or aesthetic score to cause a device to take an action.
20. The method of claim 19 , wherein using the CoB or aesthetic score to cause a device to take an action comprises using the aesthetic score to provide, via a user interface of an electronic device, feedback in an image capture or editing process.
21. The method of claim 19 , wherein using the CoB or aesthetic score to cause a device to take an action comprises:
receiving a user identifier for a user whose face appears in the digital image; and
identifying a code that corresponds to the final CoB; and
using the code as an element of an access code to access a secure system.
22. The method of claim 19 , wherein using the CoB or aesthetic score to cause a device to take an action comprises:
identifying a content item;
placing the content item in the image at the CoB to yield a modified image; and
saving the modified image to a memory or causing a display device to output the modified image.
23. A digital image processing system, comprising:
a camera;
a processor; and
a computer-readable medium containing programming instructions that are configured to cause the processor to:
detect that the camera is acquiring a digital image containing pixel data representing an image, wherein the pixel data comprises one or more color values for each pixel of a group of pixels,
in real-time while the camera is acquiring the digital image:
determine a notional weight for each pixel as the color value or a function of the color value of the pixel,
use the notional weight for each pixel to determine a center of balance (CoB) for the digital image, and
use the CoB to cause a component of the system to take an action.
24. The system of claim 23 , further comprising additional programming instructions that are configured to cause the processor to, in real-time while the camera is acquiring the digital image:
determine an eccentricity of the CoB based on a distance between the CoB and a specified location in the digital image; and
calculate an aesthetic score for the digital image as a function of the determined eccentricity.
25. The system of claim 23 , further comprising:
a display; and
wherein the programming instructions to use the CoB to cause a component of the system to take an action comprise programming instructions that are configured to cause the processor to, in real-time while the camera is acquiring the digital image:
cause the display to output the digital image, and
cause the display to output an augmented reality overlay identifying the CoB.Join the waitlist — get patent alerts
Track US10110868B2 — get alerts on status changes and closely related new filings.
We store only your email — no account needed. See our privacy policy.