US2024161319A1PendingUtilityA1
Systems, methods, and media for estimating a depth and orientation of a portion of a scene using a single-photon detector and diffuse light source
Est. expiryNov 9, 2042(~16.3 yrs left)· nominal 20-yr term from priority
G06T 7/13G06T 7/521G01S 7/4865G01S 17/10H04N 5/2256G06T 2207/10028H04N 23/56G01S 17/89G01S 7/4814G01S 7/486
51
PatentIndex Score
0
Cited by
0
References
0
Claims
Abstract
In accordance with some embodiments, systems, methods, and media for estimating a depth and orientation of a portion of a scene using a single-photon detector and diffuse light source are provided. In some embodiments, a system comprises: a light source; an image sensor comprising a pixel having a field of view of at least one degree; a hardware processor programmed to: cause the light source to emit a sequence of n defocused pulses toward the scene; receive, from the pixel, information indicative of arrival times of light from the scene; generate a transient histogram using the information indicative of arrival times of light from the scene; and estimate one or more properties of a portion of the scene within the field of view of the pixel based on the transient histogram, wherein the one or more properties of the portion of the scene includes at least a depth.
Claims
exact text as granted — not AI-modifiedWhat is claimed is:
1 . A system for estimating at least one property in a scene, the system comprising:
a light source; an image sensor comprising a pixel, wherein the pixel has a field of view of at least one degree; at least one hardware processor that is programmed to:
cause the light source to emit a sequence of n defocused pulses toward the scene;
receive, from the pixel, information indicative of arrival times of light from the scene;
generate a transient histogram using the information indicative of arrival times of light from the scene; and
estimate one or more properties of a portion of the scene within the field of view of the pixel based on the transient histogram, wherein the one or more properties of the portion of the scene includes at least a depth.
2 . The system of claim 1 , further comprising a diffuser disposed between the light source and the scene.
3 . The system of claim 1 , wherein the image sensor includes no more than one pixel.
4 . The system of claim 1 , wherein the one or more properties of the portion of the scene includes a normal orientation θ n of the portion of the scene with respect to an optical axis of the pixel.
5 . The system of claim 4 , wherein the at least one hardware processor is further programmed to:
identify a leading edge of a peak in the transient histogram; estimate a first distance D 1 based on the leading edge of the peak in the transient histogram; generate a first estimate of the normal orientation θ n based on D 1 ; identify a trailing edge of a peak in the transient histogram; estimate a second distance D 2 based on the trailing edge of the peak in the transient histogram; generate a second estimate of the normal orientation θ n based on D 2 ; and estimate the normal orientation θ n based on the first estimate and the second estimate.
6 . The system of claim 4 , wherein the at least one hardware processor is further programmed to:
estimate a Fourier transform of the transient histogram, ({tilde over (φ)}), where is the Fourier transform and {tilde over (φ)} is the transient histogram; estimate a Fourier transform of a forward rendering model R(θ n , Z 0 ), (R(θ n , Z 0 )), where Z 0 is a depth of the portion of the scene along the optical axis of the pixel; and solve an optimization problem using gradient decent, wherein the optimization problem is expressed as:
minimize
θ
n
,
Z
0
ℱ
(
R
(
θ
n
,
Z
0
)
)
-
F
(
φ
¯
)
2
2
,
and initial values for θ n and Z 0 are estimated using a leading edge of a peak in the transient histogram and a trailing edge of the peak in the transient histogram,
wherein solving the optimization problem comprises calculating an 2 norm using k Fourier coefficients for ({tilde over (φ)}) and (R(θ n , Z 0 )), where k is less than all of the Fourier coefficients.
7 . The system of claim 1 , wherein the pixel comprises a single photon avalanche diode (SPAD), and
the information indicative of arrival times of light from the scene comprises a plurality of timestamps each indicative of an arrival time of a single photon at the SPAD.
8 . The system of claim 1 , wherein the image sensor includes an array comprising a plurality of pixels including the pixel, and wherein the at least one hardware processor is further programmed to:
receive, from each of the plurality of pixels, information indicative of arrival times of light from the scene; generate a plurality of transient histograms, each of the plurality of transient histograms corresponding to a pixel of the plurality of pixels; estimate a Fourier transform of each transient histogram of the plurality of transient histograms, yielding a plurality of Fourier coefficients for each transient histogram; provide k Fourier coefficients of each of the plurality of Fourier coefficients as input to a trained machine learning model, wherein the trained machine learning model was trained to generate a depth map; and receive, from the trained machine learning model, a depth map for the scene comprising the one or more properties of the portion of the scene.
9 . A method for estimating at least one property in a scene, the method comprising:
causing a light source to emit a sequence of n defocused pulses toward the scene; receiving, from a pixel of an image sensor, information indicative of arrival times of light from the scene, wherein the pixel has a field of video of at least one degree; generating a transient histogram using the information indicative of arrival times of light from the scene; and estimating one or more properties of a portion of the scene within the field of view of the pixel based on the transient histogram, wherein the one or more properties of the portion of the scene includes at least a depth.
10 . The method of claim 9 , wherein a diffuser disposed between the light source and the scene causes the light emitted by the light source to be defocused.
11 . The method of claim 9 , wherein the image sensor includes no more than one pixel.
12 . The method of claim 9 , wherein the one or more properties of the portion of the scene includes a normal orientation θ n of the portion of the scene with respect to an optical axis of the pixel.
13 . The method of claim 12 , further comprising:
identifying a leading edge of a peak in the transient histogram; estimating a first distance D 1 based on the leading edge of the peak in the transient histogram; generating a first estimate of the normal orientation θ n based on D 1 ; identifying a trailing edge of a peak in the transient histogram; estimating a second distance D 2 based on the trailing edge of the peak in the transient histogram; generating a second estimate of the normal orientation θ n based on D 2 ; and estimating the normal orientation θ n based on the first estimate and the second estimate.
14 . The method of claim 12 , further comprising:
estimating a Fourier transform of the transient histogram, ({tilde over (φ)}), where is the Fourier transform and {tilde over (φ)} is the transient histogram; estimating a Fourier transform of a forward rendering model R(θ n , Z 0 ), (R(θ n , Z 0 )), where Z 0 is a depth of the portion of the scene along the optical axis of the pixel; and solving an optimization problem using gradient decent, wherein the optimization problem is expressed as:
minimize
θ
n
,
Z
0
ℱ
(
R
(
θ
n
,
Z
0
)
)
-
F
(
φ
¯
)
2
2
,
and initial values for θ n and Z 0 are estimated using a leading edge of a peak in the transient histogram and a trailing edge of the peak in the transient histogram,
wherein solving the optimization problem comprises calculating an 2 norm using k Fourier coefficients for ({tilde over (φ)}) and (R(θ n , Z 0 )), where k is less than all of the Fourier coefficients.
15 . The method of claim 9 , wherein the pixel comprises a single photon avalanche diode (SPAD), and
the information indicative of arrival times of light from the scene comprises a plurality of timestamps each indicative of an arrival time of a single photon at the SPAD.
16 . The method of claim 9 , wherein the image sensor includes an array comprising a plurality of pixels including the pixel, the method further comprising:
receiving, from each of the plurality of pixels, information indicative of arrival times of light from the scene; generating a plurality of transient histograms, each of the plurality of transient histograms corresponding to a pixel of the plurality of pixels; estimating a Fourier transform of each transient histogram of the plurality of transient histograms, yielding a plurality of Fourier coefficients for each transient histogram; providing k Fourier coefficients of each of the plurality of Fourier coefficients as input to a trained machine learning model, wherein the trained machine learning model was trained to generate a depth map; and receiving, from the trained machine learning model, a depth map for the scene comprising the one or more properties of the portion of the scene.
17 . A non-transitory computer readable medium containing computer executable instructions that, when executed by a processor, cause the processor to perform a method for estimating at least one property in a scene, the method comprising:
causing a light source to emit a sequence of n defocused pulses toward the scene; receiving, from a pixel of an image sensor, information indicative of arrival times of light from the scene, wherein the pixel has a field of video of at least one degree; generating a transient histogram using the information indicative of arrival times of light from the scene; and estimating one or more properties of a portion of the scene within the field of view of the pixel based on the transient histogram, wherein the one or more properties of the portion of the scene includes at least a depth.
18 . The non-transitory computer readable medium of claim 17 , wherein the one or more properties of the portion of the scene includes a normal orientation θ n of the portion of the scene with respect to an optical axis of the pixel.
19 . The non-transitory computer readable medium of claim 18 , wherein the method further comprises:
identifying a leading edge of a peak in the transient histogram; estimating a first distance D 1 based on the leading edge of the peak in the transient histogram; generating a first estimate of the normal orientation θ n based on D 1 ; identifying a trailing edge of a peak in the transient histogram; estimating a second distance D 2 based on the trailing edge of the peak in the transient histogram; generating a second estimate of the normal orientation θ n based on D 2 ; and estimating the normal orientation θ n based on the first estimate and the second estimate.
20 . The non-transitory computer readable medium of claim 18 , further comprising:
estimating a Fourier transform of the transient histogram, ({tilde over (φ)}), where is the Fourier transform and {tilde over (φ)} is the transient histogram; estimating a Fourier transform of a forward rendering model R(θ n , Z 0 ), (R(θ n , Z 0 )), where Z 0 is a depth of the portion of the scene along the optical axis of the pixel; and solving an optimization problem using gradient decent, wherein the optimization problem is expressed as:
minimize
θ
n
,
Z
0
ℱ
(
R
(
θ
n
,
Z
0
)
)
-
F
(
φ
¯
)
2
2
,
and initial values for θ n and Z 0 are estimated using a leading edge of a peak in the transient histogram and a trailing edge of the peak in the transient histogram,
wherein solving the optimization problem comprises calculating an 2 norm using k Fourier coefficients for ({tilde over (φ)}) and (R(θ n , Z 0 )), where k is less than all of the Fourier coefficients.
21 . The non-transitory computer readable medium of claim 17 , wherein the image sensor includes an array comprising a plurality of pixels including the pixel, wherein the method further comprises:
receiving, from each of the plurality of pixels, information indicative of arrival times of light from the scene; generating a plurality of transient histograms, each of the plurality of transient histograms corresponding to a pixel of the plurality of pixels; estimating a Fourier transform of each transient histogram of the plurality of transient histograms, yielding a plurality of Fourier coefficients for each transient histogram; providing k Fourier coefficients of each of the plurality of Fourier coefficients as input to a trained machine learning model, wherein the trained machine learning model was trained to generate a depth map; and receiving, from the trained machine learning model, a depth map for the scene comprising the one or more properties of the portion of the scene.Join the waitlist — get patent alerts
Track US2024161319A1 — get alerts on status changes and closely related new filings.
We store only your email — no account needed. See our privacy policy.