US2025329104A1PendingUtilityA1
Visual-inertial gaussian splatting simultaneous localization and mapping (vi-slam) system with dense reconstruction
Est. expiryApr 19, 2044(~17.7 yrs left)· nominal 20-yr term from priority
G01C 21/1656G06T 7/74G06T 2210/56G06T 2207/30244G06T 17/00G01C 21/16G06T 2207/10024G06T 15/08G06T 2207/20076G06T 7/579G06T 7/80G06T 2207/10028G06T 2207/10016
64
PatentIndex Score
0
Cited by
0
References
0
Claims
Abstract
A method for visual-inertial simultaneous localization and mapping (VI-SLAM) may estimate an inertial measurement unit (IMU) bias using Interactive Closest Point (ICP) for estimating an acceleration bias. The method may estimate a camera pose of a current frame of a camera of the IMU using Red, Green, Blue (RGB) images and IMU measurements in real-time. The method may further perform dense mapping by managing a Gaussian map by optimizing 3D Gaussian parameters, expanding the Gaussian map, and compressing the Gaussian map by removing redundant Gaussians.
Claims
exact text as granted — not AI-modified1 . A method for visual-inertial simultaneous localization and mapping (VI-SLAM), comprising:
estimating an inertial measurement unit (IMU) bias using Interactive Closest Point (ICP) for estimating an acceleration bias; estimating a camera pose of a current frame of a camera of the IMU using Red, Green, Blue (RGB) images and IMU measurements in real-time; and performing dense mapping by managing a Gaussian map by optimizing 3D Gaussian parameters, expanding the Gaussian map, and compressing the Gaussian map by removing redundant Gaussians.
2 . The method of claim 1 , wherein estimating an inertial measurement unit (IMU) bias comprises initializing the Gaussian map based on a first frame of the camera.
3 . The method of claim 1 , wherein estimating the IMU bias comprises estimating a gyroscope bias using epipolar constraints within a first predetermined number N frames of data.
4 . The method of claim 3 , wherein estimating the accelerating bias comprises performing IMU integration with the gyroscope bias removed from gyroscope measurements.
5 . The method of claim 3 , wherein the gyroscope bias is estimated by minimizing a smallest eigen value for each vector formed by using a set of normal vectors between two consecutive frames.
6 . The method of claim 3 , wherein the estimated gyroscope bias, b* g , is calculated by an equation:
b
g
*
=
arg
min
b
g
∑
(
k
-
1
,
k
)
∈
ℰ
(
λ
min
(
M
k
-
1
,
k
)
2
M
k
-
1
,
k
=
∑
i
=
1
n
(
f
i
×
R
b
c
γ
^
b
k
b
k
-
1
R
c
b
f
i
′
)
(
f
i
×
R
b
c
γ
^
b
k
b
k
-
1
R
c
b
f
i
′
)
⊤
γ
^
b
k
b
k
-
1
=
γ
b
k
b
k
-
1
Exp
(
J
b
g
γ
b
g
)
(
8
)
wherein (f i , f′ i ) is a pair of bearing vectors in the camera; n is a number of features observed by the camera; R b c is a rotation matrix of the camera—IMU extrinsic calibration; M k−1, k is formed by using normals from the camera; ε is a set of keyframe pairs at two consecutive times;
Y
^
bk
bk
-
1
is estimated using a first-order Taylor approximation of
Y
bk
bk
-
1
and
J
bg
Y
is how a pre-integration of rotation changes due to a difference in the gyroscope bias estimation.
7 . The method of claim 1 , wherein estimating the camera pose comprises estimating the camera pose using a multi-scale ICP with Compute Unified Device Architecture (CUDA) acceleration.
8 . The method of claim 7 , comprising performing rotation-translation-decoupled optimization to provide a starting point.
9 . The method of claim 8 , wherein performing the rotation-translation-decoupled optimization comprises estimating a relative rotation between a previous frame and a current frame using IMU pre-integration of rotation and a camera-IMU calibration matrix.
10 . The method of claim 3 , comprising updating the gyroscope bias during estimation of the camera pose.
11 . The method of claim 10 , wherein updating the gyroscope bias comprises differentially rendering RGB, depth, and silhouette images, and adjusting the gyroscope bias to minimize a loss between the rendered RGB, depth and silhouette images and corresponding input images while keeping Gaussian parameters fixed.
12 . The method of claim 1 , determine if an incoming frame is a new keyframe based on average parallax or visual similarity.
13 . The method of claim 12 , performing a densification procedure to determine which pixels are to be added as new Gaussians when the incoming frame is the new keyframe.
14 . The method of claim 12 , comprising:
hashing each incoming RGB image using perceptual hashing; and comparing hash values of the incoming RGB image to hash values of a current keyframe to determine if the incoming RGB image is the new keyframe.
15 . The method of claim 14 , wherein comparing hash values of the incoming RGB image to hash values of the current keyframe comprises using Hamming distance, wherein if the Hamming distance between the incoming RGB image and all current keyframes in the buffer are greater than a predetermined threshold value, the incoming RGB image is the new keyframe.
16 . The method of claim 12 , comprising:
adding new Gaussians from the new keyframe to an existing Gaussians map; and updating the Gaussians map by optimizing parameters of all 3D Gaussians.
17 . A method for visual-inertial simultaneous localization and mapping (VI-SLAM), the method implemented using a control system including a processor communicatively coupled to a memory device, the method comprising: comprising:
estimating an inertial measurement unit (IMU) bias comprising an acceleration bias and a gyroscope bias, wherein the acceleration bias is estimated using Interactive Closest Point (ICP) and the gyroscope bias is estimated using epipolar constraints within a first predetermined number N frames of data, wherein a Gaussian map is initialized based on a first frame of a camera of the IMU; estimating a camera pose of a current frame of the camera using a multi-scale ICP with Compute Unified Device Architecture (CUDA) acceleration and performing rotation-translation-decoupled optimization to provide a starting point and updating the gyroscope bias during estimation of the camera pose; and performing dense mapping by managing the Gaussian map by optimizing 3D Gaussian parameters, expanding the Gaussian map, and compressing the Gaussian map by removing redundant Gaussians.
18 . The method of claim 17 , wherein the estimated gyroscope bias, b* g , is calculated by an equation:
b
g
*
=
arg
min
b
g
∑
(
k
-
1
,
k
)
∈
ℰ
(
λ
min
(
M
k
-
1
,
k
)
2
M
k
-
1
,
k
=
∑
i
=
1
n
(
f
i
×
R
b
c
γ
^
b
k
b
k
-
1
R
c
b
f
i
′
)
(
f
i
×
R
b
c
γ
^
b
k
b
k
-
1
R
c
b
f
i
′
)
⊤
γ
^
b
k
b
k
-
1
=
γ
b
k
b
k
-
1
Exp
(
J
b
g
γ
b
g
)
(
8
)
wherein (f i , f′ i ) is a pair of bearing vectors in the camera; n is a number of features observed by the camera; R b c is a rotation matrix of the camera—IMU extrinsic calibration; M k−1, k is formed by using normals from the camera; ε is a set of keyframe pairs at two consecutive times;
Y
^
bk
bk
-
1
is estimated using a first-order Taylor approximation of
Y
bk
bk
-
1
and
J
bg
Y
is how a pre-integration of rotation changes due to a difference in the gyroscope bias estimation.
19 . The method of claim 17 , wherein updating the gyroscope bias comprises differentially rendering RGB, depth, and silhouette images, and adjusting the gyroscope bias to minimize a loss between the rendered RGB, depth and silhouette images and corresponding input images while keeping Gaussian parameters fixed.
20 . A visual-inertial simultaneous localization and mapping (VI-SLAM) comprising:
an inertial measurement unit (IMU) unit estimating an inertial measurement unit (IMU) bias comprising an acceleration bias and a gyroscope bias, wherein the acceleration bias is estimated using Interactive Closest Point (ICP) and the gyroscope bias is estimated using epipolar constraints within a first predetermined number N frames of data, wherein a Gaussian map is initialized based on a first frame of a camera of the IMU; an optimization unit estimating a camera pose of a current frame of the camera using a multi-scale ICP with Compute Unified Device Architecture (CUDA) acceleration and performing rotation-translation-decoupled optimization to provide a starting point and updating the gyroscope bias during estimation of the camera pose; and a dense reconstruction unit performing dense mapping by managing the Gaussian map by optimizing 3D Gaussian parameters, expanding the Gaussian map, and compressing the Gaussian map by removing redundant Gaussians.Join the waitlist — get patent alerts
Track US2025329104A1 — get alerts on status changes and closely related new filings.
We store only your email — no account needed. See our privacy policy.