US11830275B1ActiveUtilityA1
Person re-identification method and apparatus, device, and readable storage medium
Assignee: INSPUR SUZHOU INTELLIGENT TECHNOLOGY CO LTDPriority: Jun 29, 2021Filed: Sep 29, 2021Granted: Nov 28, 2023
Est. expiryJun 29, 2041(~14.9 yrs left)· nominal 20-yr term from priority
G06V 40/103G06V 10/82G06V 40/25G06V 40/10G06N 3/045G06F 18/2415G06V 10/454
88
PatentIndex Score
2
Cited by
18
References
18
Claims
Abstract
A person re-identification method and apparatus, a device, and a readable storage medium. A homogeneous training network of an initial person re-identification network is trained by means of an objective function such as a knowledge synergy for dynamic classification probability loss function to obtain a final person re-identification network carrying more accurate final weight parameters, and a person re-identification task is performed by means of the final person re-identification network. In this way, the accuracy and performance of the person re-identification network to process the person re-identification task may be improved, the storage space in a device may be reduced, more beneficial to the storage and deployment of the portable device, and the amount of calculation of performing the person re-identification task may be reduced, thereby increasing the processing rate of the person re-identification task.
Claims
exact text as granted — not AI-modifiedThe invention claimed is:
1. A person re-identification method, comprising:
determining a homogeneous training network corresponding to an initial person re-identification network, wherein the homogeneous training network comprises a plurality of homogeneous branches with a same network structure;
training the homogeneous training network by using a target loss function, and determining a final weight parameter of each network layer in the homogeneous training network, wherein the target loss function comprises a knowledge synergy for dynamic classification probability loss function, and the knowledge synergy for dynamic classification probability loss function is used to determine a unidirectional knowledge synergy loss value between the plurality of homogeneous branches by using classification-layer output features of each training sample in every two homogeneous branches; and
loading the final weight parameter by using the initial person re-identification network to obtain a final person re-identification network, to perform a person re-identification task by using the final person re-identification network;
wherein the training the homogeneous training network by using the target loss function and determining the final weight parameter of each network layer in the homogeneous training network comprises:
during training of the homogeneous training network, determining a cross-entropy loss value of a cross-entropy loss function, determining a triplet loss value of a triplet loss function, and determining the unidirectional knowledge synergy loss value of the knowledge synergy for dynamic classification probability loss function; and
determining the final weight parameter of each network layer in the homogeneous training network by using a total loss value of the cross-entropy loss value, the triplet loss value, and the unidirectional knowledge synergy loss value;
wherein a process of determining the unidirectional knowledge synergy loss value of the knowledge synergy for dynamic classification probability loss function comprises:
calculating the unidirectional knowledge synergy loss value by using the classification-layer output feature of each sample in each homogeneous branch and the knowledge synergy for dynamic classification probability loss function, wherein the knowledge synergy for dynamic classification probability loss function is:
L
ksp
=
min
θ
1
N
∑
n
=
1
N
∑
(
u
,
v
)
∈
ℤ
u
≠
v
∑
k
=
1
K
f
c
k
(
x
n
,
θ
u
)
log
(
f
c
k
(
x
n
,
θ
u
)
f
c
k
(
x
n
,
θ
v
)
)
where L ksp represents the unidirectional knowledge synergy loss value, N represents a total quantity of the training samples, u represents a u th homogeneous branch, ν represents a ν th with homogeneous branch, represents an optional space formed by any two homogeneous branches, K represents a dimension of the classification-layer output feature, x n represents an n th sample, ƒ c k (x n , θ u ) represents a classification-layer output feature of in the k th dimension in the u th homogeneous branch, ƒ c k (x n ,θ ν ) represents a classification-layer output feature of x n in the k th dimension in the ν th with homogeneous branch, θ u represents a network parameter of the u th homogeneous branch, and θ ν represents a network parameter of the ν th homogeneous branch, wherein N and K are positive integers.
2. The person re-identification method according to claim 1 , wherein the determining a homogeneous training network corresponding to the initial person re-identification network comprises:
deriving an auxiliary training branch from an intermediate layer of the initial person re-identification network to generate the homogeneous training network with an asymmetric network structure.
3. The person re-identification method according to claim 1 , wherein the determining a homogeneous training network corresponding to the initial person re-identification network comprises:
deriving an auxiliary training branch from an intermediate layer of the initial person re-identification network to generate a homogeneous training network with a symmetric network structure.
4. The person re-identification method according to claim 1 , wherein a process of determining the triplet loss value of the triplet loss function comprises:
determining a first loss value of each homogeneous branch according to an embedding-layer output feature of each sample in each homogeneous branch and a first triple loss function;
and selecting a first loss value that is numerically minimum from each homogeneous branch as the triplet loss value;
wherein the first triplet loss function is:
L
TriHard
b
=
-
1
N
∑
a
=
1
N
[
max
y
p
=
y
a
d
(
f
e
a
,
f
e
p
)
-
min
y
q
≠
y
a
d
(
f
e
a
,
f
e
q
)
+
m
]
+
,
where L TriHard b represents a first loss value of a b th homogeneous branch, N represents a total quantity of the training samples, a represents an anchor sample, ƒ e a represents an embedding-layer output feature of the anchor sample, y represents a classification tag of the sample, p represents a sample that belongs to a same classification tag as the anchor sample and that is at a maximum intra-class distance from the anchor sample, ƒ e p represents an embedding-layer output feature of the sample p, q represents a sample that belongs to a different classification tag from the anchor sample and that is at a minimum inter-class distance from the anchor sample, ƒ e q represents an embedding-layer output feature of the sample q, m represents a first parameter, d(·,·) is used for calculating a distance, [·] + and max d(·,·) both represent calculation of a maximum distance, min d(·,·) represents calculation of a minimum distance, y a represents a classification tag of the anchor sample, y p represents a classification tag of the sample p, and y q represents a classification tag of the sample q.
5. The person re-identification method according to claim 4 , wherein after the determining a first loss value of each homogeneous branch, the method further comprises:
determining a second loss value of each homogeneous branch by using the first loss value of each homogeneous branch and a second triplet loss function, wherein the second triplet loss function is:
L
E_TriHard
b
=
L
TriHard
b
+
β
1
N
∑
a
=
1
N
(
d
(
f
e
a
,
f
e
p
)
d
(
f
e
a
,
f
e
q
)
)
,
where L E_TriHard b represents a second loss value of the b th homogeneous branch, and β represents a second parameter; and
correspondingly, the selecting a first loss value that is numerically minimum from each homogeneous branch as the triplet loss value comprises:
selecting the second loss value that is numerically minimum from each homogeneous branch as the triplet loss value.
6. The person re-identification method according to claim 5 , wherein the cross-entropy loss function is:
L
c
b
(
x
,
y
,
θ
b
)
=
-
1
N
∑
n
=
1
N
∑
k
=
1
K
y
n
k
log
(
f
c
k
(
x
n
,
θ
b
)
)
,
wherein a network input is represented as D t ={(x n , y n )|n∈[1,N]}, N represents a total quantity of sample images, x n represents an n th image, y n represents a classification tag corresponding to this image, ƒ c (x n θ b ) represents an output feature of a network model, and the subscript c represents obtaining a classification-layer feature of the network after passing through a softmax layer, K represents a dimension of a classification-layer feature vector output by the network, L c b (x, y, θ b ) represents a cross-entropy loss function of a b th homogeneous branch, and θ b represents a network parameter of the b th homogeneous branch.
7. The person re-identification method according to claim 6 , wherein after the second loss value is calculated, the method further comprises:
obtaining a total loss function according to the cross-entropy loss calculated by using the cross-entropy loss function and the triplet loss calculated by using the second triplet loss function, wherein the total loss function is:
L
=
min
θ
∑
b
=
1
B
(
α
b
L
c
b
(
x
,
y
,
θ
b
)
+
γ
b
L
E_TriHard
b
)
where γ represents a hyperparameter, and B represents a quantity of the plurality of homogeneous branches, wherein B is a positive integer.
8. The person re-identification method according to claim 1 , wherein the training of the homogeneous training network comprises:
selecting a derivation position from a backbone network according to a network structure of the initial person Re-ID network, determining an intermediate layer from which an auxiliary training branch is derived, and constructing a homogeneous-network-based auxiliary training branch to obtain the homogeneous training network;
determining the target loss function, and calculating a loss of each homogeneous branch in the homogeneous training network by using the target loss function;
training a network according to the target loss function to converge the network; and
storing a trained weight parameter.
9. The person re-identification method according to claim 8 , wherein the training of the homogeneous training network comprises a first phase and a second phase;
the first phase is a forward propagation phase in which data is propagated from a lower layer to a higher layer, and the second phase is a back propagation phase in which an error is propagated for training from the higher layer to the lower layer when a result obtained by forward propagation is inconsistent with what is expected.
10. The person re-identification method according to claim 9 , wherein the training a network comprises:
initializing a weight of a network layer;
performing forward propagation on input training image data through each network layer, to obtain an output value;
calculating an error between the output value of the network and a target value;
back propagating the error to the network, and sequentially calculating a back propagation error of each network layer;
adjusting, by each network layer, all weight coefficients in the network according to the back propagation error of each layer;
reselecting randomly new training image data, and then performing the step of performing the forward propagation to obtain the output value of the network;
repeating infinitely iteration, and ending the training when an error between the output value of the network and a target value is less than a specific threshold or a quantity of iterations exceeds a specific threshold; and
storing trained network parameters of all layers.
11. The person re-identification method according to claim 10 , wherein the network layers comprise: a convolutional layer, a down-sampling layer, and a fully connected layer.
12. The person re-identification method according to claim 10 , wherein the calculating an error between the output value of the network and a target value comprises:
calculating the output value of the network and obtaining a total loss value based on the target loss function.
13. The person re-identification method according to claim 1 , wherein the loading the final weight parameter by using the initial person re-identification network to obtain a final person re-identification network comprises:
loading the final weight parameter by using the initial person re-identification network without auxiliary training branches to obtain the final person re-identification network.
14. The person re-identification method according to claim 1 , wherein the knowledge synergy for dynamic classification probability loss function is decomposed into a first formula and a second formula;
the first formula is:
L
ksp
u
,
v
=
∑
k
=
1
K
f
c
k
(
x
n
,
θ
u
)
log
(
f
c
k
(
x
n
,
θ
u
)
f
c
k
(
x
n
,
θ
v
)
)
,
and
the second formula is:
L
ksp
=
min
θ
1
N
∑
n
=
1
N
∑
(
u
,
v
)
∈
ℤ
u
≠
v
L
ksp
u
,
v
.
15. The person re-identification method according to claim 1 , wherein the method further comprises:
summing the classification-layer output features of the plurality of homogeneous branches to obtain total classification-layer output features;
calculating an average value of the total classification-layer output features as a virtual label of a virtual branch, wherein a calculation method of the virtual label ƒ ν is:
f
v
=
∑
b
=
1
B
f
c
(
x
n
,
θ
b
)
/
B
where B represents a total quantity of the plurality of homogeneous branches, b represents a b th homogeneous branch, x n represents an n th sample, θ b represents a network parameter of the b th homogeneous branch, and ƒ c (x n ,θ b ) represents a classification-layer output feature of x n in the b th homogeneous branches, wherein B is a positive integer.
16. The person re-identification method according to claim 15 , wherein after the virtual label ƒ ν is calculated, the method further comprises:
calculate the knowledge synergy loss function based on the virtual branches by combining the classification-level output features of the plurality of homogeneous branches with the virtual label ƒ ν , wherein the knowledge synergy loss function based on the virtual branches is:
L
v
=
∑
b
=
1
B
f
v
log
(
f
v
f
c
(
x
n
,
θ
b
)
)
.
17. An electronic device, comprising:
a memory, configured to store a computer program; and
a processor, configured to execute the computer program to implement operations comprising:
determining a homogeneous training network corresponding to an initial person re-identification network, wherein the homogeneous training network comprises a plurality of homogeneous branches with a same network structure;
training the homogeneous training network by using a target loss function, and determining a final weight parameter of each network layer in the homogeneous training network, wherein the target loss function comprises a knowledge synergy for dynamic classification probability loss function, and the knowledge synergy for dynamic classification probability loss function is used to determine a unidirectional knowledge synergy loss value between the plurality of homogeneous branches by using classification-layer output features of each training sample in every two homogeneous branches; and
loading the final weight parameter by using the initial person re-identification network to obtain a final person re-identification network, to perform a person re-identification task by using the final person re-identification network;
wherein the training the homogeneous training network by using a target loss function and determining a final weight parameter of each network layer in the homogeneous training network comprises:
during training of the homogeneous training network, determining a cross-entropy loss value of a cross-entropy loss function, determining a triplet loss value of a triplet loss function, and determining the unidirectional knowledge synergy loss value of the knowledge synergy for dynamic classification probability loss function; and
determining the final weight parameter of each network layer in the homogeneous training network by using a total loss value of the cross-entropy loss value, the triplet loss value, and the unidirectional knowledge synergy loss value;
wherein a process of determining the unidirectional knowledge synergy loss value of the knowledge synergy for dynamic classification probability loss function comprises:
calculating the unidirectional knowledge synergy loss value by using the classification-layer output feature of each sample in each homogeneous branch and the knowledge synergy for dynamic classification probability loss function, wherein the knowledge synergy for dynamic classification probability loss function is:
L
ksp
=
min
θ
1
N
∑
n
=
1
N
∑
(
u
,
v
)
∈
ℤ
u
≠
v
∑
k
=
1
K
f
c
k
(
x
n
,
θ
u
)
log
(
f
c
k
(
x
n
,
θ
u
)
f
c
k
(
x
n
,
θ
v
)
)
where L ksp represents the unidirectional knowledge synergy loss value, N represents a total quantity of the training samples, u represents a u th homogeneous branch, ν represents a ν th with homogeneous branch, represents an optional space formed by any two homogeneous branches, K represents a dimension of the classification-layer output feature, x n represents an n th sample, ƒ c k (x n ,θ u ) represents a classification-layer output feature of x n in the k th dimension in the u th homogeneous branch, ƒ c k (x n ,θ ν ) represents a classification-layer output feature of x n in the k th dimension in the with homogeneous branch, θ u represents a network parameter of the u th homogeneous branch, and θ ν represents a network parameter of the ν th homogeneous branch, wherein N and K are positive integers.
18. A non-transitory computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement operations comprising:
determining a homogeneous training network corresponding to an initial person re-identification network, wherein the homogeneous training network comprises a plurality of homogeneous branches with a same network structure;
training the homogeneous training network by using a target loss function, and determining a final weight parameter of each network layer in the homogeneous training network, wherein the target loss function comprises a knowledge synergy for dynamic classification probability loss function, and the knowledge synergy for dynamic classification probability loss function is used to determine a unidirectional knowledge synergy loss value between the plurality of homogeneous branches by using classification-layer output features of each training sample in every two homogeneous branches; and
loading the final weight parameter by using the initial person re-identification network to obtain a final person re-identification network, to perform a person re-identification task by using the final person re-identification network;
wherein the training the homogeneous training network by using the target loss function and determining the final weight parameter of each network layer in the homogeneous training network comprises:
during training of the homogeneous training network, determining a cross-entropy loss value of a cross-entropy loss function, determining a triplet loss value of a triplet loss function, and determining the unidirectional knowledge synergy loss value of the knowledge synergy for dynamic classification probability loss function; and
determining the final weight parameter of each network layer in the homogeneous training network by using a total loss value of the cross-entropy loss value, the triplet loss value, and the unidirectional knowledge synergy loss value;
wherein a process of determining the unidirectional knowledge synergy loss value of the knowledge synergy for dynamic classification probability loss function comprises:
calculating the unidirectional knowledge synergy loss value by using the classification-layer output feature of each sample in each homogeneous branch and the knowledge synergy for dynamic classification probability loss function, wherein the knowledge synergy for dynamic classification probability loss function is:
L
ksp
=
min
θ
1
N
∑
n
=
1
N
∑
(
u
,
v
)
∈
ℤ
u
≠
v
∑
k
=
1
K
f
c
k
(
x
n
,
θ
u
)
log
(
f
c
k
(
x
n
,
θ
u
)
f
c
k
(
x
n
,
θ
v
)
)
where L ksp represents the unidirectional knowledge synergy loss value, N represents a total quantity of the training samples, u represents a u th homogeneous branch, ν represents a v th homogeneous branch, represents an optional space formed by any two homogeneous branches, K represents a dimension of the classification-layer output feature, x n represents an n th sample, ƒ c k (x n ,θ u ) represents a classification-layer output feature of x n in the k th dimension in the u th homogeneous branch, ƒ c k (x n ,θ ν ) represents a classification-layer output feature of x n in the k th dimension in the ν th homogeneous branch, θ u represents a network parameter of the u th homogeneous branch, and θ ν represents a network parameter of the v th homogeneous branch, wherein N and K are positive integers.Join the waitlist — get patent alerts
Track US11830275B1 — get alerts on status changes and closely related new filings.
We store only your email — no account needed. See our privacy policy.