Robust multi-kernelized correlators for UAV tracking with adaptive context analysis and dynamic weighted filters

Abstract

In recent years, the correlation filter (CF)-based method has significantly advanced in the tracking for unmanned aerial vehicles (UAVs). As the core component of most trackers, CF is a discriminative classifier to distinguish the object from the surrounding environment. However, the poor representation of the object and lack of contextual information have restricted the tracker to gain better performance. In this work, a robust framework with multi-kernelized correlators is proposed to improve robustness and accuracy simultaneously. Both convolutional features extracted from the neural network and hand-crafted features are employed to enhance expressions for object appearances. Then, the adaptive context analysis strategy helps filters to effectively learn the surrounding information by introducing context patches with the GMSD index. In the training stage, multiple dynamic filters with time-attenuated factors are introduced to avoid tracking failure caused by dramatic appearance changes. The response maps corresponding to different features are finally fused before the novel resolution enhancement operation to increase distinguishing capability. As a result, the optimization problem is reformulated, and a closed-form solution for the proposed framework can be obtained in the kernel space. Extensive experiments on 100 challenging UAV tracking sequences demonstrate that the proposed tracker outperforms other 23 state-of-the-art trackers and can effectively handle unexpected appearance variations under the complex and constantly changing working conditions.

Appendix

In this section, a more detailed derivation from Eq. (5) to Eq. (8) is presented.

Because all operations in the Fourier domain are performed element-wise, each element of $\hat{\mathbf{w }}_{n}^{*}$ (indexed by u) can be solved independently, and Eq. (5) can be decomposed as the subproblem $\hat{{\mathcal {E}}}_{nu}$, which is defined as follows:

$$\begin{aligned} \begin{aligned} \hat{{\mathcal {E}}}_{nu} =&\big \Vert {\hat{K}}_{u}^{n0} {\hat{w}}_{nu}^{*} - {\hat{y}}_{nu} \big \Vert _2^2 + \lambda _{1} \big \Vert {\hat{w}}_{nu}^{*} \big \Vert _2^2 \\&+ \sum _{s =1}^{S} \Big \Vert F_{ns} {\hat{K}}_{u}^{ns} {\hat{w}}_{nu}^{*} \Big \Vert _2^2 \\&+ \sum _{t = 1}^{T} \Big \Vert \gamma _t \big [{\hat{w}}_{nu}^{*} - \hat{{\widetilde{w}}}_{nu}^{*}(t) \big ] \Big \Vert _2^2 \end{aligned} \ , \end{aligned}$$

(20)

where ${\hat{K}}_{u}^{n0} = {\hat{k}}_{u}^\mathbf{x _{n0} \mathbf{x} _{n0}}$ and ${\hat{K}}_{u}^{ns} = {\hat{k}}_{u}^\mathbf{x _{ns} \mathbf{x} _{ns}}$ are used to simplify the denotation. Then, Eq. (20) can be expanded according to the property of the vector operation, that is equivalent to

$$\begin{aligned} \begin{aligned} \begin{aligned} A_1&= \big \Vert {\hat{K}}_{u}^{n0} {\hat{w}}_{nu}^{*} - {\hat{y}}_{nu} \big \Vert _2^2 \\&= {\hat{K}}_{u}^{n0*} {\hat{w}}_{nu} {\hat{K}}_{u}^{n0} {\hat{w}}_{nu}^{*} + {\hat{y}}_{nu}^{*} {\hat{y}}_{nu} \\&\quad - {\hat{K}}_{u}^{n0} {\hat{w}}_{nu}^{*} {\hat{y}}_{nu}^{*} - {\hat{K}}_{u}^{n0*} {\hat{w}}_{nu} {\hat{y}}_{nu} \\ A_2&= \sum _{t = 1}^{T} \Big \Vert \gamma _t \big [{\hat{w}}_{nu}^{*} - \hat{{\widetilde{w}}}_{nu}^{*}(t) \big ] \Big \Vert _2^2 \\&= \sum _{t = 1}^{T} \Big \{ {\gamma _t}^2 \big [ {\hat{w}}_{nu} {\hat{w}}_{nu}^{*} + \hat{{\widetilde{w}}}_{nu}(t) \hat{{\widetilde{w}}}_{nu}^{*}(t) \\&\quad - {\hat{w}}_{nu} \hat{{\widetilde{w}}}_{nu}^{*}(t) - {\hat{w}}_{nu}^{*} \hat{{\widetilde{w}}}_{nu}(t) \big ] \Big \} \\ A_3&= \lambda _{1} \big \Vert {\hat{w}}_{nu}^{*} \big \Vert _2^2 = \lambda _{1} {\hat{w}}_{nu} {\hat{w}}_{nu}^{*} \\ A_4&= \sum _{s = 1}^{S} \Big \Vert F_{ns} {\hat{K}}_{u}^{ns} {\hat{w}}_{nu}^{*} \Big \Vert _2^2 \\&= \sum _{s = 1}^{S} \Big ( {F_{ns}}^2 {\hat{K}}_{u}^{ns*} {\hat{w}}_{nu} {\hat{K}}_{u}^{ns} {\hat{w}}_{nu}^{*} \Big ). \end{aligned} \end{aligned} \end{aligned}$$

(21)

Therefore, the solution to the optimization target can be calculated by setting the first derivative of ${\hat{w}}_{nu}^{*}$ to zero, i.e.,

$$\begin{aligned} \frac{\partial \hat{{\mathcal {E}}}_{nu}}{\partial {\hat{w}}_{nu}^{*}} = \frac{\partial A_1}{\partial {\hat{w}}_{nu}^{*}} + \frac{\partial A_2}{\partial {\hat{w}}_{nu}^{*}} + \frac{\partial A_3}{\partial {\hat{w}}_{nu}^{*}} + \frac{\partial A_4}{\partial {\hat{w}}_{nu}^{*}} = 0 \ . \end{aligned}$$

(22)

Hence, Eq. (22) can be reformulated as follows:

$$\begin{aligned} &\Big [ {\hat{K}}_{u}^{n0*} {\hat{K}}_{u}^{n0} + \sum \nolimits_{t = 1}^{T} {\gamma _t}^2 + \lambda _{1} + \sum \nolimits_{s = 1}^{S} \Big ( {F_{ns}}^2 {\hat{K}}_{u}^{ns*} {\hat{K}}_{u}^{ns} \Big ) \Big ] {\hat{w}}_{nu} \\&\quad= {\hat{K}}_{u}^{n0} {\hat{y}}_{nu}^{*} + \sum \nolimits_{t = 1}^{T} \Big [ {\gamma _t}^2 \hat{{\widetilde{w}}}_{nu}(t) \Big ] . \end{aligned}$$

(23)

A closed-form solution to ${\hat{w}}_{nu}^{*}$ can be obtained:

$$\begin{aligned} {\hat{w}}_{nu}^{*} = \frac{ {\hat{K}}_{u}^{n0} {\hat{y}}_{nu}^{*} + \sum \nolimits _{t = 1}^{T} \Big [ {\gamma _t}^2 \hat{{\widetilde{w}}}_{nu}(t) \Big ] }{ {\hat{K}}_{u}^{n0} {\hat{K}}_{u}^{n0*} + \sum \nolimits _{t = 1}^{T} {\gamma _t}^2 + \lambda _{1} + \sum \nolimits _{s = 1}^{S} \Big ( {F_{ns}}^2 {\hat{K}}_{u}^{ns} {\hat{K}}_{u}^{ns*} \Big ) } \ . \end{aligned}$$

(24)

which is the sub-solution of $\hat{\mathbf{w}}_{n}^{*}$ in Eq. (8). $\square$