Maintaining stable tracking of objects in domain shift scenarios is crucial for RGB-T tracking,
prompting us to explore the use of unlabeled test sample information for effective online model
adaptation. However, current Test-Time Adaptation (TTA) methods in RGB-T tracking dramatically
change the model's internal parameters during long-term adaptation. At the same time, the
gradient computations involved in the optimization process impose a significant computational
burden. To address these challenges, we propose a Parameter Update-Recovery Adaptation (PURA)
framework based on parameter decomposition. Firstly, our fast parameter update strategy adjusts
model parameters using statistical information from test samples without requiring gradient
calculations, ensuring consistency between the model and test data distribution. Secondly, our
parameter decomposition recovery employs orthogonal decomposition to identify the principal
update direction and recover parameters in this direction, aiding in the retention of critical
knowledge. Finally, we leverage the information obtained from decomposition to provide feedback
on the momentum during the update phase, ensuring a stable updating process. Experimental
results demonstrate that PURA outperforms current state-of-the-art methods across multiple
datasets, validating its effectiveness.