This paper investigates the semantic communication and cooperative tracking control for an UAV swarm comprising a leader UAV and a group of follower UAVs, all interconnected via unreliable wireless multiple-input-multiple-output (MIMO) channels. Initially, we develop a dynamic model for the UAV swarm that accounts for both the internal interactions among the cooperative follower UAVs and the imperfections inherent in the MIMO channels that interlink the leader and follower UAVs. Building on this model, we incorporate the power costs of the UAVs and formulate the communication and cooperative tracking control challenge as a drift-plus-penalty optimization problem. We then derive a closed-form optimal solution that maintains a decentralized semantic architecture, dynamically adjusting to the tracking error costs and local channel conditions within the swarm. Employing Lyapunov drift analysis, we establish closed-form sufficient conditions for the stabilization of the UAV swarm's tracking performance. Numerical results demonstrate the significant enhancements in our proposed scheme over various state-of-the-art methods.