Inductive node-wise graph incremental learning is a challenging task due to the dynamic nature of evolving graphs and the dependencies between nodes. In this paper, we propose a novel experience replay framework, called Structure-Evolution-Aware Experience Replay (SEA-ER), that addresses these challenges by leveraging the topological awareness of GNNs and importance reweighting technique. Our framework effectively addresses the data dependency of node prediction problems in evolving graphs, with a theoretical guarantee that supports its effectiveness. Through empirical evaluation, we demonstrate that our proposed framework outperforms the current state-of-the-art GNN experience replay methods on several benchmark datasets, as measured by metrics such as accuracy and forgetting.