Taking into account the dynamics of the scene is the most effective solution to obtain an accurate perception of unknown environments within the framework of a real autonomous robotic application. Many works have attempted to address the non-rigid scene assumption by taking advantage of deep learning advancements. Most new methods combine geometric and semantic approaches to determine dynamic elements that lack generalization and scene awareness. We propose a novel approach that overcomes the limitations of these methods by using scene depth information that refines the accuracy of estimates from geometric and semantic modules. In addition, the depth information is used to determine an area of influence of dynamic objects through our Objects Interaction module that estimates the state of both non-matched keypoints and out of segmented region keypoints. The obtained results demonstrate the efficacy of the proposed method in providing accurate localization and mapping in dynamic environments.