Dementia poses a growing challenge in our aging society. Frontotemporal dementia (FTD) and Alzheimer disease (AD) are the leading causes of early-onset dementia. FTD and AD display unique traits in their onset, progression, and treatment responses. In particular, FTD often faces a prolonged diagnostic process and is commonly misdiagnosed with AD due to overlapping symptoms. This study utilizes a complex network model of brain electrical activity using resting-state EEG recordings to address the misdiagnosis. It compares the network organization between AD and FTD, highlighting connectivity differences and examining the significance of EEG signals across frequency bands in distinguishing AD and FTD. The publicly available EEG dataset of 36 AD and 23 FTD patients is utilized for analyses. Cross-plot transition entropy (CPTE) is employed to measure synchronization between EEG signals and construct connection matrices. CPTE offers advantages in parameter setting, computational efficiency, and robustness. The analysis reveals significantly different clustering coefficients (CC), subgraph centrality (SC), and eigenvector centrality (EC) between the two groups. FTD shows higher connectivity, particularly in delta, theta, and gamma bands, owing to lower neurodegeneration. The CPTE-based network parameters effectively classify the two groups with an accuracy of 87.58\%, with the gamma band demonstrating the highest accuracy of 92.87%. Consequently, CPTE-based, complex network analysis of EEG data from AD and FTD patients reveals significant differences in brain network organization. The approach shows potential for identifying unique characteristics and providing insights into the underlying pathophysiological processes of the various forms of dementia, thereby assisting in accurate diagnosis and treatment.