The terahertz (THz) band is a promising solution to the increasing data traffic demands of future wireless networks. However, developing transceivers for THz communication is a complex and toilsome task due to the difficulty in designing devices that operate at this frequency and the impact of hardware impairments on performance. This paper investigates the impact of radio frequency (RF) impairment, in-phase/quadrature imbalance (IQI). To this end, we express an IQI model for the THzspecific array-of-subarrays (AoSA) architecture considering the unique features of THz communication; vast bandwidth, severe power drawdown, and pencil-like beams. We further model the impact of IQI in the power limited regime in order to investigate the power and ultra-wideband trade-off. To achieve this, we express the spectral efficiency in terms of wideband slope and bit energy to noise ratio which are the two important information theoretic metrics that reveals the performance of the ultrawideband systems as in THz communication. Our results show that THz systems with IQI have a strict limit in achievable rate although they provide immense spectrum. We also demonstrate with our simulation results that compared to low frequencies, IQI is a more serious concern in THz links.