Antenna behaviors such as mutual coupling, near-field propagation, and polarization cannot be neglected in signal and channel models for wireless communication. We present an electromagnetic-based array manifold that accounts for several complicated behaviors and can model arbitrary antenna configurations. We quantize antennas into a large number of Hertzian dipoles to develop a model for the radiated array field. The resulting abstraction provides a means to predict the electric field for general non-homogeneous array geometries through a linear model that depends on the point source location, the position of each Hertzian dipole, and a set of coefficients obtained from electromagnetic simulation. We then leverage this model to formulate a beamforming gain optimization that can be adapted to account for polarization of the receive field as well as constraints on the radiated power density. Numerical results demonstrate that the proposed method achieves accuracy that is close to that of electromagnetic simulations. By leveraging the developed array manifold for beamforming, systems can achieve higher beamforming gains compared to beamforming with less accurate models.