In this paper, we propose an improved Bayesian bidirectional long-short term memory (BiLSTM) neural networks for multi-step ahead (MSA) solar generation forecasting. The proposed technique applies alpha-beta divergence for a more appropriate consideration of outliers in the solar generation data and resulting variability of the weight parameter distribution in the neural network. The proposed method is examined on highly granular solar generation data from Ausgrid using probabilistic evaluation metrics such as Pinball loss and Winkler score. Moreover, a comparative analysis between MSA and the single-step ahead (SSA) forecasting is provided to test the effectiveness of the proposed method on variable forecasting horizons. The numerical results clearly demonstrate that the proposed Bayesian BiLSTM with alpha-beta divergence outperforms standard Bayesian BiLSTM and other benchmark methods for MSA forecasting in terms of error performance.