Active Shape Models (ASM) are an iterative segmentation technique to find a landmark-based contour of an object. In each iteration, a least-squares fit of a plausible shape to some detected target landmarks is determined. Finding these targets is a critical step: some landmarks are more reliably detected than others, and some landmarks may not be within the field of view of their detectors. To add robustness while preserving simplicity at the same time, a generalized least-squares approach can be used, where a weighting matrix incorporates reliability information about the landmarks. We propose a strategy to choose this matrix, based on the covariance of empirically determined residuals of the fit. We perform a further step to determine whether the target landmarks are within the range of their detectors. We evaluate our strategy on fluoroscopic X-ray images to segment the femur. We show that our technique outperforms the standard ASM as well as other more heuristic weighted least-squares strategies.