Weird galaxies are outliers that have either unknown or very uncommon features making them different from the normal sample. These galaxies are very interesting as they may provide new insights into current theories, or can be used to form new theories about processes in the Universe. Interesting outliers are often found by accident, but this will become increasingly more difficult with future big surveys generating an enormous amount of data. This gives the need for machine learning detection techniques to find the interesting weird objects. In this work, we inspect the galaxy spectra of the third data release of the Galaxy And Mass Assembly survey and look for the weird outlying galaxies using two different outlier detection techniques. First, we apply distance-based Unsupervised Random Forest on the galaxy spectra using the flux values as input features. Spectra with a high outlier score are inspected and divided into different categories such as blends, quasi-stellar objects, and BPT outliers. We also experiment with a reconstruction-based outlier detection method using a variational autoencoder and compare the results of the two different methods. At last, we apply dimensionality reduction techniques on the output of the methods to inspect the clustering of similar spectra. We find that both unsupervised methods extract important features from the data and can be used to find many different types of outliers.