Anomaly detection aims to identify abnormal patterns in data, and it has significant application value in various fields such as industrial inspection. The current mainstream anomaly detection methods employ unsupervised models such as auto-encoder. These models use fully connected layers or convolutional layers for the data processing during encoding and decoding, which can lead to problems such as lack of interpretability and semantic errors. This paper proposes a combination method of the semi-nonnegative matrix factorization model and network training to design a semi-nonnegative matrix factorization neural network for anomaly detection. Due to the characteristic of the semi-nonnegative matrix factorization model that “local superposition constitutes the whole”, this network can better preserve semantic information and is also interpretable. Additionally, the feature matrix of this network is updated as weights during the training of the network, which effectively solves the problem of local optimal solutions existed in the traditional semi-nonnegative matrix factorization model. The anomaly detection performance of this network was tested on three datasets. The experiment results show that it outperforms mainstream auto-encoder and variational auto-encoder methods by more than 3 percentage points in continuous data, and achieves comparable results in discrete data. Compared with the detection method based on the traditional semi-nonnegative matrix factorization model, this network has significantly improved in all detection metrics, with the highest improvement reaching 12%. This network is a beneficial exploration that utilizes traditional matrix factorization model to construct neural network, and it can effectively solve the anomaly detection problem.