Existing methods for detecting abnormal behavior in fish schools had difficulties to extract higher-level semantic information and learn features effectively, or identify key features of anomalies, making them unsuitable for large-scale aquaculture. To address this, we proposed a deep learning-based fish school abnormal behavior detection method combining pseudo-anomaly guidance, fused attention, and memory augmentation. First, a pseudo-anomaly synthesizer was developed to enhance anomaly perception by randomly skipping frames in video sequences to generate pseudo-anomalous samples. Next, the SKFca attention mechanism integrated frequency domain information into the SK attention mechanism to capture richer input features, while the BAM attention mechanism suppressed irrelevant background features in channel and spatial dimensions to emphasize foreground targets. A memory-augmented module replaced with encoded anomaly features with normal samples, amplifying reconstruction errors for anomalies. Finally, memory-augmented key features and enriched frequency domain features were merged to extract comprehensive high-level semantic information. Experiments results on two self-made fish datasets demonstrated superior performance, achieving AUC values of 0.953 and 0.957 with precise anomaly localization.