Phytoplankton, one of the most important biological components in water ecosystem, is sensitive to water environment and has been widely concerned in water environment monitoring. However, the aquatic environment is complex and diverse. Accurate and efficient identification of phytoplankton becomes a challenge in monitoring work. Current phytoplankton recognition methods can be divided into three categories:classical morphological classification, molecular markers, and artificial intelligence image recognition. Although the first two methods have been widely used, they are time-consuming and laborious, which is not conducive to the large-scale application and promotion of monitoring agencies. Similarly, it is difficult to strike a balance between high accuracy and high efficiency for automatic classification based on images. Deep learning provides new insights for phytoplankton identification. We propose a new deep convolutional neural network RAN-11. The network is based on the residual attention network Attention-56 and Attention-92, fusing the bottom and top features of the backbone through channel concatenation, simplifying the structure by adjusting the number of attention and residual module, and adopting the Leaky ReLU activation function rather than ReLU. We used 1036 images of 11 dominant genera of Taihu Lake as data sources to compare and verify our algorithm. The precision of RAN-11 for a single prevailing genus was above 90%, with 5 species achieving 100% precision, except for Asterionella sp. The accuracy of RAN-11 was 95.67%, and the inference speed was 41.5 fps (frames per second), which is not only more accurate than Attention-92 (95.19% accuracy, 23.6 fps), but also faster than Attention-56 (94.71% accuracy, 41.2 fps), truly balancing accuracy and efficiency. Results indicate that:(1) RAN-11 is superior to the original residual attention network in terms of precision, accuracy and inference speed, as well as the traditional image recognition method represented by the Bags of Words model. (2) Fusion of multi-scale features, simplification of network structure and optimization of activation function are powerful means to improve the performance of convolutional neural network. Based on the classical classification, this paper proposes a new residual attention network to improve phytoplankton identification technology, and constructs an automatic phytoplankton recognition system, which has high recognition accuracy and easy promotion, and is of great significance to realize the automatic monitoring of phytoplankton in water.