Land use optimization usually considers the various requirements of different groups, and is a complex many-objective (more than 3 objectives) problem in theory. However, it is usually simplified as a multi-objective (2-3 objectives) problem in practice and solved using a popular multi-objective optimization algorithm, nondominated sorting genetic algorithm-II (NSGA-II). The reasons behind this fact include the lack of cognition of many-objective optimization algorithms and the lack of effectiveness comparison between many- and multi-objective optimization algorithms. This paper explored NSGA-II, which is one of the most widespread multi-objective optimizations, and a many-objective optimization algorithm, namely NSGA-III, which is the latest version of NSGA series. We made an effectiveness comparison between NSGA-III and NSGA-II theoretically and experimentally, to explore the advantages and disadvantages of these two algorithms in land-use optimization. In theory, the principles of the two algorithms were compared. The experimental comparison includes two experiments, a three-objective land use optimization and a thirteen-objective land use optimization taking Lhasa as the research area. After the experiments, a four-layer framework with six indicators was used to evaluate algorithms comprehensively. The theoretical comparison results showed that the only difference between the two algorithms lied in the determination of population diversity. Specifically, NSGA-III employed the distances between solutions and reference points, while NSGA-II utilized the distances between adjacent solutions. The determination of population diversity in NSGA-III was easier to achieve global diversity and avoided the situation of local diversity but global compactness. The determination of population diversity in NSGA-II was greatly affected by the dimension. When the dimension increased, the calculation was cumbersome, time-consuming, and slows down the search process. The experimental comparison showed that the two algorithms had their own advantages in different indicators. Compared with NSGA-II, in multi-objective optimization, NSGA-III had advantages in terms of the quality of the results and the degree of optimization, and with the increase of the objective function, NSGA-III will occupy less computational time than NSGA-II, and the effect of population diversity protection was also improving. According to the comparison of the optimal individuals obtained from two algorithms, the optimal individuals generated by NSGA-III were higher than NSGA-II in terms of practical value. Therefore, the NSGA-III algorithm had great potential in the field of land use optimization and could provide more valuable references for planners. The results of this paper can assist empirical research and provide a reference for designing a more comprehensive and more realistic land use optimization model.