In ecology, the spatial point pattern, which is obtained by mapping the locations of each individual as points in space, is a very important tool for describing the spatial distribution of species. There are three generally accepted types of spatial point patterns:regular, random, and aggregated. To detect spatial patterns, quadrat sampling is commonly applied, where quadrats are randomly thrown on the space and then the number of individuals in quadrats is used to fit Poisson model or NBD model, respectively. Distance sampling is an alternative method for spatial point pattern analysis, which is flexible and efficient, especially in highly dense plant communities, and in difficult terrain. Nearest neighbor method is one effective distance sampling method in spatial distribution pattern analysis. There are two kinds of nearest neighbor distances (NND):point-to-tree NND, distances from randomly selected points (sampling points) to the nearest individuals; and tree-to-tree NND, distances from selected individuals to their nearest neighbors. In this paper, we show a probability distribution model of higher order nearest neighbor distance (tree-to-tree). As we see the expression of this model is complicated; therefore, parameter estimation using conventional method is not a trivial task. In statistics, there are many numerical methods for estimating the parameters of complicated probability distribution model such as moment method, empirical method, graphical method, and maximum likelihood method. In previous literature, maximum likelihood method has been applied for parameter estimation and the optimized estimates on the log-likelihood surface were searched by Nelder-Mead algorithm. However, maximum likelihood estimation was fraught with nontrivial numerical issues when the samples of tree-to-tree distance were rare. In this paper, we use an alternative method, genetic algorithm, to estimate the two model parameters. The computation can be further simplified by defining a suitable objective function based on the expectation and variance. The probability distribution model is then used to fit spatial distribution data of three tree species on southern Vancouver Island, western coast of Canada. It is found that the proposed probability distribution model can fit nearest neighbor distance samples well for Douglas-fir (Pseudotsuga menziesii) and western hemlock (Tsuga heterophylla). For tree species western red cedar (Thuja plicata), the fitting is not so satisfied because individuals of western red cedar are usually distributed as small clusters. As Douglas-fir is almost randomly distributed in space, the estimated parameter representing spatial aggregation nearly does not change. However, the estimated parameter increases when spatial scale increases for the other two tree species, western hemlock and western red cedar. A short discussion about the advantages and limitations of the probability model and its parameter estimation methods is also presented. Theoretically, the probability distribution model presented in this study is applicable to all kinds of spatial point patterns ranging from highly aggregated to complete random. However, as the actual point patterns of tree species usually deviate from theoretical assumptions, the probability distribution model has a few shortcomings such as scale dependence. To gain a better fitting, higher orders of nearest neighbor distances are needed. A balance between field work burden and performance of model fitting should be considered. We suggest that ideal orders of nearest neighbor distances are from 2 to 6. Another potential that can improve the fitting performance is using mixed probability distributions.