In order to consider the nonlinear characteristics of rock and soil materials in slope stability analysis, a rigorous upper bound variational analysis method for slope stability based on the toe-below failure mechanism is proposed. Based on the upper bound theorem of limit analysis, a rotational failure mechanism of homogeneous slope was established. The sliding surface and its stress under limit state were obtained according to the Euler equation of variational principle, variational transversality conditions and boundary conditions. The implicit equation of slope height is established through the energy equilibrium principle, and the critical height of slope was optimized by particle swarm optimization algorithm. The slope factor of safety (FOS) is obtained. Comparisons with the results obtained by OPTUM G2 and FLAC 3D were conducted, and it was found that the potential sliding surface of the slope and the principal stress distribution on it obtained by the proposed method are in good agreement with the OPTUM G2 and FLAC 3D calculation results, and the error of the safety factor is small. Therefore, this variational analysis method could be used to evaluate the stability of homogeneous slopes. The results of theoretical analysis have certain theoretical research significance and engineering application value.