The Pisha Sandstone has caused serious soil erosion in the Inner Mongolia area. It is the main source of coarse silt in the middle and lower reaches of the Yellow River. In order to reveal the formation mechanism of soil erosion in the Pisha Sandstone area and the research and developing methods to control soil erosion in the Pisha Sandstone area, it is necessary to clear the root cause of the collapse of Pisha Sandstone. This paper analyzes the structural composition and mineral composition of the Pisha Sandstone by means of quantitative XRD and optical microscope analysis. The results show: (1) This paper found the soft rock is a kind of mudstone with severe weathering and loose structure. The structure of Pisha Sandstone is composed of granular skeleton material and clay cementation material, and its cementation is contact cementation. (2) The granular skeleton material is mainly composed of quartz, calcium carbonate and the structure of granular skeleton. Montmorillonite, chlorite, kaolinite and iron oxide are cementation materials. (3) Pisa Sandstone is a solid mixture. Granular skeleton crystals (quartz and feldspar) are linked together by the cementing force of the hydrophilic and weak cemented clay, and the friction between crystals, mechanical bite force and particle support. Due to incomplete filling of cementing materials, there are many pores between particles and particulates. Its structure is composed of granular support and osmotic structure.