It is an important national strategy for China to stabilize its grain production by controlling the “non-grain” cultivated land. To explore the spatial-temporal evolution and influencing factors of "non-grain" cultivated land in Guangxi will provide references for differentially controlling and guiding the cultivated land use in economically underdeveloped areas, like Southwest China. Taking Guangxi as an example, this paper used comprehensive evaluation method to evaluate the “non-grain” level of cultivated land use from 2003 to 2019, and adopted spatial autocorrelation model to analyze the spatial agglomeration effects of “non-grain” cultivated land, then used geographically weighted regression model to further explore the factors that affected the regional differences of “non-grain” cultivated land. The results showed that: (1) The “non-grain” level of cultivated land use in Guangxi showed a fluctuating upward trend with three stages from 2003 to 2019. (2) The “non-grain” phenomenon showed a spatial pattern described as "high in the southwest but low in the northwest" with significant positive spatial correlations. Chongzuo city in the southwest of Guangxi was of the worst situation. (3) Eight factors, such as topographic relief, annual precipitation, proportion of primary industry, land productivity and urbanization rate, were the main influencing factors for the spatial pattern of “non-grain” cultivated land in Guangxi. In summary, there are spatial differences between the degree of "non grain" cultivated land and its influencing factors. Thus, it is of great significance to develop differentiated control strategies to alleviate the “non-grain” use of cultivated land, so as to promote the effective management and protection of cultivated land.