In order to ascertain the impact of pyrite mining on the surrounding farmland soil environment and human health, 42 surface soil samples (from 0-20 cm) were collected around the pyrite mining area in Longyou county. In this study, the concentrations of eight heavy metals (As, Cd, Cr, Cu, Hg, Ni, Pb, and Zn) and the pH in the topsoil were analyzed, and the concentration characteristics of heavy metals, source analysis, and human health risks assessment were studied using statistical analysis (SA), geo-accumulation index (I_geo), positive matrix factorization (PMF), and the health risk model. The average of ω(Cd), ω(Cu), ω(Pb), and ω(Zn) concentrations exceeded the background values of soils in Zhejiang province and China. According to the agricultural land pollution risk screening values (GB 15618-2018), Cd, Cu, Pb, and Zn were up to 82%, 49%, 42%, and 31%, respectively. The I_geo shows that the major pollutant element in the soils was Cd, followed by Cu, Pb, and Zn. The PMF analysis indicates that nature sources (As, Cr, and Ni), comprehensive pollution sources caused by high geological background and mining of ore-forming geological bodies (Cd, Cu, Pb, and Zn), and anthropogenic sources (Hg) were the three major sources of heavy metals in the study area, with contributions of 32%, 46%, and 22%, respectively. The results of the health risk assessment indicate that the major non-carcinogenic factor triggering risks was the ingestion of Pb; Cr exposure had carcinogenic risk for adults, and Cr and As exposure had carcinogenic risk for children.