In the present study, we investigated different chemical forms of soil heavy metals, soil chemical properties, and fungal diversity and composition for the contaminated soil collected from the tailings in Chenzhou Pb-Zn mining area, Hunan. The results showed that, compared with the Hunan local standard of heavy metal remediation, the polluted extent of the four kinds of heavy metals in soil samples was as follows: Zn > Pb > Cd > Cu. Compared with different chemical forms of the same heavy metal, it was found that the contents of Fe-Mn oxidation fraction of Pb, HOAc extractable fraction of Zn, HOAc extractable fraction of Cd and residual fraction of Cu were highest. Heavy metal pollution significantly altered the composition and structure of fungal community, rather than the richness and diversity. The relative abundances of sensitive fungi (e.g., Glomeromycota) were decreased with the increase of heavy metal contamination. However, the relative abundances of tolerant fungi (e.g., Ascomycota, Basidiomycota, Mortierellomycota, Mortierella, Fusarium, and Penicillium) were increased with the increase of heavy metal contamination, while they were decreased under severe pollution. RDA and VPA showed that heavy metals were the main factors affecting the fungal community.