Wa aleykum salam ar rahmatullah wa barakatuh
First, let me remind you that hygiene and purification are extremely important in Islam. Prophet Muhammad (peace be upon him) said: "cleanliness is half of faith"; and Allah tells us in the Quran:
"Truly, Allah loves those who turn
unto Him in repentance and loves those
who purify themselves." [Quran, 2:222]
Purity through ritual washing (wudu or ghusl) is the key to perform salat; but it is also compulsory to take care of the body Allah has given you. You have admitted that you "don't soap in such places often", though you really have to make efforts to clean yourself in a correct way, in order to remove dirt from your ENTIRE body.
However, don't worry, these patches are not made of dirt. Their cause is a huge amount of melanocytes in specific areas (especially under feet or near joints), this is to say a hyperpigmentation appearing on skin, when it is too dry or thick. These cells are inside epidermis, so water touches your skin anyway; your ghusl is valid, don't try to scratch them until you bleed.
I hope this answers the question.