Lets analyze your question in great detail.
It is a 'living hell' for a man to live with his wife whilst not being able to remove his sexual urges. Men have different sexual nature compared to woman, and thus they have orgasms more frequently than woman. Generally speaking: woman want romance but men want to have sex, but for a man his romance lies in sex (of course generally speaking).
Now if you refuse him, then his sexual urges have,not been fulfilled and thus his right has been refused. What do you think he will do to release his urges? The following are some possibilities:
- Possibility of zina
- Ill-thoughts and feeling of worthless (e.g. he is not good enough for you)
All of these activities are haram, and will, directly or indirectly, harm your relationship, and you both might end up in a tough spot on Day of Judgement (I pray to Allah (swt) he grants you both paradise). You might complain later that your husband doesn't give time to you. You want to give him his right to protect himself from getting lewd thoughts which might lead to worse of sins, otherwise you might regret it. For one of these reasons, in Islam the woman is a fortress against the devil.
I do,not say that man his weak but Allah says:
Allah wishes to lighten (the burden) for you; and man was created weak (cannot be patient to leave sexual intercourse with woman). [4:28]
And doing sexual intercourse with you his sadaqa on your and his part so much so that, Prophet Muhammad (pbuh) said:
Abu Dharr reported: some of the people from among the Companions of the Apostle of Allah (may peace be upon him) said to him: Messenger of Allah, the rich have taken away (air the) reward. They observe prayer as we do; they keep the fasts as we keep, and tray give Sadaqa out of their surplus riches. Upon this he (the Holy Prophet) said: Has Allah not prescribed for you (a course) by following which you can (also) do sadaqa? In every declaration of the glorification of Allah (i. e. saying Subhan Allah) there is a Sadaqa, and every Takbir (i. e. saying Allah-O-Akbar) is a sadaqa, and every praise of His (saying al-Hamdu Lillah) is a Sadaqa and every declaration that He is One (La illha ill-Allah) is a sadaqa, and enjoining of good is a sadaqa, and forbidding of that which is evil is a Sadaqa, and in man's sexual Intercourse (with his wife, ) there is a Sadaqa. They (the Companions) said: Messenger of Allah, is there reward for him who satisfies his sexual passion among us? He said: Tell me, if he were to devote it to something forbidden, would it not be a sin on his part? Similarly, if he were to devote it to something lawful, he should have a reward. [Book 5 Hadith 2198]
And lets not forget even our Prophet Muhammad (pbuh) was a man as well:
Jabir reported that Allah's Messenger (may peace be upon him) saw a woman, and so he came to his wife, Zainab, as she was tanning a leather and had sexual intercourse with her. He then went to his Companions and told them: The woman advances and retires in the shape of a devil, so when one of you sees a woman, he should come to his wife, for that will repel what he feels in his heart.[Book 8 Hadith 3240]
The following Hadith has some degree of truth in it:
Narrated Abu Huraira: Allah's Apostle said, "If a husband calls his wife to his bed (i.e. to have sexual relation) and she refuses and causes him to sleep in anger, the angels will curse her till morning." (Book 54, Hadith 460)
The above mentioned Hadith explanation: man wants to 'release' is urge and his wife refuses him. This implies that husband thoughts might be that he can't fulfil his desires in a halal way, and it is difficult to control his urges especially if he sleeps with his wife throughout the night.
Now how does this impact the relationship? One man came to a sheikh and told him about problems with his relationship with his spouse. The sheikh asked him off the bat about his sexual relationship, and he started crying.
My sincere advice would be to give him his right, regardless of how tired you are. Like it was said, you will even get reward for it.
PS: vice versa also applies if you request for sex.
May Allah guide you, and help your husband.