Visit Visa refusals (or even approvals) don't have any bearing in getting a spouse visa. The only problem it will cause the applicant is if the refusal was because of FRAUD documents or deception and such. If the Home Office found any fraudulent documents or deception to gain an approved visa, then I believe it's possible for them to give the applicant a 10-year ban to future visa applications. Otherwise, a simple refusal (i.e. the Home Office doesn't believe the applicant have strong ties to his/her home country for him/her to return) won't affect the spouse visa application.

I found this sample online of a refusal with 10-year ban.