The Con Artist - A problem with the Ad Hominem fallacy

I argue that, sometimes,  it is logical to reject a persons argument based upon their character rather than the apparent strength or weakness of their argument.
Imagine that a known but very expert confidence trickster tells you about an amazing new investment. Imagine that he provides you with evidence that you can make money by this investment; with the testimony of other seemingly trustworthy people that you can make money with the investment and that he even gives you a chance to make only a small investment and you make a little money by it.
Now if you were simply to judge the confidence tricksters investment by all the evidence he has provided you with, you would think that making a larger investment would be perfectly safe.
Your only reason to doubt the confidence trickster would be because you know he's a confidence trickster but, because this is the ad hominem fallacy, "judging someones argument by their character rather than the strength of their argument" you would reject it.
However you would be very wrong to reject it because there are certain people, for example confidence trickers, who are in a position to counterfeit evidence to support their claims and thereby create the illusion that their argument is valid.
In a situation where a person can provide fake evidence which appears to verify an argument, I argue that it makes good sense to be suspicious of their argument merely because you know they are a con artist.
Where is there an example of this in everyday life?  The book selling conspiracy theorist.
Conspiracy theorists  are well known for supporting their arguments with a lot of fake documents and unreliable sources. Alot of people take the claims of conspiracy theorists with a pinch of salt simply because they are conspiracy theorists; in other words they practice the adhominem fallacy but if they did not then they would end up being conned by a lot of very official and reliable looking but fake evidence.