Smart people new that something was wrong with the trading system of Bernie Madoff.
Smart people thought that he was front running trades. Front running trades is illegal. Some people think that front running is a victimless crime. The smart people did not want to dig or investigate very deeply into the Madoff operations because they were afraid of exposing the front running activities and they were profiting from what they thought was easy money.
The problem was that the smart people thought that they were profiting from the illegal front running trades and not a Ponzi scheme.