It's not the speaking - it's the payment.
Yes, but it's broader than that. Consider these thought experiments that are also unethical:
1) Trump builds a border wall. After his presidency he gets paid to give speeches to the trade association Border Walls 'R Us, whose members include companies that profited from the border wall.
2) Alternate universe where Bernie Sanders won the presidency and passes a law that makes unions powerful. After his presidency, he gets paid by the trade unions to give speeches.
Also consider the slippery slope that Obama getting money creates. Even if it wasn't a pay-for-play in Obama's case, it's an incentive for other political figures to do pay-for-play and allows industry figures to induce them to do so. For example, the firm that is paying Obama for a speech could go to Trump and intimate that he could get the same treatment if he treats them favorably.
trump would laugh at them if all they offer is $400,000. he is milking the taxpayers for millions now