I know why you don't talk. You're angry. You're angry because they make you wear a dress.
Algren has a silent Samurai following him everywhere he goes. He tries to get the Samurai to speak and has no luck. He then jokingly states that he knows why the Samurai is silent and it is because the Samurai is angry they make him wear a dress.