Wis. governor says protests haven't swayed him