No, Stig worries about the wear of the soft metals which change the part's shape and eventually impairs the function of your device, leading to some troubles (at least).
Your idea looks fine to me. Just find a hard enough brass, I know there are many different brass alloys.