If the issue is people not wanting to show their good or bad scores then make it optional to show.
If the issue is people cheating to get 5 stars/hearts/dragon eggs then limit how they can get the votes.
but as with everything, nothing is in black and white. I think that maybe you can make...