I mean my vision of this system is obviously always going to be an ideal version of what would happen. That said...
I do think that the rulers would necessarily have to have the same kind of ideas about everything, so I think that the specific vision of a ruler would always be similar in some aspects, but not nearly all, at least not necessarily. Even if the person is vetted as showing similar sort of opinions regarding key topics, the actions could be different. A way to fix this problem would be to have the same sort of a re-election every for example three emperors. This would mean that a system would really be given a chance to rule, and after more than likely 100+ years there could be another decision made to judge fairly how the leaders did.
I agree that the process of removing someone would be difficult in this system. Again my proposition stated before could in the long term fix this problem, but it would be a problem in the short term. I don't really have a good fix for this, at least a nonviolent. Of course the public has always the option of revolting if the ruler is trurly unbearable, but it wouldn't be a nonviolent way.
The Roman empire is also a great example of how the system can be great, as shown by the five good emperors, who did exactly as I suggest, selected their candidates, before Marcus Aurelius' fatal mistake.