Physics like this gets complicated very quickly, so I'll try to keep it simple (and in the realms of my own understanding).
I would start by noting these objects are not an isolated system, they exist in the universe and are therefore moving relative to many other objects as well as each other. That is essential. Notably, in your example, you propose the objects would be moving away from each other. Well, us, an independent observer, could actually quite easily deduce space is not expanding in that scenario: the objects would need to be moving away from us, too. Alternatively, if the objects were moving away from us and each other, we could not deduce whether space is expanding or not. That is somewhat in the ballpark of special relativity. I digress, however, the objects are not in an isolated system!
Firstly, it's also essential to note two fundamental characteristics of the universe: the Cosmological Principle and the Copernican Principle. The Cosmological Principle states the universe is both isotropic (looks the same in all directions) and homogenous (the same everywhere) when viewed at a large enough scale. The Copernican Principle simply states that we (humans) are not the center of the universe; we are not special.
Alright, so, how do we know the universe is expanding? There are many theoretical predictions in agreement here, with empirical observations of expected consequences of the expansion backing them. (Why observations of consequences? Because you simply can't directly measure space itself expanding, we can only observe the effects of the expansion). I won't go into the theoretical predictions as those are mathematically beyond me since they originate in general relativity. The important takeaway is these theories predicted the metric expansion of the universe and therefore, for the two principles I stated earlier to be satisfied, space itself must be expanding.
Empirical observations, then... Most famously, Hubble demonstrated all distant* galaxies are moving away from us and that their speed is proportional to their distance from us. As you may know, this observation gave rise to Hubble's Law. Other observations include that as we look into the past (i.e. the farther out we look, as light takes time to reach us) we can see galaxies were clustered closer together than they are today and the Cosmic Microwave Background was at a higher temperature (so a shorter wavelength - it didn't use to be microwave radiation) than it is today.
*I'll note, to avoid any confusion, that we say "distant" galaxies as our local galaxies' motion is dominated by gravitational dynamics on the scale of the Local Group. This makes sense, as on smaller scales the universe appears to be 'lumpy' with clusters of galaxies, but as you increase said scale the universe smooths out into a uniform distribution.
There's a huge amount to cover and I'm really quite tired, so I've tried to stick to the basics and what I know off the top of my head.