Why didn't you invade then?
I'll make it easy for you
Germany had no military invasion fleet, being forced to use wooden river barges, which were collected in rivers of northern France, where they were bombed repeatedly by the RAF.
For the seaborne invasion to have crossed the English Channel, Germany needed to sink the Royal Navy, a feat it never looked liked being able to accomplish.
To have crossed the Channel and maintained logistics, Germany also needed to defeat the RAF, so after Germany had lost the Battle of Britain by 11 October, 1940, such an invasion was doomed to failure.
TL;DR - WE FAKKED U AND U HAD TO FLEE TO TRY AND BEAT USSR, WHICH U COULDNT, LUL