Well, that'd be fine if they wouldn't then also ignore the wall on their loop around and harass T1 directly.
Extra micro on a map that already has you juggle strong camps, several timed ground effects and constant attacks that ramp up to artillery and whisperers isn't really a passable option.
On the flipside, the strategic option of letting the Vilebloods pass through for the loop wouldn't be gone at all because they never auto-aggro. Only if attacked. It's the map-to-map inconsistent behavior that's annoying. The default and more common occurrence seems to be that they are supposed to walk in range to be attacked. But sometimes they simply don't and you don't know if you hit that odd behavior until they pass.