I haven’t tried that either, and in theory it might work. However if people are still members of teams then they’ll still see things come up in their activity feeds, notification emails, etc.