listen_loop(): When shutting down gracefully, close listening ports