Plug memory leak if listen-address only specifies the port.