free_pattern_spec(): Don't try to free an invalid pointer