Fix a tokenize() regression that caused it to miss tokens with trailing comments.