Miss Islington (bot) ffafa9b91d
gh-96268: Fix loading invalid UTF-8 (GH-96270)
This makes tokenizer.c:valid_utf8 match stringlib/codecs.h:decode_utf8.

It also fixes an off-by-one error introduced in 3.10 for the line number when the tokenizer reports bad UTF8.
(cherry picked from commit 8bc356a7dd50cbdb46d10b8c7e457832431f5d9e)

Co-authored-by: Michael Droettboom <mdboom@gmail.com>
2022-09-07 14:49:17 -07:00
..