Lysandros Nikolaou
01481f2dc1
gh-104169: Refactor tokenizer into lexer and wrappers ( #110684 )
...
* The lexer, which include the actual lexeme producing logic, goes into
the `lexer` directory.
* The wrappers, one wrapper per input mode (file, string, utf-8, and
readline), go into the `tokenizer` directory and include logic for
creating a lexer instance and managing the buffer for different modes.
---------
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2023-10-11 15:14:44 +00:00
..
2023-10-10 19:00:09 +09:00
2022-10-17 12:01:00 +02:00
2022-10-17 12:01:00 +02:00
2023-08-24 23:35:47 +00:00
2023-10-04 16:08:02 +01:00
2022-11-09 10:50:09 -08:00
2022-10-25 23:34:22 +01:00
2023-08-25 01:01:30 +02:00
2023-09-08 09:48:28 +00:00
2023-02-28 14:16:39 -07:00
2023-09-13 09:00:39 -07:00
2023-09-30 20:06:45 +00:00
2023-09-30 22:21:20 +00:00
2023-10-04 14:52:28 -07:00
2023-10-10 08:26:29 +09:00
2023-10-04 16:09:48 +01:00
2023-10-07 17:04:51 -07:00
2023-09-03 18:54:27 +02:00
2023-09-14 17:06:08 +01:00
2023-09-30 19:25:54 +02:00
2023-08-04 16:24:50 -07:00
2023-09-29 10:56:49 +02:00
2023-09-02 16:50:18 +02:00
2023-10-04 22:50:29 +00:00
2023-10-04 22:50:29 +00:00
2023-10-04 22:50:29 +00:00
2023-10-04 22:50:29 +00:00
2023-09-08 17:54:45 +01:00
2023-09-15 15:04:21 -07:00
2023-09-29 10:56:49 +02:00
2023-10-04 14:52:28 -07:00
2023-10-10 08:45:23 +00:00
2023-09-30 20:06:45 +00:00
2023-09-28 20:33:28 +01:00
2023-06-02 01:33:17 +02:00
2023-08-31 09:56:06 +02:00
2023-09-08 10:34:40 +01:00
2023-09-30 20:06:45 +00:00
2023-07-03 22:35:46 +00:00
2023-10-04 14:52:28 -07:00
2023-09-29 10:56:49 +02:00
2023-01-08 09:13:25 -06:00
2023-09-29 10:56:49 +02:00
2022-11-15 09:45:11 -07:00
2023-08-24 18:53:50 +02:00
2023-08-31 18:33:34 +02:00
2023-10-04 22:50:29 +00:00
2023-10-04 22:50:29 +00:00
2023-10-10 19:00:09 +09:00
2023-10-04 16:09:48 +01:00
2023-09-06 15:56:08 +02:00
2023-10-09 09:38:45 +01:00
2023-09-19 09:54:29 -06:00
2023-07-22 17:04:34 +02:00
2023-10-11 14:44:06 +03:00
2022-10-07 11:49:53 -07:00
2023-09-03 18:54:27 +02:00
2023-09-13 10:25:45 -07:00
2023-08-23 18:39:00 +01:00
2023-10-05 16:12:06 +01:00
2023-09-19 09:54:29 -06:00
2023-09-29 10:56:49 +02:00
2023-09-06 15:56:08 +02:00
2023-07-04 09:41:43 +00:00
2023-09-29 10:56:49 +02:00
2023-10-05 09:46:33 -06:00
2023-10-09 07:39:51 -06:00
2023-09-01 18:36:53 +02:00
2023-09-02 23:15:54 +02:00
2023-09-06 15:56:08 +02:00
2023-09-08 09:48:28 +00:00
2023-10-11 15:14:44 +00:00
2023-09-29 10:56:49 +02:00
2023-10-07 19:33:22 +02:00
2023-10-05 16:12:06 +01:00
2023-10-04 22:50:29 +00:00
2023-07-25 15:28:30 +02:00
2023-09-17 14:23:31 +03:00
2023-09-20 10:55:56 -06:00
2023-10-10 19:00:09 +09:00
2022-12-08 17:50:58 -07:00
2023-09-06 15:56:08 +02:00
2023-09-30 19:25:54 +02:00
2023-09-30 19:25:54 +02:00
2023-09-30 20:06:45 +00:00
2023-09-06 15:56:08 +02:00