ladybird/Libraries/LibJS
Timothy Flynn 00182a2405 LibJS: Port the JS lexer and parser to UTF-16
This ports the lexer to UTF-16 and deals with the immediate fallout up
to the AST. The AST will be dealt with in upcoming commits.

The lexer will still accept UTF-8 strings as input, and will transcode
them to UTF-16 for lexing. This doesn't actually incur a new allocation,
as we were already converting the input StringView to a ByteString for
each lexer.

One immediate logical benefit here is that we do not need to know off-
hand how many UTF-8 bytes some special code points occupy. They all
happen to be a single UTF-16 code unit. So instead of advancing the
lexer by 3 positions in some cases, we can just always advance by 1.
2025-08-13 09:56:13 -04:00
..
Bytecode LibJS+LibWeb+WebContent: Port JS::PropertyKey to UTF-16 2025-08-05 07:07:15 -04:00
Contrib/Test262 LibJS+LibWeb+WebContent: Port JS::PropertyKey to UTF-16 2025-08-05 07:07:15 -04:00
Heap LibJS: Enable EXPLICIT_SYMBOL_EXPORT and annotate minimum symbol set 2025-07-22 11:51:29 -04:00
Runtime LibJS: Port the JS lexer and parser to UTF-16 2025-08-13 09:56:13 -04:00
Tests LibJS: Handle out-of-range prefixed numbers in Token::double_value 2025-08-10 13:35:37 +02:00
AST.cpp LibJS+LibWeb+WebContent: Port JS::PropertyKey to UTF-16 2025-08-05 07:07:15 -04:00
AST.h LibJS+LibWeb+WebContent: Port JS::PropertyKey to UTF-16 2025-08-05 07:07:15 -04:00
CMakeLists.txt LibJS: Remove MarkupGenerator 2025-08-07 02:05:50 +02:00
Console.cpp LibJS+LibWeb+WebContent: Port JS::PropertyKey to UTF-16 2025-08-05 07:07:15 -04:00
Console.h LibJS: Enable EXPLICIT_SYMBOL_EXPORT and annotate minimum symbol set 2025-07-22 11:51:29 -04:00
CyclicModule.cpp LibJS: Update spec steps / links for the import-assertions proposal 2025-04-29 07:33:08 -04:00
CyclicModule.h LibJS: Enable EXPLICIT_SYMBOL_EXPORT and annotate minimum symbol set 2025-07-22 11:51:29 -04:00
Forward.h LibJS: Revert Enable EXPLICIT_SYMBOL_EXPORT 2025-07-22 11:51:29 -04:00
Lexer.cpp LibJS: Port the JS lexer and parser to UTF-16 2025-08-13 09:56:13 -04:00
Lexer.h LibJS: Port the JS lexer and parser to UTF-16 2025-08-13 09:56:13 -04:00
LocalVariable.h LibJS: Preserve information about local variables declaration kind 2025-05-06 12:06:23 +02:00
Module.cpp LibJS+LibWeb+WebContent: Port JS::PropertyKey to UTF-16 2025-08-05 07:07:15 -04:00
Module.h LibJS+LibWeb+WebContent: Port JS::PropertyKey to UTF-16 2025-08-05 07:07:15 -04:00
ModuleLoading.h LibGC+Everywhere: Factor out a LibGC from LibJS 2024-11-15 14:49:20 +01:00
Parser.cpp LibJS: Port the JS lexer and parser to UTF-16 2025-08-13 09:56:13 -04:00
Parser.h LibJS: Port the JS lexer and parser to UTF-16 2025-08-13 09:56:13 -04:00
ParserError.cpp LibJS: Port the JS lexer and parser to UTF-16 2025-08-13 09:56:13 -04:00
ParserError.h LibJS: Port the JS lexer and parser to UTF-16 2025-08-13 09:56:13 -04:00
Position.h LibJS: Cache source code positions more often 2024-12-02 11:42:11 +01:00
Print.cpp LibJS+LibWeb: Replace JS::Utf16String with AK::Utf16String 2025-07-18 12:45:38 -04:00
Print.h LibJS: Enable EXPLICIT_SYMBOL_EXPORT and annotate minimum symbol set 2025-07-22 11:51:29 -04:00
Script.cpp LibGC+Everywhere: Factor out a LibGC from LibJS 2024-11-15 14:49:20 +01:00
Script.h LibJS: Enable EXPLICIT_SYMBOL_EXPORT and annotate minimum symbol set 2025-07-22 11:51:29 -04:00
SourceCode.cpp LibJS: Port the JS lexer and parser to UTF-16 2025-08-13 09:56:13 -04:00
SourceCode.h LibJS: Port the JS lexer and parser to UTF-16 2025-08-13 09:56:13 -04:00
SourceRange.h LibJS: Enable EXPLICIT_SYMBOL_EXPORT and annotate minimum symbol set 2025-07-22 11:51:29 -04:00
SourceTextModule.cpp LibJS+LibWeb+WebContent: Port JS::PropertyKey to UTF-16 2025-08-05 07:07:15 -04:00
SourceTextModule.h LibJS+LibWeb+WebContent: Port JS::PropertyKey to UTF-16 2025-08-05 07:07:15 -04:00
SyntaxHighlighter.cpp LibJS: Port the JS lexer and parser to UTF-16 2025-08-13 09:56:13 -04:00
SyntaxHighlighter.h LibJS: Enable EXPLICIT_SYMBOL_EXPORT and annotate minimum symbol set 2025-07-22 11:51:29 -04:00
SyntheticModule.cpp LibJS+LibWeb+WebContent: Port JS::PropertyKey to UTF-16 2025-08-05 07:07:15 -04:00
SyntheticModule.h LibJS+LibWeb+WebContent: Port JS::PropertyKey to UTF-16 2025-08-05 07:07:15 -04:00
Token.cpp LibJS: Port the JS lexer and parser to UTF-16 2025-08-13 09:56:13 -04:00
Token.h LibJS: Port the JS lexer and parser to UTF-16 2025-08-13 09:56:13 -04:00