Stowage/ladybird
2
0
Fork 0
mirror of https://github.com/LadybirdBrowser/ladybird.git synced 2025-10-24 01:53:20 +00:00
Code Issues Projects Releases Packages Wiki Activity
ladybird/Userland/Libraries/LibWeb/CSS/Parser/Token.h

167 lines
3.4 KiB
C
Raw Normal View History

LibWeb: Add specification-based CSS tokenizer Original work by @stelar7 for #2628.
2021-03-09 17:18:08 +01:00
/*
Everywhere: Use "the SerenityOS developers." in copyright headers We had some inconsistencies before: - Sometimes "The", sometimes "the" - Sometimes trailing ".", sometimes no trailing "." I picked the most common one (lowecase "the", trailing ".") and applied it to all copyright headers. By using the exact same string everywhere we can ensure nothing gets missed during a global search (and replace), and that these inconsistencies are not spread any further (as copyright headers are commonly copied to new files).
2021-04-28 22:46:44 +02:00
* Copyright (c) 2020-2021, the SerenityOS developers.
LibWeb: Use CSS::Number for Token numeric values
2022-03-21 21:01:27 +00:00
* Copyright (c) 2021-2022, Sam Atkins <atkinssj@serenityos.org>
LibWeb: Add specification-based CSS tokenizer Original work by @stelar7 for #2628.
2021-03-09 17:18:08 +01:00
*
Everything: Move to SPDX license identifiers in all files. SPDX License Identifiers are a more compact / standardized way of representing file license information. See: https://spdx.dev/resources/use/#identifiers This was done with the `ambr` search and replace tool. ambr --no-parent-ignore --key-from-file --rep-from-file key.txt rep.txt *
2021-04-22 01:24:48 -07:00
* SPDX-License-Identifier: BSD-2-Clause
LibWeb: Add specification-based CSS tokenizer Original work by @stelar7 for #2628.
2021-03-09 17:18:08 +01:00
*/
#pragma once
LibWeb: Convert CSS Token::m_unit from StringBuilder to FlyString This value doesn't change once it's assigned to the Token, so it can be more lightweight than a StringBuilder.
2021-10-28 12:01:15 +01:00
#include <AK/FlyString.h>
LibWeb: Add specification-based CSS tokenizer Original work by @stelar7 for #2628.
2021-03-09 17:18:08 +01:00
#include <AK/String.h>
LibWeb: Use a u32 for a delim tokens value The spec says: > <delim-token> has a value composed of a single code point. So using StringView is a bit overkill. This also allows us to use switch statements in the future.
2022-03-13 17:34:08 +01:00
#include <AK/Utf8View.h>
LibWeb: Use CSS::Number for Token numeric values
2022-03-21 21:01:27 +00:00
#include <LibWeb/CSS/Number.h>
LibWeb: Add specification-based CSS tokenizer Original work by @stelar7 for #2628.
2021-03-09 17:18:08 +01:00
namespace Web::CSS {
class Token {
friend class Tokenizer;
public:
LibWeb: Rename CSS::Token::TokenType -> Type
2021-07-09 20:54:06 +01:00
enum class Type {
LibWeb: Add specification-based CSS tokenizer Original work by @stelar7 for #2628.
2021-03-09 17:18:08 +01:00
Invalid,
EndOfFile,
Ident,
Function,
AtKeyword,
Hash,
String,
BadString,
Url,
BadUrl,
Delim,
Number,
Percentage,
Dimension,
Whitespace,
CDO,
CDC,
Colon,
Semicolon,
Comma,
OpenSquare,
CloseSquare,
OpenParen,
CloseParen,
OpenCurly,
CloseCurly
};
enum class HashType {
Id,
Unrestricted,
};
LibWeb: Record position information in CSS Tokens This is a requirement to be able to use the Tokens for syntax highlighting.
2021-10-21 21:25:14 +01:00
struct Position {
size_t line { 0 };
size_t column { 0 };
};
Type type() const { return m_type; }
LibWeb: Rename CSS::Token::TokenType -> Type
2021-07-09 20:54:06 +01:00
bool is(Type type) const { return m_type == type; }
LibWeb: Add convenience methods to CSS::Parser::Token Some of these will be removed later, when we move to using is() exclusively.
2021-07-01 14:52:12 +01:00
StringView ident() const
{
LibWeb: Give CSS Token and StyleComponentValueRule matching is() funcs The end goal here is to make the two classes mostly interchangeable, as the CSS spec requires that the various parser algorithms can take a stream of either class, and we want to have that functionality without needing to duplicate all of the code.
2021-07-03 15:06:53 +01:00
VERIFY(m_type == Type::Ident);
LibWeb: Convert CSS Token::m_value from StringBuilder to FlyString Again, this value does not change once we have finished creating the Token, so it can be more lightweight.
2021-11-18 12:30:46 +00:00
return m_value.view();
LibWeb: Add convenience methods to CSS::Parser::Token Some of these will be removed later, when we move to using is() exclusively.
2021-07-01 14:52:12 +01:00
}
LibWeb: Break friendship between CSS Token and Parser :^( The Parser no longer needs to mess with Token's internals, since we have getter functions that are safer.
2021-11-18 12:04:05 +00:00
StringView function() const
{
VERIFY(m_type == Type::Function);
LibWeb: Convert CSS Token::m_value from StringBuilder to FlyString Again, this value does not change once we have finished creating the Token, so it can be more lightweight.
2021-11-18 12:30:46 +00:00
return m_value.view();
LibWeb: Break friendship between CSS Token and Parser :^( The Parser no longer needs to mess with Token's internals, since we have getter functions that are safer.
2021-11-18 12:04:05 +00:00
}
LibWeb: Use a u32 for a delim tokens value The spec says: > <delim-token> has a value composed of a single code point. So using StringView is a bit overkill. This also allows us to use switch statements in the future.
2022-03-13 17:34:08 +01:00
u32 delim() const
LibWeb: Add convenience methods to CSS::Parser::Token Some of these will be removed later, when we move to using is() exclusively.
2021-07-01 14:52:12 +01:00
{
LibWeb: Give CSS Token and StyleComponentValueRule matching is() funcs The end goal here is to make the two classes mostly interchangeable, as the CSS spec requires that the various parser algorithms can take a stream of either class, and we want to have that functionality without needing to duplicate all of the code.
2021-07-03 15:06:53 +01:00
VERIFY(m_type == Type::Delim);
LibWeb: Use a u32 for a delim tokens value The spec says: > <delim-token> has a value composed of a single code point. So using StringView is a bit overkill. This also allows us to use switch statements in the future.
2022-03-13 17:34:08 +01:00
return *Utf8View(m_value.view()).begin();
LibWeb: Add convenience methods to CSS::Parser::Token Some of these will be removed later, when we move to using is() exclusively.
2021-07-01 14:52:12 +01:00
}
StringView string() const
{
LibWeb: Give CSS Token and StyleComponentValueRule matching is() funcs The end goal here is to make the two classes mostly interchangeable, as the CSS spec requires that the various parser algorithms can take a stream of either class, and we want to have that functionality without needing to duplicate all of the code.
2021-07-03 15:06:53 +01:00
VERIFY(m_type == Type::String);
LibWeb: Convert CSS Token::m_value from StringBuilder to FlyString Again, this value does not change once we have finished creating the Token, so it can be more lightweight.
2021-11-18 12:30:46 +00:00
return m_value.view();
LibWeb: Add convenience methods to CSS::Parser::Token Some of these will be removed later, when we move to using is() exclusively.
2021-07-01 14:52:12 +01:00
}
LibWeb: Add specification-based CSS tokenizer Original work by @stelar7 for #2628.
2021-03-09 17:18:08 +01:00
LibWeb: Implement ImageStyleValue parsing Later we will want to make a distinction between URL and Image values, but this works for now.
2021-07-22 13:00:29 +01:00
StringView url() const
{
VERIFY(m_type == Type::Url);
LibWeb: Convert CSS Token::m_value from StringBuilder to FlyString Again, this value does not change once we have finished creating the Token, so it can be more lightweight.
2021-11-18 12:30:46 +00:00
return m_value.view();
LibWeb: Implement ImageStyleValue parsing Later we will want to make a distinction between URL and Image values, but this works for now.
2021-07-22 13:00:29 +01:00
}
LibWeb: Get CSS @import rules working in new parser Also added css-import.html, which tests the 3 syntax variations on `@import` statements. Note that the optional media-query parameter to `@import` is not handled yet.
2021-07-22 17:51:07 +01:00
StringView at_keyword() const
{
VERIFY(m_type == Type::AtKeyword);
LibWeb: Convert CSS Token::m_value from StringBuilder to FlyString Again, this value does not change once we have finished creating the Token, so it can be more lightweight.
2021-11-18 12:30:46 +00:00
return m_value.view();
LibWeb: Get CSS @import rules working in new parser Also added css-import.html, which tests the 3 syntax variations on `@import` statements. Note that the optional media-query parameter to `@import` is not handled yet.
2021-07-22 17:51:07 +01:00
}
LibWeb: Parse CSS selectors according to the spec The spec does not directly tell us how to parse selectors, so there are likely some bugs here, but I've used the spec language where possible. This is very much based on the previous selector parsing code. Any parse error inside a selector makes the entire SelectorList invalid, so nothing is returned.
2021-07-23 16:13:07 +01:00
HashType hash_type() const
{
VERIFY(m_type == Type::Hash);
return m_hash_type;
}
StringView hash_value() const
{
VERIFY(m_type == Type::Hash);
LibWeb: Convert CSS Token::m_value from StringBuilder to FlyString Again, this value does not change once we have finished creating the Token, so it can be more lightweight.
2021-11-18 12:30:46 +00:00
return m_value.view();
LibWeb: Parse CSS selectors according to the spec The spec does not directly tell us how to parse selectors, so there are likely some bugs here, but I've used the spec language where possible. This is very much based on the previous selector parsing code. Any parse error inside a selector makes the entire SelectorList invalid, so nothing is returned.
2021-07-23 16:13:07 +01:00
}
LibWeb: Use CSS::Number for Token numeric values
2022-03-21 21:01:27 +00:00
Number const& number() const
LibWeb: Make CSS 'An+B' parsing spec-compliant Parsing this pattern from CSS tokens turns out to be slightly crazy, but thankfully well documented in the spec. The spec lists the cases in order of simple -> complicated, but this would cause problems in code, since `<n-dimension> <signed-.integer>` would never by reached, as `<n-dimension>` comes before. Instead, I have grouped them by their first token. Also renamed the NthChildPattern class to ANPlusBPattern, to match spec terminology.
2021-07-24 21:22:44 +01:00
{
LibWeb: Use CSS::Number for Token numeric values
2022-03-21 21:01:27 +00:00
VERIFY(m_type == Type::Number || m_type == Type::Dimension || m_type == Type::Percentage);
return m_number_value;
LibWeb: Make CSS 'An+B' parsing spec-compliant Parsing this pattern from CSS tokens turns out to be slightly crazy, but thankfully well documented in the spec. The spec lists the cases in order of simple -> complicated, but this would cause problems in code, since `<n-dimension> <signed-.integer>` would never by reached, as `<n-dimension>` comes before. Instead, I have grouped them by their first token. Also renamed the NthChildPattern class to ANPlusBPattern, to match spec terminology.
2021-07-24 21:22:44 +01:00
}
LibWeb: Use floats instead of doubles for CSS numbers Using doubles isn't necessary, and they make things slightly bigger and slower, so let's use floats instead.
2022-03-21 17:22:05 +00:00
float number_value() const
LibWeb: Return numeric values from Token value getters This saves user code from having to parse the numbers, as we already did that while Tokenizing. :^) As a bonus, we now round extremely large integers to the closest available value, like the spec tells us to.
2021-11-18 11:46:27 +00:00
{
VERIFY(m_type == Type::Number);
LibWeb: Use CSS::Number for Token numeric values
2022-03-21 21:01:27 +00:00
return m_number_value.value();
LibWeb: Return numeric values from Token value getters This saves user code from having to parse the numbers, as we already did that while Tokenizing. :^) As a bonus, we now round extremely large integers to the closest available value, like the spec tells us to.
2021-11-18 11:46:27 +00:00
}
LibWeb: Distinguish between integer and float in NumericStyleValue We have this information when parsing, and some properties specifically only allow integers, so it makes sense to keep that around.
2021-10-19 16:43:56 +01:00
i64 to_integer() const
LibWeb: Implement CSS::parse_css_value() A lot of this is not spec-compliant and copied from the old parser. In future PRs, we can revise it.
2021-07-09 21:04:34 +01:00
{
LibWeb: Use CSS::Number for Token numeric values
2022-03-21 21:01:27 +00:00
VERIFY(m_type == Type::Number && m_number_value.is_integer());
return m_number_value.integer_value();
LibWeb: Make CSS 'An+B' parsing spec-compliant Parsing this pattern from CSS tokens turns out to be slightly crazy, but thankfully well documented in the spec. The spec lists the cases in order of simple -> complicated, but this would cause problems in code, since `<n-dimension> <signed-.integer>` would never by reached, as `<n-dimension>` comes before. Instead, I have grouped them by their first token. Also renamed the NthChildPattern class to ANPlusBPattern, to match spec terminology.
2021-07-24 21:22:44 +01:00
}
StringView dimension_unit() const
{
VERIFY(m_type == Type::Dimension);
LibWeb: Convert CSS Token::m_unit from StringBuilder to FlyString This value doesn't change once it's assigned to the Token, so it can be more lightweight than a StringBuilder.
2021-10-28 12:01:15 +01:00
return m_unit.view();
LibWeb: Make CSS 'An+B' parsing spec-compliant Parsing this pattern from CSS tokens turns out to be slightly crazy, but thankfully well documented in the spec. The spec lists the cases in order of simple -> complicated, but this would cause problems in code, since `<n-dimension> <signed-.integer>` would never by reached, as `<n-dimension>` comes before. Instead, I have grouped them by their first token. Also renamed the NthChildPattern class to ANPlusBPattern, to match spec terminology.
2021-07-24 21:22:44 +01:00
}
LibWeb: Use floats instead of doubles for CSS numbers Using doubles isn't necessary, and they make things slightly bigger and slower, so let's use floats instead.
2022-03-21 17:22:05 +00:00
float dimension_value() const
LibWeb: Make CSS 'An+B' parsing spec-compliant Parsing this pattern from CSS tokens turns out to be slightly crazy, but thankfully well documented in the spec. The spec lists the cases in order of simple -> complicated, but this would cause problems in code, since `<n-dimension> <signed-.integer>` would never by reached, as `<n-dimension>` comes before. Instead, I have grouped them by their first token. Also renamed the NthChildPattern class to ANPlusBPattern, to match spec terminology.
2021-07-24 21:22:44 +01:00
{
VERIFY(m_type == Type::Dimension);
LibWeb: Use CSS::Number for Token numeric values
2022-03-21 21:01:27 +00:00
return m_number_value.value();
LibWeb: Return numeric values from Token value getters This saves user code from having to parse the numbers, as we already did that while Tokenizing. :^) As a bonus, we now round extremely large integers to the closest available value, like the spec tells us to.
2021-11-18 11:46:27 +00:00
}
LibWeb: Use CSS::Number for Token numeric values
2022-03-21 21:01:27 +00:00
i64 dimension_value_int() const { return m_number_value.integer_value(); }
LibWeb: Return numeric values from Token value getters This saves user code from having to parse the numbers, as we already did that while Tokenizing. :^) As a bonus, we now round extremely large integers to the closest available value, like the spec tells us to.
2021-11-18 11:46:27 +00:00
LibWeb: Use floats instead of doubles for CSS numbers Using doubles isn't necessary, and they make things slightly bigger and slower, so let's use floats instead.
2022-03-21 17:22:05 +00:00
float percentage() const
LibWeb: Return numeric values from Token value getters This saves user code from having to parse the numbers, as we already did that while Tokenizing. :^) As a bonus, we now round extremely large integers to the closest available value, like the spec tells us to.
2021-11-18 11:46:27 +00:00
{
VERIFY(m_type == Type::Percentage);
LibWeb: Use CSS::Number for Token numeric values
2022-03-21 21:01:27 +00:00
return m_number_value.value();
LibWeb: Implement CSS::parse_css_value() A lot of this is not spec-compliant and copied from the old parser. In future PRs, we can revise it.
2021-07-09 21:04:34 +01:00
}
LibWeb: Rename CSS::Token::TokenType -> Type
2021-07-09 20:54:06 +01:00
Type mirror_variant() const;
LibWeb: Add specification-based CSS tokenizer Original work by @stelar7 for #2628.
2021-03-09 17:18:08 +01:00
String bracket_string() const;
String bracket_mirror_string() const;
LibWeb: Increase clarity with CSS token debug logging Had to adjust some places that were using Token.to_string() for non-debug-logging purposes. Changed its name to to_debug_string() to make the usage clearer.
2021-07-07 21:29:24 +01:00
LibWeb: Implement CSS::Token::to_string() This outputs valid CSS, as opposed to to_debug_string().
2021-11-24 12:57:21 +00:00
String to_string() const;
LibWeb: Increase clarity with CSS token debug logging Had to adjust some places that were using Token.to_string() for non-debug-logging purposes. Changed its name to to_debug_string() to make the usage clearer.
2021-07-07 21:29:24 +01:00
String to_debug_string() const;
LibWeb: Add specification-based CSS tokenizer Original work by @stelar7 for #2628.
2021-03-09 17:18:08 +01:00
LibWeb: Record position information in CSS Tokens This is a requirement to be able to use the Tokens for syntax highlighting.
2021-10-21 21:25:14 +01:00
Position const& start_position() const { return m_start_position; }
Position const& end_position() const { return m_end_position; }
LibWeb: Add specification-based CSS tokenizer Original work by @stelar7 for #2628.
2021-03-09 17:18:08 +01:00
private:
LibWeb: Rename CSS::Token::TokenType -> Type
2021-07-09 20:54:06 +01:00
Type m_type { Type::Invalid };
LibWeb: Add specification-based CSS tokenizer Original work by @stelar7 for #2628.
2021-03-09 17:18:08 +01:00
LibWeb: Convert CSS Token::m_value from StringBuilder to FlyString Again, this value does not change once we have finished creating the Token, so it can be more lightweight.
2021-11-18 12:30:46 +00:00
FlyString m_value;
LibWeb: Convert CSS Token::m_unit from StringBuilder to FlyString This value doesn't change once it's assigned to the Token, so it can be more lightweight than a StringBuilder.
2021-10-28 12:01:15 +01:00
FlyString m_unit;
LibWeb: Use CSS::Number for Token numeric values
2022-03-21 21:01:27 +00:00
Number m_number_value;
LibWeb: Use floats instead of doubles for CSS numbers Using doubles isn't necessary, and they make things slightly bigger and slower, so let's use floats instead.
2022-03-21 17:22:05 +00:00
HashType m_hash_type { HashType::Unrestricted };
LibWeb: Record position information in CSS Tokens This is a requirement to be able to use the Tokens for syntax highlighting.
2021-10-21 21:25:14 +01:00
Position m_start_position;
Position m_end_position;
LibWeb: Add specification-based CSS tokenizer Original work by @stelar7 for #2628.
2021-03-09 17:18:08 +01:00
};
}
Reference in a new issue Copy permalink