ladybird/Userland/Libraries/LibUnicode/Forward.h

/*
 * Copyright (c) 2021, Tim Flynn <trflynn89@pm.me>
 *
 * SPDX-License-Identifier: BSD-2-Clause
 */

#pragma once

#include <AK/Types.h>

namespace Unicode {

enum class Condition : u8;
enum class GeneralCategory : u8;
enum class Language : u8;
enum class Locale : u16;
enum class Property : u8;
enum class Script : u8;
enum class Territory : u8;
enum class WordBreakProperty : u8;

struct SpecialCasing;
struct UnicodeData;

}
LibUnicode: Add public methods to compare and lookup Unicode properties Adds methods to retrieve a Unicode property from a string and to check if a code point matches a Unicode property. Also adds a <LibUnicode/Forward.h> header. 2021-07-28 21:45:09 -04:00			`/*`
			`* Copyright (c) 2021, Tim Flynn <trflynn89@pm.me>`
			`*`
			`* SPDX-License-Identifier: BSD-2-Clause`
			`*/`

			`#pragma once`

			`#include <AK/Types.h>`

			`namespace Unicode {`

LibUnicode: Generate separate tables for General Category properties Previously, each code point's General Category was part of the generated UnicodeData structure. This ultimately presented two problems, one functional and one performance related: * Some General Categories are applied to unassigned code points, for example the Unassigned (Cn) category. Unassigned code points are strictly excluded from UnicodeData.txt, so by relying on that file, the generator is unable to handle these categories. * Lookups for General Categories are slower when searching through the large UnicodeData hash map. Even though lookups are O(1), the hash function turned out to be slower than binary searching through a category-specific table. So, now a table is generated for each General Category. When querying a code point for a category, a binary search is done on each code point range in that category's table to check if code point has that category. Further, General Categories are now parsed from the UCD file DerivedGeneralCategory.txt. This file is a normal "prop list" file and contains the categories for unassigned code points. 2021-08-10 07:17:24 -04:00			`enum class Condition : u8;`
			`enum class GeneralCategory : u8;`
LibUnicode: Download Unicode's CLDR database and generate locale data The Unicode standard publishes a database known as the Common Locale Data Repository (CLDR). This is a massive set of data from which anyone implementing Unicode's Technical Standard #35 may generate their implementation: https://www.unicode.org/reports/tr35/ This commit updates LibUnicode to download the compressed database and extract a small subset. That subset is used to generate a list of available locales and the territories (AKA regions) associated with each locale. 2021-08-24 19:50:18 -04:00			`enum class Language : u8;`
LibUnicode: Move Locale enumeration from generated UCD data to CLDR data The UCD set of data contained a very small subset of all locales just to handle some special casing rules. This enumeration will be needed within the CLDR generator as well. So rather than duplicate the enum, remove it from the UCD generator in favor of the full list of locales known by the CLDR generator. 2021-08-26 06:42:11 -04:00			`enum class Locale : u16;`
LibUnicode: Generate separate tables for Unicode properties Similar to General Categories, this generates separate tables for the Property list. 2021-08-10 08:27:28 -04:00			`enum class Property : u8;`
LibUnicode: Generate separate tables for General Category properties Previously, each code point's General Category was part of the generated UnicodeData structure. This ultimately presented two problems, one functional and one performance related: * Some General Categories are applied to unassigned code points, for example the Unassigned (Cn) category. Unassigned code points are strictly excluded from UnicodeData.txt, so by relying on that file, the generator is unable to handle these categories. * Lookups for General Categories are slower when searching through the large UnicodeData hash map. Even though lookups are O(1), the hash function turned out to be slower than binary searching through a category-specific table. So, now a table is generated for each General Category. When querying a code point for a category, a binary search is done on each code point range in that category's table to check if code point has that category. Further, General Categories are now parsed from the UCD file DerivedGeneralCategory.txt. This file is a normal "prop list" file and contains the categories for unassigned code points. 2021-08-10 07:17:24 -04:00			`enum class Script : u8;`
LibUnicode: Download Unicode's CLDR database and generate locale data The Unicode standard publishes a database known as the Common Locale Data Repository (CLDR). This is a massive set of data from which anyone implementing Unicode's Technical Standard #35 may generate their implementation: https://www.unicode.org/reports/tr35/ This commit updates LibUnicode to download the compressed database and extract a small subset. That subset is used to generate a list of available locales and the territories (AKA regions) associated with each locale. 2021-08-24 19:50:18 -04:00			`enum class Territory : u8;`
LibUnicode: Generate separate tables for General Category properties Previously, each code point's General Category was part of the generated UnicodeData structure. This ultimately presented two problems, one functional and one performance related: * Some General Categories are applied to unassigned code points, for example the Unassigned (Cn) category. Unassigned code points are strictly excluded from UnicodeData.txt, so by relying on that file, the generator is unable to handle these categories. * Lookups for General Categories are slower when searching through the large UnicodeData hash map. Even though lookups are O(1), the hash function turned out to be slower than binary searching through a category-specific table. So, now a table is generated for each General Category. When querying a code point for a category, a binary search is done on each code point range in that category's table to check if code point has that category. Further, General Categories are now parsed from the UCD file DerivedGeneralCategory.txt. This file is a normal "prop list" file and contains the categories for unassigned code points. 2021-08-10 07:17:24 -04:00			`enum class WordBreakProperty : u8;`
LibUnicode: Add public methods to compare and lookup Unicode properties Adds methods to retrieve a Unicode property from a string and to check if a code point matches a Unicode property. Also adds a <LibUnicode/Forward.h> header. 2021-07-28 21:45:09 -04:00
			`struct SpecialCasing;`
			`struct UnicodeData;`

			`}`