Stowage/ladybird
2
0
Fork 0
mirror of https://github.com/LadybirdBrowser/ladybird.git synced 2025-12-08 06:09:58 +00:00
Code Issues Projects Releases Packages Wiki Activity
ladybird/Tests/AK/TestStringConversions.cpp

693 lines
53 KiB
C++
Raw Normal View History

AK: Add an exact and fast floating point parsing algorithm This is based on the paper by Daniel Lemire called "Number parsing at a Gigabyte per second", currently available at https://arxiv.org/abs/2101.11408 An implementation can be found at https://github.com/fastfloat/fast_float To support both strtod like methods and String::to_double we have two different APIs. The parse_first_floating_point gives back both the result, next character to read and the error/out of range status. Out of range here means we rounded to infinity 0. The other API, parse_floating_point_completely, will return a floating point only if the given character range contains just the floating point and nothing else. This can be much faster as we can skip actually computing the value if we notice we did not parse the whole range. Both of these APIs support a very lenient format to be usable in as many places as possible. Also it does not check for "named" values like "nan", "inf", "NAN" etc. Because this can be different for every usage. For integers and small values this new method is not faster and often even a tiny bit slower than the current strtod implementation. However the strtod implementation is wrong for a lot of values and has a much less predictable running time. For correctness this method was tested against known string -> double datasets from https://github.com/nigeltao/parse-number-fxx-test-data This method gives 100% accuracy. The old strtod gave an incorrect value in over 50% of the numbers tested.
2022-10-13 02:18:56 +02:00
/*
* Copyright (c) 2020, the SerenityOS developers.
*
* SPDX-License-Identifier: BSD-2-Clause
*/
#include <LibTest/TestCase.h>
AK+Everywhere: Replace custom number parsers with fast_float Our floating point number parser was based on the fast_float library: https://github.com/fastfloat/fast_float However, our implementation only supports 8-bit characters. To support UTF-16, we will need to be able to convert char16_t-based strings to numbers as well. This works out-of-the-box with fast_float. We can also use fast_float for integer parsing.
2025-06-26 19:06:46 -04:00
#include <AK/StringConversions.h>
AK: Implement UTF-16 string-to-number conversions
2025-06-27 13:56:17 -04:00
#include <AK/Utf16View.h>
AK: Add an exact and fast floating point parsing algorithm This is based on the paper by Daniel Lemire called "Number parsing at a Gigabyte per second", currently available at https://arxiv.org/abs/2101.11408 An implementation can be found at https://github.com/fastfloat/fast_float To support both strtod like methods and String::to_double we have two different APIs. The parse_first_floating_point gives back both the result, next character to read and the error/out of range status. Out of range here means we rounded to infinity 0. The other API, parse_floating_point_completely, will return a floating point only if the given character range contains just the floating point and nothing else. This can be much faster as we can skip actually computing the value if we notice we did not parse the whole range. Both of these APIs support a very lenient format to be usable in as many places as possible. Also it does not check for "named" values like "nan", "inf", "NAN" etc. Because this can be different for every usage. For integers and small values this new method is not faster and often even a tiny bit slower than the current strtod implementation. However the strtod implementation is wrong for a lot of values and has a much less predictable running time. For correctness this method was tested against known string -> double datasets from https://github.com/nigeltao/parse-number-fxx-test-data This method gives 100% accuracy. The old strtod gave an incorrect value in over 50% of the numbers tested.
2022-10-13 02:18:56 +02:00
AK: Implement UTF-16 string-to-number conversions
2025-06-27 13:56:17 -04:00
template<typename ViewType>
static double parse_complete_double(ViewType const& view)
AK: Add an exact and fast floating point parsing algorithm This is based on the paper by Daniel Lemire called "Number parsing at a Gigabyte per second", currently available at https://arxiv.org/abs/2101.11408 An implementation can be found at https://github.com/fastfloat/fast_float To support both strtod like methods and String::to_double we have two different APIs. The parse_first_floating_point gives back both the result, next character to read and the error/out of range status. Out of range here means we rounded to infinity 0. The other API, parse_floating_point_completely, will return a floating point only if the given character range contains just the floating point and nothing else. This can be much faster as we can skip actually computing the value if we notice we did not parse the whole range. Both of these APIs support a very lenient format to be usable in as many places as possible. Also it does not check for "named" values like "nan", "inf", "NAN" etc. Because this can be different for every usage. For integers and small values this new method is not faster and often even a tiny bit slower than the current strtod implementation. However the strtod implementation is wrong for a lot of values and has a much less predictable running time. For correctness this method was tested against known string -> double datasets from https://github.com/nigeltao/parse-number-fxx-test-data This method gives 100% accuracy. The old strtod gave an incorrect value in over 50% of the numbers tested.
2022-10-13 02:18:56 +02:00
{
AK+Everywhere: Replace custom number parsers with fast_float Our floating point number parser was based on the fast_float library: https://github.com/fastfloat/fast_float However, our implementation only supports 8-bit characters. To support UTF-16, we will need to be able to convert char16_t-based strings to numbers as well. This works out-of-the-box with fast_float. We can also use fast_float for integer parsing.
2025-06-26 19:06:46 -04:00
return AK::parse_number<double>(view, TrimWhitespace::No).release_value();
AK: Add an exact and fast floating point parsing algorithm This is based on the paper by Daniel Lemire called "Number parsing at a Gigabyte per second", currently available at https://arxiv.org/abs/2101.11408 An implementation can be found at https://github.com/fastfloat/fast_float To support both strtod like methods and String::to_double we have two different APIs. The parse_first_floating_point gives back both the result, next character to read and the error/out of range status. Out of range here means we rounded to infinity 0. The other API, parse_floating_point_completely, will return a floating point only if the given character range contains just the floating point and nothing else. This can be much faster as we can skip actually computing the value if we notice we did not parse the whole range. Both of these APIs support a very lenient format to be usable in as many places as possible. Also it does not check for "named" values like "nan", "inf", "NAN" etc. Because this can be different for every usage. For integers and small values this new method is not faster and often even a tiny bit slower than the current strtod implementation. However the strtod implementation is wrong for a lot of values and has a much less predictable running time. For correctness this method was tested against known string -> double datasets from https://github.com/nigeltao/parse-number-fxx-test-data This method gives 100% accuracy. The old strtod gave an incorrect value in over 50% of the numbers tested.
2022-10-13 02:18:56 +02:00
}
AK: Implement UTF-16 string-to-number conversions
2025-06-27 13:56:17 -04:00
template<typename ViewType>
static float parse_complete_float(ViewType const& view)
AK: Add an exact and fast floating point parsing algorithm This is based on the paper by Daniel Lemire called "Number parsing at a Gigabyte per second", currently available at https://arxiv.org/abs/2101.11408 An implementation can be found at https://github.com/fastfloat/fast_float To support both strtod like methods and String::to_double we have two different APIs. The parse_first_floating_point gives back both the result, next character to read and the error/out of range status. Out of range here means we rounded to infinity 0. The other API, parse_floating_point_completely, will return a floating point only if the given character range contains just the floating point and nothing else. This can be much faster as we can skip actually computing the value if we notice we did not parse the whole range. Both of these APIs support a very lenient format to be usable in as many places as possible. Also it does not check for "named" values like "nan", "inf", "NAN" etc. Because this can be different for every usage. For integers and small values this new method is not faster and often even a tiny bit slower than the current strtod implementation. However the strtod implementation is wrong for a lot of values and has a much less predictable running time. For correctness this method was tested against known string -> double datasets from https://github.com/nigeltao/parse-number-fxx-test-data This method gives 100% accuracy. The old strtod gave an incorrect value in over 50% of the numbers tested.
2022-10-13 02:18:56 +02:00
{
AK+Everywhere: Replace custom number parsers with fast_float Our floating point number parser was based on the fast_float library: https://github.com/fastfloat/fast_float However, our implementation only supports 8-bit characters. To support UTF-16, we will need to be able to convert char16_t-based strings to numbers as well. This works out-of-the-box with fast_float. We can also use fast_float for integer parsing.
2025-06-26 19:06:46 -04:00
return AK::parse_number<float>(view, TrimWhitespace::No).release_value();
AK: Add an exact and fast floating point parsing algorithm This is based on the paper by Daniel Lemire called "Number parsing at a Gigabyte per second", currently available at https://arxiv.org/abs/2101.11408 An implementation can be found at https://github.com/fastfloat/fast_float To support both strtod like methods and String::to_double we have two different APIs. The parse_first_floating_point gives back both the result, next character to read and the error/out of range status. Out of range here means we rounded to infinity 0. The other API, parse_floating_point_completely, will return a floating point only if the given character range contains just the floating point and nothing else. This can be much faster as we can skip actually computing the value if we notice we did not parse the whole range. Both of these APIs support a very lenient format to be usable in as many places as possible. Also it does not check for "named" values like "nan", "inf", "NAN" etc. Because this can be different for every usage. For integers and small values this new method is not faster and often even a tiny bit slower than the current strtod implementation. However the strtod implementation is wrong for a lot of values and has a much less predictable running time. For correctness this method was tested against known string -> double datasets from https://github.com/nigeltao/parse-number-fxx-test-data This method gives 100% accuracy. The old strtod gave an incorrect value in over 50% of the numbers tested.
2022-10-13 02:18:56 +02:00
}
TEST_CASE(simple_cases)
{
AK: Implement UTF-16 string-to-number conversions
2025-06-27 13:56:17 -04:00
#define DOES_PARSE_DOUBLE_LIKE_CPP(value) \
do { \
EXPECT_EQ(static_cast<double>(value), parse_complete_double(#value##sv)); \
EXPECT_EQ(-static_cast<double>(value), parse_complete_double("-" #value##sv)); \
\
EXPECT_EQ(static_cast<double>(value), parse_complete_double(u"" #value##sv)); \
EXPECT_EQ(-static_cast<double>(value), parse_complete_double(u"-" #value##sv)); \
AK: Add an exact and fast floating point parsing algorithm This is based on the paper by Daniel Lemire called "Number parsing at a Gigabyte per second", currently available at https://arxiv.org/abs/2101.11408 An implementation can be found at https://github.com/fastfloat/fast_float To support both strtod like methods and String::to_double we have two different APIs. The parse_first_floating_point gives back both the result, next character to read and the error/out of range status. Out of range here means we rounded to infinity 0. The other API, parse_floating_point_completely, will return a floating point only if the given character range contains just the floating point and nothing else. This can be much faster as we can skip actually computing the value if we notice we did not parse the whole range. Both of these APIs support a very lenient format to be usable in as many places as possible. Also it does not check for "named" values like "nan", "inf", "NAN" etc. Because this can be different for every usage. For integers and small values this new method is not faster and often even a tiny bit slower than the current strtod implementation. However the strtod implementation is wrong for a lot of values and has a much less predictable running time. For correctness this method was tested against known string -> double datasets from https://github.com/nigeltao/parse-number-fxx-test-data This method gives 100% accuracy. The old strtod gave an incorrect value in over 50% of the numbers tested.
2022-10-13 02:18:56 +02:00
} while (false)
#define DOES_PARSE_FLOAT_LIKE_CPP(value) \
do { \
float val = parse_complete_float(#value##sv); \
EXPECT_EQ(static_cast<float>(value##f), val); \
AK: Implement UTF-16 string-to-number conversions
2025-06-27 13:56:17 -04:00
EXPECT_EQ(-static_cast<float>(value##f), parse_complete_float("-" #value##sv)); \
\
val = parse_complete_float(u"" #value##sv); \
EXPECT_EQ(static_cast<float>(value##f), val); \
AK: Add an exact and fast floating point parsing algorithm This is based on the paper by Daniel Lemire called "Number parsing at a Gigabyte per second", currently available at https://arxiv.org/abs/2101.11408 An implementation can be found at https://github.com/fastfloat/fast_float To support both strtod like methods and String::to_double we have two different APIs. The parse_first_floating_point gives back both the result, next character to read and the error/out of range status. Out of range here means we rounded to infinity 0. The other API, parse_floating_point_completely, will return a floating point only if the given character range contains just the floating point and nothing else. This can be much faster as we can skip actually computing the value if we notice we did not parse the whole range. Both of these APIs support a very lenient format to be usable in as many places as possible. Also it does not check for "named" values like "nan", "inf", "NAN" etc. Because this can be different for every usage. For integers and small values this new method is not faster and often even a tiny bit slower than the current strtod implementation. However the strtod implementation is wrong for a lot of values and has a much less predictable running time. For correctness this method was tested against known string -> double datasets from https://github.com/nigeltao/parse-number-fxx-test-data This method gives 100% accuracy. The old strtod gave an incorrect value in over 50% of the numbers tested.
2022-10-13 02:18:56 +02:00
EXPECT_EQ(-static_cast<float>(value##f), parse_complete_float("-" #value##sv)); \
} while (false)
#define DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(value) \
AK: Implement UTF-16 string-to-number conversions
2025-06-27 13:56:17 -04:00
do { \
DOES_PARSE_DOUBLE_LIKE_CPP(value); \
DOES_PARSE_FLOAT_LIKE_CPP(value); \
} while (false)
AK: Add an exact and fast floating point parsing algorithm This is based on the paper by Daniel Lemire called "Number parsing at a Gigabyte per second", currently available at https://arxiv.org/abs/2101.11408 An implementation can be found at https://github.com/fastfloat/fast_float To support both strtod like methods and String::to_double we have two different APIs. The parse_first_floating_point gives back both the result, next character to read and the error/out of range status. Out of range here means we rounded to infinity 0. The other API, parse_floating_point_completely, will return a floating point only if the given character range contains just the floating point and nothing else. This can be much faster as we can skip actually computing the value if we notice we did not parse the whole range. Both of these APIs support a very lenient format to be usable in as many places as possible. Also it does not check for "named" values like "nan", "inf", "NAN" etc. Because this can be different for every usage. For integers and small values this new method is not faster and often even a tiny bit slower than the current strtod implementation. However the strtod implementation is wrong for a lot of values and has a much less predictable running time. For correctness this method was tested against known string -> double datasets from https://github.com/nigeltao/parse-number-fxx-test-data This method gives 100% accuracy. The old strtod gave an incorrect value in over 50% of the numbers tested.
2022-10-13 02:18:56 +02:00
DOES_PARSE_DOUBLE_LIKE_CPP(2.22507385850720138309e-308);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(10090518465521146875.);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(10052108125844341766.);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(0.);
constexpr u64 negative_zero = 1ull << 63;
EXPECT_EQ(0ull, bit_cast<u64>(parse_complete_double("0"sv)));
EXPECT_EQ(negative_zero, bit_cast<u64>(parse_complete_double("-0"sv)));
EXPECT_EQ(negative_zero, bit_cast<u64>(parse_complete_double("-0."sv)));
EXPECT_EQ(negative_zero, bit_cast<u64>(parse_complete_double("-0.0"sv)));
DOES_PARSE_DOUBLE_LIKE_CPP(2.2222222222223e-322);
DOES_PARSE_DOUBLE_LIKE_CPP(2.2250738585072013e-308);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(1.0);
DOES_PARSE_DOUBLE_LIKE_CPP(0.54e-85);
DOES_PARSE_DOUBLE_LIKE_CPP(123);
DOES_PARSE_DOUBLE_LIKE_CPP(1e10);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(001234.0);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(123.456);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(0.456);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(0.45689544977495495495197116546843576574949654);
DOES_PARSE_DOUBLE_LIKE_CPP(0.45689544977495495495197116546843576574949654e81);
DOES_PARSE_DOUBLE_LIKE_CPP(0.45689544977495495495197116546843576574949654e-81);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(7.2057594037927933e+8);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(234532.3426362);
DOES_PARSE_DOUBLE_LIKE_CPP(860228122.6654514319E+90);
DOES_PARSE_DOUBLE_LIKE_CPP(69294956446009195);
DOES_PARSE_DOUBLE_LIKE_CPP(69294956446009200);
DOES_PARSE_DOUBLE_LIKE_CPP(69294956446009199);
DOES_PARSE_DOUBLE_LIKE_CPP(69294956446009198);
DOES_PARSE_DOUBLE_LIKE_CPP(69294956446009208);
DOES_PARSE_DOUBLE_LIKE_CPP(69294956446009204);
DOES_PARSE_DOUBLE_LIKE_CPP(69294956446009200);
DOES_PARSE_DOUBLE_LIKE_CPP(69294956446009201);
DOES_PARSE_DOUBLE_LIKE_CPP(69294956446009202);
DOES_PARSE_DOUBLE_LIKE_CPP(69294956446009203);
DOES_PARSE_DOUBLE_LIKE_CPP(69294956446009205);
DOES_PARSE_DOUBLE_LIKE_CPP(69294956446009206);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(69294956446009204.001);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(69294956446009204.000);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(69294956446009204.0001);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(69294956446009204.0000);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(69294956446009204.00001);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(69294956446009204.000001);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(69294956446009204.0000001);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(69294956446009204.00000001);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(69294956446009204.000000001);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(69294956446009204.0000000001);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(69294956446009204.00000000001);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(69294956446009204.000000000001);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(69294956446009204.0000000000001);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(69294956446009204.00000000000001);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(69294956446009204.000000000000001);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(69294956446009204.0000000000000001);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(69294956446009204.00000000000000001);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(69294956446009204.000000000000000001);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(69294956446009204.0000000000000000001);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(69294956446009204.00000000000000000001);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(69294956446009204.000000000000000000001);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(69294956446009204.0000000000000000000001);
DOES_PARSE_DOUBLE_LIKE_CPP(69294956446009196);
DOES_PARSE_DOUBLE_LIKE_CPP(69294956446009115);
DOES_PARSE_DOUBLE_LIKE_CPP(692949564460091155);
DOES_PARSE_DOUBLE_LIKE_CPP(6929495644600911557);
DOES_PARSE_DOUBLE_LIKE_CPP(7.0420557077594588669468784357561207962098443483187940792729600000e+59);
DOES_PARSE_DOUBLE_LIKE_CPP(7.0420557077594588669468784357561207962098443483187940792729600000e+59);
DOES_PARSE_DOUBLE_LIKE_CPP(1.7339253062092163730578609458683877051596800000000000000000000000e+42);
DOES_PARSE_DOUBLE_LIKE_CPP(2.0972622234386619214559824785284023792871122537545728000000000000e+52);
DOES_PARSE_DOUBLE_LIKE_CPP(1.0001803374372191849407179462120053338028379051879898808320000000e+57);
DOES_PARSE_DOUBLE_LIKE_CPP(1.8607245283054342363818436991534856973992070520151142825984000000e+58);
DOES_PARSE_DOUBLE_LIKE_CPP(1.9189205311132686907264385602245237137907390376574976000000000000e+52);
DOES_PARSE_DOUBLE_LIKE_CPP(2.8184483231688951563253238886553506793085187889855201280000000000e+54);
DOES_PARSE_DOUBLE_LIKE_CPP(1.7664960224650106892054063261344555646357024359107788800000000000e+53);
DOES_PARSE_DOUBLE_LIKE_CPP(2.1470977154320536489471030463761883783915110400000000000000000000e+45);
DOES_PARSE_DOUBLE_LIKE_CPP(4.4900312744003159009338275160799498340862630046359789166919680000e+61);
DOES_PARSE_DOUBLE_LIKE_CPP(2.2222222222223e-322);
DOES_PARSE_DOUBLE_LIKE_CPP(860228122.6654514319E+90);
DOES_PARSE_DOUBLE_LIKE_CPP(4.9406564584124653e-324);
DOES_PARSE_DOUBLE_LIKE_CPP(4.9406564584124654e-324);
DOES_PARSE_DOUBLE_LIKE_CPP(2.2250738585072009e-308);
DOES_PARSE_DOUBLE_LIKE_CPP(2.2250738585072014e-308);
DOES_PARSE_DOUBLE_LIKE_CPP(1.7976931348623157e308);
DOES_PARSE_DOUBLE_LIKE_CPP(1.7976931348623158e308);
DOES_PARSE_DOUBLE_LIKE_CPP(0.000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000044501477170144022721148195934182639518696390927032912960468522194496444440421538910330590478162701758282983178260792422137401728773891892910553144148156412434867599762821265346585071045737627442980259622449029037796981144446145705102663115100318287949527959668236039986479250965780342141637013812613333119898765515451440315261253813266652951306000184917766328660755595837392240989947807556594098101021612198814605258742579179000071675999344145086087205681577915435923018910334964869420614052182892431445797605163650903606514140377217442262561590244668525767372446430075513332450079650686719491377688478005309963967709758965844137894433796621993967316936280457084866613206797017728916080020698679408551343728867675409720757232455434770912461317493580281734466552734375);
DOES_PARSE_DOUBLE_LIKE_CPP(0.000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000022250738585072008890245868760858598876504231122409594654935248025624400092282356951787758888037591552642309780950434312085877387158357291821993020294379224223559819827501242041788969571311791082261043971979604000454897391938079198936081525613113376149842043271751033627391549782731594143828136275113838604094249464942286316695429105080201815926642134996606517803095075913058719846423906068637102005108723282784678843631944515866135041223479014792369585208321597621066375401613736583044193603714778355306682834535634005074073040135602968046375918583163124224521599262546494300836851861719422417646455137135420132217031370496583210154654068035397417906022589503023501937519773030945763173210852507299305089761582519159720757232455434770912461317493580281734466552734375);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(9007199254740993.);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(9007199254740993.1);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(9007199254740993.0000000000000000000000000000000000000000000000000);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(9007199254740993.0000000000000000000000000000000000000000000000001);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(9007199254740993.00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(9007199254740993.000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(9007199254740993.000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000009007199254740993.000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000009007199254740993.000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(1.17549414062751785924617589866280818433124586473279624003138594271817467598606476997247227700427174568176269531250000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000e-38);
DOES_PARSE_DOUBLE_LIKE_CPP(179769313486231570814527423731704356798070567525844996598917476803157260780028538760589558632766878171540458953514382464234321326889464182768467546703537516986049910576551282076245490090389328944075868508455133942304583236903222948165808559332123348274797826204144723168738177180919299881250404026184124858368.);
DOES_PARSE_DOUBLE_LIKE_CPP(179769313486231570814527423731704356798070567525844996598917476803157260780028538760589558632766878171540458953514382464234321326889464182768467546703537516986049910576551282076245490090389328944075868508455133942304583236903222948165808559332123348274797826204144723168738177180919299881250404026184124858369.);
DOES_PARSE_DOUBLE_LIKE_CPP(179769313486231579999999999999999999999999999999999999999917476803157260780028538760589558632766878171540458953514382464234321326889464182768467546703537516986049910576551282076245490090389328944075868508455133942304583236903222948165808559332123348274797826204144723168738177180919299881250404026184124858368.);
DOES_PARSE_DOUBLE_LIKE_CPP(179769313486231580000000000000000000000000000000000000000000000000057260780028538760589558632766878171540458953514382464234321326889464182768467546703537516986049910576551282076245490090389328944075868508455133942304583236903222948165808559332123348274797826204144723168738177180919299881250404026184124858368.);
DOES_PARSE_DOUBLE_LIKE_CPP(179769313486231580790000000000000000000000000000000000000000000000057260780028538760589558632766878171540458953514382464234321326889464182768467546703537516986049910576551282076245490090389328944075868508455133942304583236903222948165808559332123348274797826204144723168738177180919299881250404026184124858368.);
DOES_PARSE_DOUBLE_LIKE_CPP(179769313486231580793700000000000000000000000000000000000000000000057260780028538760589558632766878171540458953514382464234321326889464182768467546703537516986049910576551282076245490090389328944075868508455133942304583236903222948165808559332123348274797826204144723168738177180919299881250404026184124858368.);
DOES_PARSE_DOUBLE_LIKE_CPP(179769313486231570814527423731704356798070567525844996598917476803157260780028538760589558632766878171540458953514382464234321326889464182768467546703537516986049910576551282076245490090389328944075868508455133942304583236903222948165808559332123348274797826204144723168738177180919299881250404026184124858368.);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(7.2057594037927933e+16);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(7.3177701707893310e+15);
DOES_PARSE_DOUBLE_LIKE_CPP(4.2523296908380055e94);
DOES_PARSE_DOUBLE_LIKE_CPP(4.2523296908380052e94);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(6865415254.161212);
DOES_PARSE_DOUBLE_LIKE_CPP(4.9406564584124654416987984e-324);
DOES_PARSE_DOUBLE_LIKE_CPP(4.94065645841246544177987491e-324);
DOES_PARSE_DOUBLE_LIKE_CPP(1.4821969375237396325297063786046641170951794078429742932767570475020265218106262555958995090849079771393896940863371531927799701310678193891963243880323456343789021395709342135835374515035469463110661559081709961921691500191622274606949531619374201918195088454200951461561942223787156967735130799756700603045611186809318747958358147744773659879163696332033824403891299986257959682272412496899735742714436070441803404780657158346504044105794804160581370804321322475109996680534260007162497295808277148680375104180318034518509429259235026831954987743714947574192329127781743623968254334611203409098600941053918033152755376981653597394514673304353113588214501752867512169200796980994429823492617107911270837728302633695687262616047519259154796600341796875e-323);
DOES_PARSE_DOUBLE_LIKE_CPP(0.14821969375237396325297063786046641170951794078429742932767570475020265218106262555958995090849079771393896940863371531927799701310678193891963243880323456343789021395709342135835374515035469463110661559081709961921691500191622274606949531619374201918195088454200951461561942223787156967735130799756700603045611186809318747958358147744773659879163696332033824403891299986257959682272412496899735742714436070441803404780657158346504044105794804160581370804321322475109996680534260007162497295808277148680375104180318034518509429259235026831954987743714947574192329127781743623968254334611203409098600941053918033152755376981653597394514673304353113588214501752867512169200796980994429823492617107911270837728302633695687262616047519259154796600341796875e-322);
DOES_PARSE_DOUBLE_LIKE_CPP(0000000000000000000000000000.14821969375237396325297063786046641170951794078429742932767570475020265218106262555958995090849079771393896940863371531927799701310678193891963243880323456343789021395709342135835374515035469463110661559081709961921691500191622274606949531619374201918195088454200951461561942223787156967735130799756700603045611186809318747958358147744773659879163696332033824403891299986257959682272412496899735742714436070441803404780657158346504044105794804160581370804321322475109996680534260007162497295808277148680375104180318034518509429259235026831954987743714947574192329127781743623968254334611203409098600941053918033152755376981653597394514673304353113588214501752867512169200796980994429823492617107911270837728302633695687262616047519259154796600341796875e-322);
DOES_PARSE_DOUBLE_LIKE_CPP(0000000000000000000000000000.000000000014821969375237396325297063786046641170951794078429742932767570475020265218106262555958995090849079771393896940863371531927799701310678193891963243880323456343789021395709342135835374515035469463110661559081709961921691500191622274606949531619374201918195088454200951461561942223787156967735130799756700603045611186809318747958358147744773659879163696332033824403891299986257959682272412496899735742714436070441803404780657158346504044105794804160581370804321322475109996680534260007162497295808277148680375104180318034518509429259235026831954987743714947574192329127781743623968254334611203409098600941053918033152755376981653597394514673304353113588214501752867512169200796980994429823492617107911270837728302633695687262616047519259154796600341796875e-312);
DOES_PARSE_DOUBLE_LIKE_CPP(6.422853395936205074295394307286877840745777433986221937532613872508781594512713774248897872701267900937355341040794330502046537234627217353184072348140164415641909271474048258861995623182036767347953342268740983499399650083036318996344797035062154164551204996820412300010174963641101352685223346561236927986431514284038124115288530689401919280970935077214657241686229994045115862318045415323218821842922297191448142071618101950151752445844415136251927e-323);
DOES_PARSE_DOUBLE_LIKE_CPP(6.522853395936205074295394307286877840745777433986221937532613872508781594512713774248897872701267900937355341040794330502046537234627217353184072348140164415641909271474048258861995623182036767347953342268740983499399650083036318996344797035062154164551204996820412300010174963641101352685223346561236927986431514284038124115288530689401919280970935077214657241686229994045115862318045415323218821842922297191448142071618101950151752445844415136251927e-323);
DOES_PARSE_DOUBLE_LIKE_CPP(7.522853395936205074295394307286877840745777433986221937532613872508781594512713774248897872701267900937355341040794330502046537234627217353184072348140164415641909271474048258861995623182036767347953342268740983499399650083036318996344797035062154164551204996820412300010174963641101352685223346561236927986431514284038124115288530689401919280970935077214657241686229994045115862318045415323218821842922297191448142071618101950151752445844415136251927e-323);
DOES_PARSE_DOUBLE_LIKE_CPP(7.5228498395936205074295394307286877840745777433986221937532613872508781594512713774248897872701267900937355341040794330502046537234627217353184072348140164415641909271474048258861995623182036767347953342268740983499399650083036318996344797035062154164551204996820412300010174963641101352685223346561236927986431514284038124115288530689401919280970935077214657241686229994045115862318045415323218821842922297191448142071618101950151752445844415136251927e-323);
DOES_PARSE_DOUBLE_LIKE_CPP(0.5228498395936205074295394307286877840745777433986221937532613872508781594512713774248897872701267900937355341040794330502046537234627217353184072348140164415641909271474048258861995623182036767347953342268740983499399650083036318996344797035062154164551204996820412300010174963641101352685223346561236927986431514284038124115288530689401919280970935077214657241686229994045115862318045415323218821842922297191448142071618101950151752445844415136251927e-323);
// actual interesting (non 19+ digit) failures from current strtod'
DOES_PARSE_DOUBLE_LIKE_CPP(1e126);
DOES_PARSE_DOUBLE_LIKE_CPP(1e210);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(358416272e-33);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(89255e-22);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(8925.5e-21);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(8.9255e-18);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(8925500e-24);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(89256e-22);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(89254e-22);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(3.518437208883201171875e13);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(62.5364939768271845828);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(8.10109172351e-10);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(1.50000000000000011102230246251565404236316680908203125);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(9007199254740991.4999999999999999999999999999999995);
DOES_PARSE_DOUBLE_LIKE_CPP(7.4109846876186981626485318930233205854758970392148714663837852375101326090531312779794975454245398856969484704316857659638998506553390969459816219401617281718945106978546710679176872575177347315553307795408549809608457500958111373034747658096871009590975442271004757307809711118935784838675653998783503015228055934046593739791790738723868299395818481660169122019456499931289798411362062484498678713572180352209017023903285791732520220528974020802906854021606612375549983402671300035812486479041385743401875520901590172592547146296175134159774938718574737870961645638908718119841271673056017045493004705269590165763776884908267986972573366521765567941072508764337560846003984904972149117463085539556354188641513168478436313080237596295773983001708984375001e-324);
DOES_PARSE_DOUBLE_LIKE_CPP(7.4109846876186981626485318930233205854758970392148714663837852375101326090531312779794975454245398856969484704316857659638998506553390969459816219401617281718945106978546710679176872575177347315553307795408549809608457500958111373034747658096871009590975442271004757307809711118935784838675653998783503015228055934046593739791790738723868299395818481660169122019456499931289798411362062484498678713572180352209017023903285791732520220528974020802906854021606612375549983402671300035812486479041385743401875520901590172592547146296175134159774938718574737870961645638908718119841271673056017045493004705269590165763776884908267986972573366521765567941072508764337560846003984904972149117463085539556354188641513168478436313080237596295773983001708984375e-324);
DOES_PARSE_DOUBLE_LIKE_CPP(7.4109846876186981626485318930233205854758970392148714663837852375101326090531312779794975454245398856969484704316857659638998506553390969459816219401617281718945106978546710679176872575177347315553307795408549809608457500958111373034747658096871009590975442271004757307809711118935784838675653998783503015228055934046593739791790738723868299395818481660169122019456499931289798411362062484498678713572180352209017023903285791732520220528974020802906854021606612375549983402671300035812486479041385743401875520901590172592547146296175134159774938718574737870961645638908718119841271673056017045493004705269590165763776884908267986972573366521765567941072508764337560846003984904972149117463085539556354188641513168478436313080237596295773983001708984374999e-324);
DOES_PARSE_DOUBLE_LIKE_CPP(2.4703282292062327208828439643411068618252990130716238221279284125033775363510437593264991818081799618989828234772285886546332835517796989819938739800539093906315035659515570226392290858392449105184435931802849936536152500319370457678249219365623669863658480757001585769269903706311928279558551332927834338409351978015531246597263579574622766465272827220056374006485499977096599470454020828166226237857393450736339007967761930577506740176324673600968951340535537458516661134223766678604162159680461914467291840300530057530849048765391711386591646239524912623653881879636239373280423891018672348497668235089863388587925628302755995657524455507255189313690836254779186948667994968324049705821028513185451396213837722826145437693412532098591327667236328125001e-324);
DOES_PARSE_DOUBLE_LIKE_CPP(2.22507385850720138309e-308);
DOES_PARSE_DOUBLE_LIKE_CPP(1e55);
DOES_PARSE_DOUBLE_LIKE_CPP(1e300);
DOES_PARSE_DOUBLE_LIKE_CPP(1e301);
DOES_PARSE_DOUBLE_LIKE_CPP(1e302);
DOES_PARSE_DOUBLE_LIKE_CPP(1e303);
DOES_PARSE_DOUBLE_LIKE_CPP(1e304);
DOES_PARSE_DOUBLE_LIKE_CPP(1e305);
DOES_PARSE_DOUBLE_LIKE_CPP(1e299);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(3.4028235E38);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(4e31);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(9007199254740991.);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(7.038531E-26);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(46116538.);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(20040229.);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(9771305410219737088.);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(1146.);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(7.0064923216240854e-46);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(1.1877630352973938);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(2.1665680640000002384185791015625e9);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(8.589934335999999523162841796875e+09);
DOES_PARSE_FLOAT_AND_DOUBLE_LIKE_CPP(0.09289376810193062);
AK: Implement UTF-16 string-to-number conversions
2025-06-27 13:56:17 -04:00
#define DOES_PARSE_INT_LIKE_VALUE_LIKE_CPP(value) \
do { \
EXPECT_EQ(static_cast<double>(value##.), parse_complete_double(#value##sv)); \
EXPECT_EQ(-static_cast<double>(value##.), parse_complete_double("-" #value##sv)); \
\
EXPECT_EQ(static_cast<double>(value##.), parse_complete_double(u"" #value##sv)); \
EXPECT_EQ(-static_cast<double>(value##.), parse_complete_double(u"-" #value##sv)); \
AK: Add an exact and fast floating point parsing algorithm This is based on the paper by Daniel Lemire called "Number parsing at a Gigabyte per second", currently available at https://arxiv.org/abs/2101.11408 An implementation can be found at https://github.com/fastfloat/fast_float To support both strtod like methods and String::to_double we have two different APIs. The parse_first_floating_point gives back both the result, next character to read and the error/out of range status. Out of range here means we rounded to infinity 0. The other API, parse_floating_point_completely, will return a floating point only if the given character range contains just the floating point and nothing else. This can be much faster as we can skip actually computing the value if we notice we did not parse the whole range. Both of these APIs support a very lenient format to be usable in as many places as possible. Also it does not check for "named" values like "nan", "inf", "NAN" etc. Because this can be different for every usage. For integers and small values this new method is not faster and often even a tiny bit slower than the current strtod implementation. However the strtod implementation is wrong for a lot of values and has a much less predictable running time. For correctness this method was tested against known string -> double datasets from https://github.com/nigeltao/parse-number-fxx-test-data This method gives 100% accuracy. The old strtod gave an incorrect value in over 50% of the numbers tested.
2022-10-13 02:18:56 +02:00
} while (false)
DOES_PARSE_INT_LIKE_VALUE_LIKE_CPP(0);
DOES_PARSE_INT_LIKE_VALUE_LIKE_CPP(1);
DOES_PARSE_INT_LIKE_VALUE_LIKE_CPP(2);
DOES_PARSE_INT_LIKE_VALUE_LIKE_CPP(20);
DOES_PARSE_INT_LIKE_VALUE_LIKE_CPP(200);
DOES_PARSE_INT_LIKE_VALUE_LIKE_CPP(234);
DOES_PARSE_INT_LIKE_VALUE_LIKE_CPP(8419841);
EXPECT_EQ(67677557565221539913., parse_complete_double("67677557565221539913"sv));
EXPECT_EQ(0., parse_complete_double("2.4703282292062327208828439643411068618252990130716238221279284125033775363510437593264991818081799618989828234772285886546332835517796989819938739800539093906315035659515570226392290858392449105184435931802849936536152500319370457678249219365623669863658480757001585769269903706311928279558551332927834338409351978015531246597263579574622766465272827220056374006485499977096599470454020828166226237857393450736339007967761930577506740176324673600968951340535537458516661134223766678604162159680461914467291840300530057530849048765391711386591646239524912623653881879636239373280423891018672348497668235089863388587925628302755995657524455507255189313690836254779186948667994968324049705821028513185451396213837722826145437693412532098591327667236328125e-324"sv));
EXPECT_EQ(0., parse_complete_double("2.4703282292062327208828439643411068618252990130716238221279284125033775363510437593264991818081799618989828234772285886546332835517796989819938739800539093906315035659515570226392290858392449105184435931802849936536152500319370457678249219365623669863658480757001585769269903706311928279558551332927834338409351978015531246597263579574622766465272827220056374006485499977096599470454020828166226237857393450736339007967761930577506740176324673600968951340535537458516661134223766678604162159680461914467291840300530057530849048765391711386591646239524912623653881879636239373280423891018672348497668235089863388587925628302755995657524455507255189313690836254779186948667994968324049705821028513185451396213837722826145437693412532098591327667236328124999e-324"sv));
AK: Implement UTF-16 string-to-number conversions
2025-06-27 13:56:17 -04:00
#define EXPECT_TO_PARSE_TO_VALUE_EQUAL_TO(expected_val, str) \
do { \
EXPECT_EQ(bit_cast<u64>(expected_val), bit_cast<u64>(parse_complete_double(str##sv))); \
EXPECT_EQ(bit_cast<u64>(expected_val), bit_cast<u64>(parse_complete_double(u##str##sv))); \
} while (false)
AK: Add an exact and fast floating point parsing algorithm This is based on the paper by Daniel Lemire called "Number parsing at a Gigabyte per second", currently available at https://arxiv.org/abs/2101.11408 An implementation can be found at https://github.com/fastfloat/fast_float To support both strtod like methods and String::to_double we have two different APIs. The parse_first_floating_point gives back both the result, next character to read and the error/out of range status. Out of range here means we rounded to infinity 0. The other API, parse_floating_point_completely, will return a floating point only if the given character range contains just the floating point and nothing else. This can be much faster as we can skip actually computing the value if we notice we did not parse the whole range. Both of these APIs support a very lenient format to be usable in as many places as possible. Also it does not check for "named" values like "nan", "inf", "NAN" etc. Because this can be different for every usage. For integers and small values this new method is not faster and often even a tiny bit slower than the current strtod implementation. However the strtod implementation is wrong for a lot of values and has a much less predictable running time. For correctness this method was tested against known string -> double datasets from https://github.com/nigeltao/parse-number-fxx-test-data This method gives 100% accuracy. The old strtod gave an incorrect value in over 50% of the numbers tested.
2022-10-13 02:18:56 +02:00
EXPECT_TO_PARSE_TO_VALUE_EQUAL_TO(0., "1e-324");
EXPECT_TO_PARSE_TO_VALUE_EQUAL_TO(-0., "-1e-324");
EXPECT_TO_PARSE_TO_VALUE_EQUAL_TO(.09289376810193062, "+.09289376810193062");
EXPECT_TO_PARSE_TO_VALUE_EQUAL_TO(-.09289376810193062, "-.09289376810193062");
EXPECT_TO_PARSE_TO_VALUE_EQUAL_TO(0., "+.0e10");
EXPECT_TO_PARSE_TO_VALUE_EQUAL_TO(-0., "-.0e10");
AK: Implement UTF-16 string-to-number conversions
2025-06-27 13:56:17 -04:00
#define EXPECT_TO_PARSE_TO_INFINITY(str) \
do { \
EXPECT_EQ(__builtin_huge_val(), parse_complete_double(str##sv)); \
EXPECT_EQ(__builtin_huge_val(), parse_complete_double("+" str##sv)); \
EXPECT_EQ(-__builtin_huge_val(), parse_complete_double("-" str##sv)); \
EXPECT_EQ(static_cast<float>(__builtin_huge_valf()), parse_complete_float(str##sv)); \
EXPECT_EQ(static_cast<float>(__builtin_huge_valf()), parse_complete_float("+" str##sv)); \
EXPECT_EQ(static_cast<float>(-__builtin_huge_valf()), parse_complete_float("-" str##sv)); \
\
EXPECT_EQ(__builtin_huge_val(), parse_complete_double(u##str##sv)); \
EXPECT_EQ(__builtin_huge_val(), parse_complete_double(u"+" str##sv)); \
EXPECT_EQ(-__builtin_huge_val(), parse_complete_double(u"-" str##sv)); \
EXPECT_EQ(static_cast<float>(__builtin_huge_valf()), parse_complete_float(u##str##sv)); \
EXPECT_EQ(static_cast<float>(__builtin_huge_valf()), parse_complete_float(u"+" str##sv)); \
EXPECT_EQ(static_cast<float>(-__builtin_huge_valf()), parse_complete_float(u"-" str##sv)); \
} while (false)
AK: Add an exact and fast floating point parsing algorithm This is based on the paper by Daniel Lemire called "Number parsing at a Gigabyte per second", currently available at https://arxiv.org/abs/2101.11408 An implementation can be found at https://github.com/fastfloat/fast_float To support both strtod like methods and String::to_double we have two different APIs. The parse_first_floating_point gives back both the result, next character to read and the error/out of range status. Out of range here means we rounded to infinity 0. The other API, parse_floating_point_completely, will return a floating point only if the given character range contains just the floating point and nothing else. This can be much faster as we can skip actually computing the value if we notice we did not parse the whole range. Both of these APIs support a very lenient format to be usable in as many places as possible. Also it does not check for "named" values like "nan", "inf", "NAN" etc. Because this can be different for every usage. For integers and small values this new method is not faster and often even a tiny bit slower than the current strtod implementation. However the strtod implementation is wrong for a lot of values and has a much less predictable running time. For correctness this method was tested against known string -> double datasets from https://github.com/nigeltao/parse-number-fxx-test-data This method gives 100% accuracy. The old strtod gave an incorrect value in over 50% of the numbers tested.
2022-10-13 02:18:56 +02:00
EXPECT_TO_PARSE_TO_INFINITY("123.456e789");
EXPECT_TO_PARSE_TO_INFINITY("123456.456789e789");
EXPECT_TO_PARSE_TO_INFINITY("1438456663141390273526118207642235581183227845246331231162636653790368152091394196930365828634687637948157940776599182791387527135353034738357134110310609455693900824193549772792016543182680519740580354365467985440183598701312257624545562331397018329928613196125590274187720073914818062530830316533158098624984118889298281371812288789537310599037529113415438738954894752124724983067241108764488346454376699018673078404751121414804937224240805993123816932326223683090770561597570457793932985826162604255884529134126396282202126526253389383421806727954588525596114379801269094096329805054803089299736996870951258573010877404407451953846698609198213926882692078557033228265259305481198526059813164469187586693257335779522020407645498684263339921905227556616698129967412891282231685504660671277927198290009824680186319750978665734576683784255802269708917361719466043175201158849097881370477111850171579869056016061666173029059588433776015644439705050377554277696143928278093453792803846252715966016733222646442382892123940052441346822429721593884378212558701004356924243030059517489346646577724622498919752597382095222500311124181823512251071356181769376577651390028297796156208815375089159128394945710515861334486267101797497111125909272505194792870889617179758703442608016143343262159998149700606597792535574457560429226974273443630323818747730771316763398572110874959981923732463076884528677392654150010269822239401993427482376513231389212353583573566376915572650916866553612366187378959554983566712767093372906030188976220169058025354973622211666504549316958271880975697143546564469806791358707318873075708383345004090151974068325838177531266954177406661392229801349994695941509935655355652985723782153570084089560139142231.738475042362596875449154552392299548947138162081694168675340677843807613129780449323363759027012972466987370921816813162658754726545121090545507240267000456594786540949605260722461937870630634874991729398208026467698131898691830012167897399682179601734569071423681e-733");
EXPECT_TO_PARSE_TO_INFINITY("3e182947912346759234");
EXPECT_TO_PARSE_TO_INFINITY("3e70000000000000");
EXPECT_TO_PARSE_TO_INFINITY("3e70000000000");
EXPECT_TO_PARSE_TO_INFINITY("3e70000000");
EXPECT_TO_PARSE_TO_INFINITY("1e681");
EXPECT_TO_PARSE_TO_INFINITY("7e312");
EXPECT_TO_PARSE_TO_INFINITY("184467440737095516151234567890e2147483639");
}
TEST_CASE(partial_parse_stops_at_right_spot)
{
AK+Everywhere: Replace custom number parsers with fast_float Our floating point number parser was based on the fast_float library: https://github.com/fastfloat/fast_float However, our implementation only supports 8-bit characters. To support UTF-16, we will need to be able to convert char16_t-based strings to numbers as well. This works out-of-the-box with fast_float. We can also use fast_float for integer parsing.
2025-06-26 19:06:46 -04:00
#define EXPECT_PARSE_TO_VALUE_AND_CONSUME_CHARS(string_value, double_value, chars_parsed) \
do { \
auto result = AK::parse_first_number<double>(string_value##sv); \
VERIFY(result.has_value()); \
EXPECT_EQ(bit_cast<u64>(result->value), bit_cast<u64>(static_cast<double>(double_value))); \
AK: Implement UTF-16 string-to-number conversions
2025-06-27 13:56:17 -04:00
EXPECT_EQ(result->characters_parsed, chars_parsed##uz); \
\
result = AK::parse_first_number<double>(u##string_value##sv); \
VERIFY(result.has_value()); \
EXPECT_EQ(bit_cast<u64>(result->value), bit_cast<u64>(static_cast<double>(double_value))); \
AK+Everywhere: Replace custom number parsers with fast_float Our floating point number parser was based on the fast_float library: https://github.com/fastfloat/fast_float However, our implementation only supports 8-bit characters. To support UTF-16, we will need to be able to convert char16_t-based strings to numbers as well. This works out-of-the-box with fast_float. We can also use fast_float for integer parsing.
2025-06-26 19:06:46 -04:00
EXPECT_EQ(result->characters_parsed, chars_parsed##uz); \
AK: Add an exact and fast floating point parsing algorithm This is based on the paper by Daniel Lemire called "Number parsing at a Gigabyte per second", currently available at https://arxiv.org/abs/2101.11408 An implementation can be found at https://github.com/fastfloat/fast_float To support both strtod like methods and String::to_double we have two different APIs. The parse_first_floating_point gives back both the result, next character to read and the error/out of range status. Out of range here means we rounded to infinity 0. The other API, parse_floating_point_completely, will return a floating point only if the given character range contains just the floating point and nothing else. This can be much faster as we can skip actually computing the value if we notice we did not parse the whole range. Both of these APIs support a very lenient format to be usable in as many places as possible. Also it does not check for "named" values like "nan", "inf", "NAN" etc. Because this can be different for every usage. For integers and small values this new method is not faster and often even a tiny bit slower than the current strtod implementation. However the strtod implementation is wrong for a lot of values and has a much less predictable running time. For correctness this method was tested against known string -> double datasets from https://github.com/nigeltao/parse-number-fxx-test-data This method gives 100% accuracy. The old strtod gave an incorrect value in over 50% of the numbers tested.
2022-10-13 02:18:56 +02:00
} while (false)
EXPECT_PARSE_TO_VALUE_AND_CONSUME_CHARS("0x", 0., 1);
EXPECT_PARSE_TO_VALUE_AND_CONSUME_CHARS("0e", 0., 1);
EXPECT_PARSE_TO_VALUE_AND_CONSUME_CHARS("0e+", 0., 1);
EXPECT_PARSE_TO_VALUE_AND_CONSUME_CHARS("0e1", 0., 3);
EXPECT_PARSE_TO_VALUE_AND_CONSUME_CHARS("0beef", 0., 1);
EXPECT_PARSE_TO_VALUE_AND_CONSUME_CHARS("0p123", 0., 1);
EXPECT_PARSE_TO_VALUE_AND_CONSUME_CHARS("0e1", 0., 3);
EXPECT_PARSE_TO_VALUE_AND_CONSUME_CHARS("0e1abc", 0., 3);
EXPECT_PARSE_TO_VALUE_AND_CONSUME_CHARS("0e1e1", 0., 3);
EXPECT_PARSE_TO_VALUE_AND_CONSUME_CHARS("0e1+", 0., 3);
EXPECT_PARSE_TO_VALUE_AND_CONSUME_CHARS("0e-+1", 0., 1);
}
TEST_CASE(invalid_parse)
{
AK: Implement UTF-16 string-to-number conversions
2025-06-27 13:56:17 -04:00
#define EXPECT_PARSE_TO_FAIL(string_value) \
do { \
EXPECT(!AK::parse_first_number<double>(string_value##sv).has_value()); \
EXPECT(!AK::parse_first_number<double>(u##string_value##sv).has_value()); \
AK: Add an exact and fast floating point parsing algorithm This is based on the paper by Daniel Lemire called "Number parsing at a Gigabyte per second", currently available at https://arxiv.org/abs/2101.11408 An implementation can be found at https://github.com/fastfloat/fast_float To support both strtod like methods and String::to_double we have two different APIs. The parse_first_floating_point gives back both the result, next character to read and the error/out of range status. Out of range here means we rounded to infinity 0. The other API, parse_floating_point_completely, will return a floating point only if the given character range contains just the floating point and nothing else. This can be much faster as we can skip actually computing the value if we notice we did not parse the whole range. Both of these APIs support a very lenient format to be usable in as many places as possible. Also it does not check for "named" values like "nan", "inf", "NAN" etc. Because this can be different for every usage. For integers and small values this new method is not faster and often even a tiny bit slower than the current strtod implementation. However the strtod implementation is wrong for a lot of values and has a much less predictable running time. For correctness this method was tested against known string -> double datasets from https://github.com/nigeltao/parse-number-fxx-test-data This method gives 100% accuracy. The old strtod gave an incorrect value in over 50% of the numbers tested.
2022-10-13 02:18:56 +02:00
} while (false)
EXPECT_PARSE_TO_FAIL("");
EXPECT_PARSE_TO_FAIL("e");
EXPECT_PARSE_TO_FAIL(".");
EXPECT_PARSE_TO_FAIL("-.");
EXPECT_PARSE_TO_FAIL("+.");
EXPECT_PARSE_TO_FAIL(".e");
EXPECT_PARSE_TO_FAIL("-.e");
EXPECT_PARSE_TO_FAIL("+.e");
EXPECT_PARSE_TO_FAIL(".e1");
EXPECT_PARSE_TO_FAIL("-.e1");
EXPECT_PARSE_TO_FAIL("+.e1");
EXPECT_PARSE_TO_FAIL("++2");
EXPECT_PARSE_TO_FAIL("++1");
EXPECT_PARSE_TO_FAIL("++0");
EXPECT_PARSE_TO_FAIL("++2e1");
EXPECT_PARSE_TO_FAIL("++1e1");
EXPECT_PARSE_TO_FAIL("++0e1");
#define EXPECT_MULTI_SIGNS_TO_FAIL(base_string) \
EXPECT_PARSE_TO_FAIL("++" base_string); \
EXPECT_PARSE_TO_FAIL("--" base_string); \
EXPECT_PARSE_TO_FAIL("+-" base_string); \
EXPECT_PARSE_TO_FAIL("-+" base_string)
EXPECT_MULTI_SIGNS_TO_FAIL("1");
EXPECT_MULTI_SIGNS_TO_FAIL("1.");
EXPECT_MULTI_SIGNS_TO_FAIL("1e1");
EXPECT_MULTI_SIGNS_TO_FAIL("1.e1");
EXPECT_MULTI_SIGNS_TO_FAIL("1.0e1");
EXPECT_MULTI_SIGNS_TO_FAIL("1.0e+1");
EXPECT_MULTI_SIGNS_TO_FAIL("1.0e-1");
}
TEST_CASE(detect_out_of_range_values)
{
AK+Everywhere: Replace custom number parsers with fast_float Our floating point number parser was based on the fast_float library: https://github.com/fastfloat/fast_float However, our implementation only supports 8-bit characters. To support UTF-16, we will need to be able to convert char16_t-based strings to numbers as well. This works out-of-the-box with fast_float. We can also use fast_float for integer parsing.
2025-06-26 19:06:46 -04:00
#define EXPECT_PARSE_TO_HAVE_ERROR(string_value, double_value) \
do { \
auto result = AK::parse_first_number<double>(string_value##sv); \
VERIFY(result.has_value()); \
AK: Implement UTF-16 string-to-number conversions
2025-06-27 13:56:17 -04:00
EXPECT_EQ(bit_cast<u64>(result->value), bit_cast<u64>(static_cast<double>(double_value))); \
\
result = AK::parse_first_number<double>(u##string_value##sv); \
VERIFY(result.has_value()); \
AK+Everywhere: Replace custom number parsers with fast_float Our floating point number parser was based on the fast_float library: https://github.com/fastfloat/fast_float However, our implementation only supports 8-bit characters. To support UTF-16, we will need to be able to convert char16_t-based strings to numbers as well. This works out-of-the-box with fast_float. We can also use fast_float for integer parsing.
2025-06-26 19:06:46 -04:00
EXPECT_EQ(bit_cast<u64>(result->value), bit_cast<u64>(static_cast<double>(double_value))); \
AK: Add an exact and fast floating point parsing algorithm This is based on the paper by Daniel Lemire called "Number parsing at a Gigabyte per second", currently available at https://arxiv.org/abs/2101.11408 An implementation can be found at https://github.com/fastfloat/fast_float To support both strtod like methods and String::to_double we have two different APIs. The parse_first_floating_point gives back both the result, next character to read and the error/out of range status. Out of range here means we rounded to infinity 0. The other API, parse_floating_point_completely, will return a floating point only if the given character range contains just the floating point and nothing else. This can be much faster as we can skip actually computing the value if we notice we did not parse the whole range. Both of these APIs support a very lenient format to be usable in as many places as possible. Also it does not check for "named" values like "nan", "inf", "NAN" etc. Because this can be different for every usage. For integers and small values this new method is not faster and often even a tiny bit slower than the current strtod implementation. However the strtod implementation is wrong for a lot of values and has a much less predictable running time. For correctness this method was tested against known string -> double datasets from https://github.com/nigeltao/parse-number-fxx-test-data This method gives 100% accuracy. The old strtod gave an incorrect value in over 50% of the numbers tested.
2022-10-13 02:18:56 +02:00
} while (false)
AK+Everywhere: Replace custom number parsers with fast_float Our floating point number parser was based on the fast_float library: https://github.com/fastfloat/fast_float However, our implementation only supports 8-bit characters. To support UTF-16, we will need to be able to convert char16_t-based strings to numbers as well. This works out-of-the-box with fast_float. We can also use fast_float for integer parsing.
2025-06-26 19:06:46 -04:00
EXPECT_PARSE_TO_HAVE_ERROR("10e-10000", 0.0);
EXPECT_PARSE_TO_HAVE_ERROR("-10e-10000", -0.0);
EXPECT_PARSE_TO_HAVE_ERROR("10e10000", INFINITY);
EXPECT_PARSE_TO_HAVE_ERROR("-10e10000", -INFINITY);
AK: Add an exact and fast floating point parsing algorithm This is based on the paper by Daniel Lemire called "Number parsing at a Gigabyte per second", currently available at https://arxiv.org/abs/2101.11408 An implementation can be found at https://github.com/fastfloat/fast_float To support both strtod like methods and String::to_double we have two different APIs. The parse_first_floating_point gives back both the result, next character to read and the error/out of range status. Out of range here means we rounded to infinity 0. The other API, parse_floating_point_completely, will return a floating point only if the given character range contains just the floating point and nothing else. This can be much faster as we can skip actually computing the value if we notice we did not parse the whole range. Both of these APIs support a very lenient format to be usable in as many places as possible. Also it does not check for "named" values like "nan", "inf", "NAN" etc. Because this can be different for every usage. For integers and small values this new method is not faster and often even a tiny bit slower than the current strtod implementation. However the strtod implementation is wrong for a lot of values and has a much less predictable running time. For correctness this method was tested against known string -> double datasets from https://github.com/nigeltao/parse-number-fxx-test-data This method gives 100% accuracy. The old strtod gave an incorrect value in over 50% of the numbers tested.
2022-10-13 02:18:56 +02:00
}
TEST_CASE(parse_completely_must_be_just_floating_point)
{
AK: Implement UTF-16 string-to-number conversions
2025-06-27 13:56:17 -04:00
#define EXPECT_PARSE_COMPLETELY_TO_FAIL(string_value) \
do { \
EXPECT(!AK::parse_number<double>(string_value##sv, TrimWhitespace::No).has_value()); \
EXPECT(!AK::parse_number<double>(u##string_value##sv, TrimWhitespace::No).has_value()); \
} while (false)
AK: Add an exact and fast floating point parsing algorithm This is based on the paper by Daniel Lemire called "Number parsing at a Gigabyte per second", currently available at https://arxiv.org/abs/2101.11408 An implementation can be found at https://github.com/fastfloat/fast_float To support both strtod like methods and String::to_double we have two different APIs. The parse_first_floating_point gives back both the result, next character to read and the error/out of range status. Out of range here means we rounded to infinity 0. The other API, parse_floating_point_completely, will return a floating point only if the given character range contains just the floating point and nothing else. This can be much faster as we can skip actually computing the value if we notice we did not parse the whole range. Both of these APIs support a very lenient format to be usable in as many places as possible. Also it does not check for "named" values like "nan", "inf", "NAN" etc. Because this can be different for every usage. For integers and small values this new method is not faster and often even a tiny bit slower than the current strtod implementation. However the strtod implementation is wrong for a lot of values and has a much less predictable running time. For correctness this method was tested against known string -> double datasets from https://github.com/nigeltao/parse-number-fxx-test-data This method gives 100% accuracy. The old strtod gave an incorrect value in over 50% of the numbers tested.
2022-10-13 02:18:56 +02:00
EXPECT_PARSE_COMPLETELY_TO_FAIL("");
EXPECT_PARSE_COMPLETELY_TO_FAIL("-");
EXPECT_PARSE_COMPLETELY_TO_FAIL("+");
EXPECT_PARSE_COMPLETELY_TO_FAIL("++1");
EXPECT_PARSE_COMPLETELY_TO_FAIL("+-1");
EXPECT_PARSE_COMPLETELY_TO_FAIL("-+1");
EXPECT_PARSE_COMPLETELY_TO_FAIL("--1");
EXPECT_PARSE_COMPLETELY_TO_FAIL("1 ");
EXPECT_PARSE_COMPLETELY_TO_FAIL("1. ");
EXPECT_PARSE_COMPLETELY_TO_FAIL("1e ");
EXPECT_PARSE_COMPLETELY_TO_FAIL("1.e ");
EXPECT_PARSE_COMPLETELY_TO_FAIL("1.e123 ");
EXPECT_PARSE_COMPLETELY_TO_FAIL("-1 ");
EXPECT_PARSE_COMPLETELY_TO_FAIL("-1. ");
EXPECT_PARSE_COMPLETELY_TO_FAIL("-1e ");
EXPECT_PARSE_COMPLETELY_TO_FAIL("-1.e ");
EXPECT_PARSE_COMPLETELY_TO_FAIL("-1.e123 ");
EXPECT_PARSE_COMPLETELY_TO_FAIL("1A");
EXPECT_PARSE_COMPLETELY_TO_FAIL("1.C");
EXPECT_PARSE_COMPLETELY_TO_FAIL("1e*");
EXPECT_PARSE_COMPLETELY_TO_FAIL("1.e(");
EXPECT_PARSE_COMPLETELY_TO_FAIL("1.e123]");
EXPECT_PARSE_COMPLETELY_TO_FAIL("1.e123&");
EXPECT_PARSE_COMPLETELY_TO_FAIL("1.e123 ");
EXPECT_PARSE_COMPLETELY_TO_FAIL(":1234567890");
EXPECT_PARSE_COMPLETELY_TO_FAIL("1:234567890");
EXPECT_PARSE_COMPLETELY_TO_FAIL("12:34567890");
EXPECT_PARSE_COMPLETELY_TO_FAIL("123:4567890");
EXPECT_PARSE_COMPLETELY_TO_FAIL("1234:567890");
EXPECT_PARSE_COMPLETELY_TO_FAIL("12345:67890");
EXPECT_PARSE_COMPLETELY_TO_FAIL("123456:7890");
EXPECT_PARSE_COMPLETELY_TO_FAIL("1234567:890");
EXPECT_PARSE_COMPLETELY_TO_FAIL("12345678:90");
EXPECT_PARSE_COMPLETELY_TO_FAIL("123456789:0");
EXPECT_PARSE_COMPLETELY_TO_FAIL("1234567890:");
EXPECT_PARSE_COMPLETELY_TO_FAIL("1;234567890");
EXPECT_PARSE_COMPLETELY_TO_FAIL("1234567;890");
EXPECT_PARSE_COMPLETELY_TO_FAIL("1=234567890");
EXPECT_PARSE_COMPLETELY_TO_FAIL("1234567=890");
}
AK: Add an exact and fast hex float parsing algorithm Similar to decimal floating point parsing the current strtod hex float parsing gives a lot of incorrect results. We can use a similar technique as with decimal parsing however hex floats are much simpler as we don't need to scale with a power of 5. For hex floats we just provide the parse_first_hexfloat API as there is currently no need for a parse_hexfloat_completely API. Again the accepted input for parse_first_hexfloat is very lenient and any validation should be done before calling this method.
2022-10-13 02:23:07 +02:00
Everywhere: Use to_number<T> instead of to_{int,uint,float,double} In a bunch of cases, this actually ends up simplifying the code as to_number will handle something such as: ``` Optional<I> opt; if constexpr (IsSigned<I>) opt = view.to_int<I>(); else opt = view.to_uint<I>(); ``` For us. The main goal here however is to have a single generic number conversion API between all of the String classes.
2023-12-23 15:59:14 +13:00
#define BENCHMARK_DOUBLE_PARSING(value, iterations) \
do { \
auto data = #value##sv; \
auto true_result = value; \
for (int i = 0; i < iterations * 10'000; ++i) { \
AK::taint_for_optimizer(data); \
EXPECT_EQ(data.to_number<double>(), true_result); \
} \
AK: Add benchmarks for floating point parsing
2023-03-09 01:04:18 +03:00
} while (false)
BENCHMARK_CASE(one)
{
BENCHMARK_DOUBLE_PARSING(1, 500);
}
BENCHMARK_CASE(long_float_without_exponent)
{
BENCHMARK_DOUBLE_PARSING(1234567812345678, 400);
}
BENCHMARK_CASE(float_with_exponent)
{
BENCHMARK_DOUBLE_PARSING(1.234567e20, 400);
}
BENCHMARK_CASE(inadequate_float)
{
BENCHMARK_DOUBLE_PARSING(7.4109846876186981626485318930233205854758970392148714663837852375101326090531312779794975454245398856969484704316857659638998506553390969459816219401617281718945106978546710679176872575177347315553307795408549809608457500958111373034747658096871009590975442271004757307809711118935784838675653998783503015228055934046593739791790738723868299395818481660169122019456499931289798411362062484498678713572180352209017023903285791732520220528974020802906854021606612375549983402671300035812486479041385743401875520901590172592547146296175134159774938718574737870961645638908718119841271673056017045493004705269590165763776884908267986972573366521765567941072508764337560846003984904972149117463085539556354188641513168478436313080237596295773983001708984375001e-324, 4);
}
AK+Everywhere: Replace custom number parsers with fast_float Our floating point number parser was based on the fast_float library: https://github.com/fastfloat/fast_float However, our implementation only supports 8-bit characters. To support UTF-16, we will need to be able to convert char16_t-based strings to numbers as well. This works out-of-the-box with fast_float. We can also use fast_float for integer parsing.
2025-06-26 19:06:46 -04:00
TEST_CASE(signed_integer)
{
auto value = AK::parse_number<int>(StringView());
EXPECT(!value.has_value());
value = AK::parse_number<int>(""sv);
EXPECT(!value.has_value());
value = AK::parse_number<int>("a"sv);
EXPECT(!value.has_value());
value = AK::parse_number<int>("+"sv);
EXPECT(!value.has_value());
value = AK::parse_number<int>("-"sv);
EXPECT(!value.has_value());
auto actual = AK::parse_number<int>("0"sv);
EXPECT_EQ(actual.has_value(), true);
EXPECT_EQ(actual.value(), 0);
actual = AK::parse_number<int>("1"sv);
EXPECT_EQ(actual.has_value(), true);
EXPECT_EQ(actual.value(), 1);
actual = AK::parse_number<int>("+1"sv);
EXPECT_EQ(actual.has_value(), true);
EXPECT_EQ(actual.value(), 1);
actual = AK::parse_number<int>("-1"sv);
EXPECT_EQ(actual.has_value(), true);
EXPECT_EQ(actual.value(), -1);
actual = AK::parse_number<int>("01"sv);
EXPECT_EQ(actual.has_value(), true);
EXPECT_EQ(actual.value(), 1);
actual = AK::parse_number<int>("12345"sv);
EXPECT_EQ(actual.has_value(), true);
EXPECT_EQ(actual.value(), 12345);
actual = AK::parse_number<int>("-12345"sv);
EXPECT_EQ(actual.has_value(), true);
EXPECT_EQ(actual.value(), -12345);
actual = AK::parse_number<int>(" \t-12345 \n\n"sv);
EXPECT_EQ(actual.has_value(), true);
EXPECT_EQ(actual.value(), -12345);
auto actual_i8 = AK::parse_number<i8>("-1"sv);
EXPECT(actual_i8.has_value());
EXPECT_EQ(actual_i8.value(), -1);
EXPECT_EQ(sizeof(actual_i8.value()), (size_t)1);
actual_i8 = AK::parse_number<i8>("128"sv);
EXPECT(!actual_i8.has_value());
auto actual_i16 = AK::parse_number<i16>("-1"sv);
EXPECT(actual_i16.has_value());
EXPECT_EQ(actual_i16.value(), -1);
EXPECT_EQ(sizeof(actual_i16.value()), (size_t)2);
actual_i16 = AK::parse_number<i16>("32768"sv);
EXPECT(!actual_i16.has_value());
auto actual_i32 = AK::parse_number<i32>("-1"sv);
EXPECT(actual_i32.has_value());
EXPECT_EQ(actual_i32.value(), -1);
EXPECT_EQ(sizeof(actual_i32.value()), (size_t)4);
actual_i32 = AK::parse_number<i32>("2147483648"sv);
EXPECT(!actual_i32.has_value());
auto actual_i64 = AK::parse_number<i64>("-1"sv);
EXPECT(actual_i64.has_value());
EXPECT_EQ(actual_i64.value(), -1);
EXPECT_EQ(sizeof(actual_i64.value()), (size_t)8);
actual_i64 = AK::parse_number<i64>("9223372036854775808"sv);
EXPECT(!actual_i64.has_value());
}
TEST_CASE(unsigned_integer)
{
auto value = AK::parse_number<unsigned>(StringView());
EXPECT(!value.has_value());
value = AK::parse_number<unsigned>(""sv);
EXPECT(!value.has_value());
value = AK::parse_number<unsigned>("a"sv);
EXPECT(!value.has_value());
value = AK::parse_number<unsigned>("+"sv);
EXPECT(!value.has_value());
value = AK::parse_number<unsigned>("-"sv);
EXPECT(!value.has_value());
value = AK::parse_number<unsigned>("+1"sv);
EXPECT(!value.has_value());
value = AK::parse_number<unsigned>("-1"sv);
EXPECT(!value.has_value());
auto actual = AK::parse_number<unsigned>("0"sv);
EXPECT_EQ(actual.has_value(), true);
EXPECT_EQ(actual.value(), 0u);
actual = AK::parse_number<unsigned>("1"sv);
EXPECT_EQ(actual.has_value(), true);
EXPECT_EQ(actual.value(), 1u);
actual = AK::parse_number<unsigned>("01"sv);
EXPECT_EQ(actual.has_value(), true);
EXPECT_EQ(actual.value(), 1u);
actual = AK::parse_number<unsigned>("12345"sv);
EXPECT_EQ(actual.has_value(), true);
EXPECT_EQ(actual.value(), 12345u);
actual = AK::parse_number<unsigned>(" \t12345 \n\n"sv);
EXPECT_EQ(actual.has_value(), true);
EXPECT_EQ(actual.value(), 12345u);
auto actual_u8 = AK::parse_number<u8>("255"sv);
EXPECT(actual_u8.has_value());
EXPECT_EQ(actual_u8.value(), 255u);
EXPECT_EQ(sizeof(actual_u8.value()), (size_t)1);
actual_u8 = AK::parse_number<u8>("256"sv);
EXPECT(!actual_u8.has_value());
auto actual_u16 = AK::parse_number<u16>("65535"sv);
EXPECT(actual_u16.has_value());
EXPECT_EQ(actual_u16.value(), 65535u);
EXPECT_EQ(sizeof(actual_u16.value()), (size_t)2);
actual_u16 = AK::parse_number<u16>("65536"sv);
EXPECT(!actual_u16.has_value());
auto actual_u32 = AK::parse_number<u32>("4294967295"sv);
EXPECT(actual_u32.has_value());
EXPECT_EQ(actual_u32.value(), 4294967295ul);
EXPECT_EQ(sizeof(actual_u32.value()), (size_t)4);
actual_u32 = AK::parse_number<u32>("4294967296"sv);
EXPECT(!actual_u32.has_value());
auto actual_u64 = AK::parse_number<u64>("18446744073709551615"sv);
EXPECT(actual_u64.has_value());
EXPECT_EQ(actual_u64.value(), 18446744073709551615ull);
EXPECT_EQ(sizeof(actual_u64.value()), (size_t)8);
actual_u64 = AK::parse_number<u64>("18446744073709551616"sv);
EXPECT(!actual_u64.has_value());
}
Meta+Tests: Update fast-float to version 8.1.0 This release comes with a fix for a bug where certain unicode emoji characters encoded in UTF-16 were mistakenly parsed as integers. This manifested in keys of an JS object being coerced into integers, i.e. `{ "⤵️": 42 }` would become `{ "5": 42 }`. Relevant upstream PR: https://github.com/fastfloat/fast_float/pull/325
2025-09-23 22:48:22 +02:00
TEST_CASE(unicode_emojis_with_low_byte_in_UTF_16_matching_the_value_of_an_ascii_digit)
{
#define EXPECT_PARSE_TO_FAIL_FOR_EMOJI(string_value) \
do { \
EXPECT(!AK::parse_number<u32>(string_value##sv).has_value()); \
EXPECT(!AK::parse_number<u32>(u##string_value##sv).has_value()); \
} while (false)
EXPECT_PARSE_TO_FAIL_FOR_EMOJI("ℹ");
EXPECT_PARSE_TO_FAIL_FOR_EMOJI("ℹ️");
EXPECT_PARSE_TO_FAIL_FOR_EMOJI("☸");
EXPECT_PARSE_TO_FAIL_FOR_EMOJI("☸️");
EXPECT_PARSE_TO_FAIL_FOR_EMOJI("☹");
EXPECT_PARSE_TO_FAIL_FOR_EMOJI("☹️");
EXPECT_PARSE_TO_FAIL_FOR_EMOJI("✳");
EXPECT_PARSE_TO_FAIL_FOR_EMOJI("✳️");
EXPECT_PARSE_TO_FAIL_FOR_EMOJI("✴");
EXPECT_PARSE_TO_FAIL_FOR_EMOJI("✴️");
EXPECT_PARSE_TO_FAIL_FOR_EMOJI("⤴");
EXPECT_PARSE_TO_FAIL_FOR_EMOJI("⤴️");
EXPECT_PARSE_TO_FAIL_FOR_EMOJI("⤵");
EXPECT_PARSE_TO_FAIL_FOR_EMOJI("⤵️");
EXPECT_PARSE_TO_FAIL_FOR_EMOJI("〰");
EXPECT_PARSE_TO_FAIL_FOR_EMOJI("〰️");
}
AK+Everywhere: Replace custom number parsers with fast_float Our floating point number parser was based on the fast_float library: https://github.com/fastfloat/fast_float However, our implementation only supports 8-bit characters. To support UTF-16, we will need to be able to convert char16_t-based strings to numbers as well. This works out-of-the-box with fast_float. We can also use fast_float for integer parsing.
2025-06-26 19:06:46 -04:00
TEST_CASE(octal)
{
auto value = AK::parse_number<u16>(StringView(), AK::TrimWhitespace::No, 8);
EXPECT(!value.has_value());
value = AK::parse_number<u16>(""sv, AK::TrimWhitespace::No, 8);
EXPECT(!value.has_value());
value = AK::parse_number<u16>("a"sv, AK::TrimWhitespace::No, 8);
EXPECT(!value.has_value());
value = AK::parse_number<u16>("+"sv, AK::TrimWhitespace::No, 8);
EXPECT(!value.has_value());
value = AK::parse_number<u16>("-"sv, AK::TrimWhitespace::No, 8);
EXPECT(!value.has_value());
value = AK::parse_number<u16>("+1"sv, AK::TrimWhitespace::No, 8);
EXPECT(!value.has_value());
value = AK::parse_number<u16>("-1"sv, AK::TrimWhitespace::No, 8);
EXPECT(!value.has_value());
value = AK::parse_number<u16>("8"sv, AK::TrimWhitespace::No, 8);
EXPECT(!value.has_value());
auto actual = AK::parse_number<u16>("77777777"sv, AK::TrimWhitespace::No, 8);
EXPECT(!actual.has_value());
actual = AK::parse_number<u16>("0"sv, AK::TrimWhitespace::No, 8);
EXPECT_EQ(actual.has_value(), true);
EXPECT_EQ(actual.value(), 0u);
actual = AK::parse_number<u16>("1"sv, AK::TrimWhitespace::No, 8);
EXPECT_EQ(actual.has_value(), true);
EXPECT_EQ(actual.value(), 1u);
actual = AK::parse_number<u16>("0755"sv, AK::TrimWhitespace::No, 8);
EXPECT_EQ(actual.has_value(), true);
EXPECT_EQ(actual.value(), 0755u);
actual = AK::parse_number<u16>("755"sv, AK::TrimWhitespace::No, 8);
EXPECT_EQ(actual.has_value(), true);
EXPECT_EQ(actual.value(), 0755u);
actual = AK::parse_number<u16>(" \t644 \n\n"sv, AK::TrimWhitespace::Yes, 8);
EXPECT_EQ(actual.has_value(), true);
EXPECT_EQ(actual.value(), 0644u);
actual = AK::parse_number<u16>("177777"sv, AK::TrimWhitespace::No, 8);
EXPECT_EQ(actual.has_value(), true);
EXPECT_EQ(actual.value(), 0177777u);
}
Reference in a new issue Copy permalink