LibRegex: Add a regular expression library
This commit is a mix of several commits, squashed into one because the
commits before 'Move regex to own Library and fix all the broken stuff'
were not fixable in any elegant way.
The commits are listed below for "historical" purposes:
- AK: Add options/flags and Errors for regular expressions
Flags can be provided for any possible flavour by adding a new scoped enum.
Handling of flags is done by templated Options class and the overloaded
'|' and '&' operators.
- AK: Add Lexer for regular expressions
The lexer parses the input and extracts tokens needed to parse a regular
expression.
- AK: Add regex Parser and PosixExtendedParser
This patchset adds a abstract parser class that can be derived to implement
different parsers. A parser produces bytecode to be executed within the
regex matcher.
- AK: Add regex matcher
This patchset adds an regex matcher based on the principles of the T-REX VM.
The bytecode pruduced by the respective Parser is put into the matcher and
the VM will recursively execute the bytecode according to the available OpCodes.
Possible improvement: the recursion could be replaced by multi threading capabilities.
To match a Regular expression, e.g. for the Posix standard regular expression matcher
use the following API:
```
Pattern<PosixExtendedParser> pattern("^.*$");
auto result = pattern.match("Well, hello friends!\nHello World!"); // Match whole needle
EXPECT(result.count == 1);
EXPECT(result.matches.at(0).view.starts_with("Well"));
EXPECT(result.matches.at(0).view.end() == "!");
result = pattern.match("Well, hello friends!\nHello World!", PosixFlags::Multiline); // Match line by line
EXPECT(result.count == 2);
EXPECT(result.matches.at(0).view == "Well, hello friends!");
EXPECT(result.matches.at(1).view == "Hello World!");
EXPECT(pattern.has_match("Well,....")); // Just check if match without a result, which saves some resources.
```
- AK: Rework regex to work with opcodes objects
This patchsets reworks the matcher to work on a more structured base.
For that an abstract OpCode class and derived classes for the specific
OpCodes have been added. The respective opcode logic is contained in
each respective execute() method.
- AK: Add benchmark for regex
- AK: Some optimization in regex for runtime and memory
- LibRegex: Move regex to own Library and fix all the broken stuff
Now regex works again and grep utility is also in place for testing.
This commit also fixes the use of regex.h in C by making `regex_t`
an opaque (-ish) type, which makes its behaviour consistent between
C and C++ compilers.
Previously, <regex.h> would've blown C compilers up, and even if it
didn't, would've caused a leak in C code, and not in C++ code (due to
the existence of `OwnPtr` inside the struct).
To make this whole ordeal easier to deal with (for now), this pulls the
definitions of `reg*()` into LibRegex.
pros:
- The circular dependency between LibC and LibRegex is broken
- Eaiser to test (without accidentally pulling in the host's libc!)
cons:
- Using any of the regex.h functions will require the user to link -lregex
- The symbols will be missing from libc, which will be a big surprise
  down the line (especially with shared libs).
Co-Authored-By: Ali Mohammad Pur <ali.mpfard@gmail.com>
											 
										 
										
											2020-04-26 14:45:10 +02:00 
										
									 
								 
							 
							
								
							 
							
								 
							
							
								/*
  
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								 *  Copyright  ( c )  2020 ,  Emanuel  Sprung  < emanuel . sprung @ gmail . com > 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								 * 
							 
						 
					
						
							
								
									
										
										
										
											2021-04-22 01:24:48 -07:00 
										
									 
								 
							 
							
								
									
										 
								
							 
							
								 
							
							
								 *  SPDX - License - Identifier :  BSD - 2 - Clause 
							 
						 
					
						
							
								
									
										
											 
										
											
												LibRegex: Add a regular expression library
This commit is a mix of several commits, squashed into one because the
commits before 'Move regex to own Library and fix all the broken stuff'
were not fixable in any elegant way.
The commits are listed below for "historical" purposes:
- AK: Add options/flags and Errors for regular expressions
Flags can be provided for any possible flavour by adding a new scoped enum.
Handling of flags is done by templated Options class and the overloaded
'|' and '&' operators.
- AK: Add Lexer for regular expressions
The lexer parses the input and extracts tokens needed to parse a regular
expression.
- AK: Add regex Parser and PosixExtendedParser
This patchset adds a abstract parser class that can be derived to implement
different parsers. A parser produces bytecode to be executed within the
regex matcher.
- AK: Add regex matcher
This patchset adds an regex matcher based on the principles of the T-REX VM.
The bytecode pruduced by the respective Parser is put into the matcher and
the VM will recursively execute the bytecode according to the available OpCodes.
Possible improvement: the recursion could be replaced by multi threading capabilities.
To match a Regular expression, e.g. for the Posix standard regular expression matcher
use the following API:
```
Pattern<PosixExtendedParser> pattern("^.*$");
auto result = pattern.match("Well, hello friends!\nHello World!"); // Match whole needle
EXPECT(result.count == 1);
EXPECT(result.matches.at(0).view.starts_with("Well"));
EXPECT(result.matches.at(0).view.end() == "!");
result = pattern.match("Well, hello friends!\nHello World!", PosixFlags::Multiline); // Match line by line
EXPECT(result.count == 2);
EXPECT(result.matches.at(0).view == "Well, hello friends!");
EXPECT(result.matches.at(1).view == "Hello World!");
EXPECT(pattern.has_match("Well,....")); // Just check if match without a result, which saves some resources.
```
- AK: Rework regex to work with opcodes objects
This patchsets reworks the matcher to work on a more structured base.
For that an abstract OpCode class and derived classes for the specific
OpCodes have been added. The respective opcode logic is contained in
each respective execute() method.
- AK: Add benchmark for regex
- AK: Some optimization in regex for runtime and memory
- LibRegex: Move regex to own Library and fix all the broken stuff
Now regex works again and grep utility is also in place for testing.
This commit also fixes the use of regex.h in C by making `regex_t`
an opaque (-ish) type, which makes its behaviour consistent between
C and C++ compilers.
Previously, <regex.h> would've blown C compilers up, and even if it
didn't, would've caused a leak in C code, and not in C++ code (due to
the existence of `OwnPtr` inside the struct).
To make this whole ordeal easier to deal with (for now), this pulls the
definitions of `reg*()` into LibRegex.
pros:
- The circular dependency between LibC and LibRegex is broken
- Eaiser to test (without accidentally pulling in the host's libc!)
cons:
- Using any of the regex.h functions will require the user to link -lregex
- The symbols will be missing from libc, which will be a big surprise
  down the line (especially with shared libs).
Co-Authored-By: Ali Mohammad Pur <ali.mpfard@gmail.com>
											 
										 
										
											2020-04-26 14:45:10 +02:00 
										
									 
								 
							 
							
								
							 
							
								 
							
							
								 */ 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								# pragma once 
  
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								# include  <AK/String.h> 
  
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								# include  <AK/Types.h> 
  
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								# ifdef __serenity__ 
  
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								#     include <regex.h> 
  
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								# else 
  
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								#     include <LibC / regex.h> 
  
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								# endif 
  
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								namespace  regex  {  
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								enum  class  Error  :  u8  {  
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								    NoError  =  __Regex_NoError , 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								    InvalidPattern  =  __Regex_InvalidPattern ,                          // Invalid regular expression.
 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								    InvalidCollationElement  =  __Regex_InvalidCollationElement ,        // Invalid collating element referenced.
 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								    InvalidCharacterClass  =  __Regex_InvalidCharacterClass ,            // Invalid character class type referenced.
 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								    InvalidTrailingEscape  =  __Regex_InvalidTrailingEscape ,            // Trailing \ in pattern.
 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								    InvalidNumber  =  __Regex_InvalidNumber ,                            // Number in \digit invalid or in error.
 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								    MismatchingBracket  =  __Regex_MismatchingBracket ,                  // [ ] imbalance.
 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								    MismatchingParen  =  __Regex_MismatchingParen ,                      // ( ) imbalance.
 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								    MismatchingBrace  =  __Regex_MismatchingBrace ,                      // { } imbalance.
 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								    InvalidBraceContent  =  __Regex_InvalidBraceContent ,                // Content of {} invalid: not a number, number too large, more than two numbers, first larger than second.
 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								    InvalidBracketContent  =  __Regex_InvalidBracketContent ,            // Content of [] invalid.
 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								    InvalidRange  =  __Regex_InvalidRange ,                              // Invalid endpoint in range expression.
 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								    InvalidRepetitionMarker  =  __Regex_InvalidRepetitionMarker ,        // ?, * or + not preceded by valid regular expression.
 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								    ReachedMaxRecursion  =  __Regex_ReachedMaxRecursion ,                // MaximumRecursion has been reached.
 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								    EmptySubExpression  =  __Regex_EmptySubExpression ,                  // Sub expression has empty content.
 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								    InvalidCaptureGroup  =  __Regex_InvalidCaptureGroup ,                // Content of capture group is invalid.
 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								    InvalidNameForCaptureGroup  =  __Regex_InvalidNameForCaptureGroup ,  // Name of capture group is invalid.
 
							 
						 
					
						
							
								
									
										
										
										
											2021-07-29 14:18:51 -04:00 
										
									 
								 
							 
							
								
									
										 
								
							 
							
								 
							
							
								    InvalidNameForProperty  =  __Regex_InvalidNameForProperty ,          // Name of property is invalid.
 
							 
						 
					
						
							
								
									
										
											 
										
											
												LibRegex: Add a regular expression library
This commit is a mix of several commits, squashed into one because the
commits before 'Move regex to own Library and fix all the broken stuff'
were not fixable in any elegant way.
The commits are listed below for "historical" purposes:
- AK: Add options/flags and Errors for regular expressions
Flags can be provided for any possible flavour by adding a new scoped enum.
Handling of flags is done by templated Options class and the overloaded
'|' and '&' operators.
- AK: Add Lexer for regular expressions
The lexer parses the input and extracts tokens needed to parse a regular
expression.
- AK: Add regex Parser and PosixExtendedParser
This patchset adds a abstract parser class that can be derived to implement
different parsers. A parser produces bytecode to be executed within the
regex matcher.
- AK: Add regex matcher
This patchset adds an regex matcher based on the principles of the T-REX VM.
The bytecode pruduced by the respective Parser is put into the matcher and
the VM will recursively execute the bytecode according to the available OpCodes.
Possible improvement: the recursion could be replaced by multi threading capabilities.
To match a Regular expression, e.g. for the Posix standard regular expression matcher
use the following API:
```
Pattern<PosixExtendedParser> pattern("^.*$");
auto result = pattern.match("Well, hello friends!\nHello World!"); // Match whole needle
EXPECT(result.count == 1);
EXPECT(result.matches.at(0).view.starts_with("Well"));
EXPECT(result.matches.at(0).view.end() == "!");
result = pattern.match("Well, hello friends!\nHello World!", PosixFlags::Multiline); // Match line by line
EXPECT(result.count == 2);
EXPECT(result.matches.at(0).view == "Well, hello friends!");
EXPECT(result.matches.at(1).view == "Hello World!");
EXPECT(pattern.has_match("Well,....")); // Just check if match without a result, which saves some resources.
```
- AK: Rework regex to work with opcodes objects
This patchsets reworks the matcher to work on a more structured base.
For that an abstract OpCode class and derived classes for the specific
OpCodes have been added. The respective opcode logic is contained in
each respective execute() method.
- AK: Add benchmark for regex
- AK: Some optimization in regex for runtime and memory
- LibRegex: Move regex to own Library and fix all the broken stuff
Now regex works again and grep utility is also in place for testing.
This commit also fixes the use of regex.h in C by making `regex_t`
an opaque (-ish) type, which makes its behaviour consistent between
C and C++ compilers.
Previously, <regex.h> would've blown C compilers up, and even if it
didn't, would've caused a leak in C code, and not in C++ code (due to
the existence of `OwnPtr` inside the struct).
To make this whole ordeal easier to deal with (for now), this pulls the
definitions of `reg*()` into LibRegex.
pros:
- The circular dependency between LibC and LibRegex is broken
- Eaiser to test (without accidentally pulling in the host's libc!)
cons:
- Using any of the regex.h functions will require the user to link -lregex
- The symbols will be missing from libc, which will be a big surprise
  down the line (especially with shared libs).
Co-Authored-By: Ali Mohammad Pur <ali.mpfard@gmail.com>
											 
										 
										
											2020-04-26 14:45:10 +02:00 
										
									 
								 
							 
							
								
							 
							
								 
							
							
								} ;  
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								inline  String  get_error_string ( Error  error )  
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								{  
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								    switch  ( error )  { 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								    case  Error : : NoError : 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								        return  " No error " ; 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								    case  Error : : InvalidPattern : 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								        return  " Invalid regular expression. " ; 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								    case  Error : : InvalidCollationElement : 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								        return  " Invalid collating element referenced. " ; 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								    case  Error : : InvalidCharacterClass : 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								        return  " Invalid character class type referenced. " ; 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								    case  Error : : InvalidTrailingEscape : 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								        return  " Trailing  \\  in pattern. " ; 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								    case  Error : : InvalidNumber : 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								        return  " Number in  \\ digit invalid or in error. " ; 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								    case  Error : : MismatchingBracket : 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								        return  " [ ] imbalance. " ; 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								    case  Error : : MismatchingParen : 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								        return  " ( ) imbalance. " ; 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								    case  Error : : MismatchingBrace : 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								        return  " { } imbalance. " ; 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								    case  Error : : InvalidBraceContent : 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								        return  " Content of {} invalid: not a number, number too large, more than two numbers, first larger than second. " ; 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								    case  Error : : InvalidBracketContent : 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								        return  " Content of [] invalid. " ; 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								    case  Error : : InvalidRange : 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								        return  " Invalid endpoint in range expression. " ; 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								    case  Error : : InvalidRepetitionMarker : 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								        return  " ?, * or + not preceded by valid regular expression. " ; 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								    case  Error : : ReachedMaxRecursion : 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								        return  " Maximum recursion has been reached. " ; 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								    case  Error : : EmptySubExpression : 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								        return  " Sub expression has empty content. " ; 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								    case  Error : : InvalidCaptureGroup : 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								        return  " Content of capture group is invalid. " ; 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								    case  Error : : InvalidNameForCaptureGroup : 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								        return  " Name of capture group is invalid. " ; 
							 
						 
					
						
							
								
									
										
										
										
											2021-07-29 14:18:51 -04:00 
										
									 
								 
							 
							
								
									
										 
								
							 
							
								 
							
							
								    case  Error : : InvalidNameForProperty : 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								        return  " Name of property is invalid. " ; 
							 
						 
					
						
							
								
									
										
											 
										
											
												LibRegex: Add a regular expression library
This commit is a mix of several commits, squashed into one because the
commits before 'Move regex to own Library and fix all the broken stuff'
were not fixable in any elegant way.
The commits are listed below for "historical" purposes:
- AK: Add options/flags and Errors for regular expressions
Flags can be provided for any possible flavour by adding a new scoped enum.
Handling of flags is done by templated Options class and the overloaded
'|' and '&' operators.
- AK: Add Lexer for regular expressions
The lexer parses the input and extracts tokens needed to parse a regular
expression.
- AK: Add regex Parser and PosixExtendedParser
This patchset adds a abstract parser class that can be derived to implement
different parsers. A parser produces bytecode to be executed within the
regex matcher.
- AK: Add regex matcher
This patchset adds an regex matcher based on the principles of the T-REX VM.
The bytecode pruduced by the respective Parser is put into the matcher and
the VM will recursively execute the bytecode according to the available OpCodes.
Possible improvement: the recursion could be replaced by multi threading capabilities.
To match a Regular expression, e.g. for the Posix standard regular expression matcher
use the following API:
```
Pattern<PosixExtendedParser> pattern("^.*$");
auto result = pattern.match("Well, hello friends!\nHello World!"); // Match whole needle
EXPECT(result.count == 1);
EXPECT(result.matches.at(0).view.starts_with("Well"));
EXPECT(result.matches.at(0).view.end() == "!");
result = pattern.match("Well, hello friends!\nHello World!", PosixFlags::Multiline); // Match line by line
EXPECT(result.count == 2);
EXPECT(result.matches.at(0).view == "Well, hello friends!");
EXPECT(result.matches.at(1).view == "Hello World!");
EXPECT(pattern.has_match("Well,....")); // Just check if match without a result, which saves some resources.
```
- AK: Rework regex to work with opcodes objects
This patchsets reworks the matcher to work on a more structured base.
For that an abstract OpCode class and derived classes for the specific
OpCodes have been added. The respective opcode logic is contained in
each respective execute() method.
- AK: Add benchmark for regex
- AK: Some optimization in regex for runtime and memory
- LibRegex: Move regex to own Library and fix all the broken stuff
Now regex works again and grep utility is also in place for testing.
This commit also fixes the use of regex.h in C by making `regex_t`
an opaque (-ish) type, which makes its behaviour consistent between
C and C++ compilers.
Previously, <regex.h> would've blown C compilers up, and even if it
didn't, would've caused a leak in C code, and not in C++ code (due to
the existence of `OwnPtr` inside the struct).
To make this whole ordeal easier to deal with (for now), this pulls the
definitions of `reg*()` into LibRegex.
pros:
- The circular dependency between LibC and LibRegex is broken
- Eaiser to test (without accidentally pulling in the host's libc!)
cons:
- Using any of the regex.h functions will require the user to link -lregex
- The symbols will be missing from libc, which will be a big surprise
  down the line (especially with shared libs).
Co-Authored-By: Ali Mohammad Pur <ali.mpfard@gmail.com>
											 
										 
										
											2020-04-26 14:45:10 +02:00 
										
									 
								 
							 
							
								
							 
							
								 
							
							
								    } 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								    return  " Undefined error. " ; 
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								}  
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								}  
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								
							 
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								using  regex : : Error ;  
						 
					
						
							
								
							 
							
								
							 
							
								 
							
							
								using  regex : : get_error_string ;