Tokenize_a_string_with_escaping.md 550 B

Tokenize a string with escaping

sub tokenize ($string, :$sep!, :$esc!) {
    return $string.match(/([ <!before $sep | $esc> . | $esc . ]*)+ % $sep/)\
                  .[0].map(*.subst: /$esc )> ./, '', :g);
}
 
say "'$_'" for tokenize 'one^|uno||three^^^^|four^^^|^cuatro|', sep => '|', esc => '^';

Output:

'one|uno'
''
'three^^'
'four^|cuatro'
''

Notable Perl 6 innovations that make this different from the equivalent #Perl solution: