* conditionals on @set/@value, output format
* implement command_warn properly
* floats
* Check everywhere we store values, we check properly what to do if the
  values are already defined.
* check valid nestings
* Implement more parser options
* Character encoding issues - read and pass to Perl correctly.
* gnulib integration for strrchr, strchrnul, asprintf, etc.  Eliminate
  #define _GNU_SOURCE.
* gettext integration
* figure out what to do with parse_texi_line, etc.  Integration with the 
test suites.
* find where "strlen" and "text_append" are used and try to remove them, for 
efficiency.  Also newSVpv (..., 0);
* Add "TODO" anywhere in the code with explanations of what is not done yet.

================================================================

Texinfo::Report::gdt and parse_texi_line -

Multiple Texinfo::Parser objects are created in a run of texi2any.  The
main one is to parse the input file, but it is also used by gdt - the 
string translation function.  (I haven't ruled out that it is also used 
somewhere else as well.)

This makes replacing the Texinfo::Parser module more complicated.

gdt parses pieces Texinfo code that are the translations of strings in 
po_document/texinfo_document.pot.  It does this by adding "@value" 
before strings surrounded by braces, and then calling "parse_texi_line" 
with these @value's set to the appropriate value.  These values could be 
a string, a hash reference (representing a tree element), or an array 
reference (representing a "content array" of tree elements).

parse_texi_line returns a Perl Texinfo tree which is at last converted 
into the output Info/HTML/etc.

These values that are not strings are a challenge for how they can be 
passed back into C.  We have code (in api.c) to translate our C 
representations of the Texinfo tree to Perl data structures, but not 
vice versa.

We deal with this by leaving the @value's undefined, letting an element 
representing a use of an unused @value appear in the parse tree, and 
then substituting in the parse trees for the @value's afterwards, in 
Texinfo::Report::gdt.  See diff below.

Other suggestions:
* Don't call parse_texi_line at all.  This would be a simplification of 
the overall structure of the makeinfo program and might not lose much 
flexibility in return.  Most of the strings in texinfo_document.pot 
don't use any Texinfo commands, and the others could be expressed 
directly in the output format, e.g. instead of

msgid "{category} on {class}: @strong{{name}}

it could be

msgid "{category} on {class}: <strong>{name}</strong>"

Some of the translation strings use Texinfo commands for characters, 
e.g. "@'e".  This would have to be replaced with whatever method other 
programs using gettext use for special characters.  It would also stop 
translation strings being reused across output formats, creating more 
work for translators.

* Write code translating Perl tree elements into C data structures and 
allow them to be passed to the parser as @value's.  I am very reluctant 
to do this.

To use the current implementation, do the following to 
tp/Texinfo/Report.pm:

Index: Texinfo/Report.pm
===================================================================
--- Texinfo/Report.pm   (revision 6344)
+++ Texinfo/Report.pm   (working copy)
@@ -49,6 +49,7 @@ use File::Basename;
 use Locale::Messages;
 # to be able to load a parser if none was given to gdt.
 use Texinfo::Parser;
+use Parsetexi;
 
 # return the errors and warnings
 sub errors($)
@@ -244,7 +245,8 @@ sub _encode_i18n_string($$)
   return Encode::decode($encoding, $string);
 }
 
-# handle translations of in-document strings.
+# Get document translation - handle translations of in-document strings.
+# Return a parsed Texinfo tree
 sub gdt($$;$$)
 {
   my $self = shift;
@@ -376,7 +378,11 @@ sub gdt($$;$$)
     $translation_result =~ s/\{($re)\}/\@value\{_$1\}/g;
     foreach my $substitution(keys %$context) {
       #print STDERR "$translation_result $substitution $context->{$substitution}\n";
-      $parser_conf->{'values'}->{'_'.$substitution} = $context->{$substitution};
+      if (!ref($context->{$substitution})) {
+        #warn "setting $substitution to $context->{$substitution}\n";
+        $parser_conf->{'values'}->{'_'.$substitution}
+          = $context->{$substitution};
+      }
     }
   }
 
@@ -399,7 +405,8 @@ sub gdt($$;$$)
     }
   }
   #my $parser = Texinfo::Parser::parser($parser_conf);
-  my $parser = Texinfo::Parser::simple_parser($parser_conf);
+  #my $parser = Texinfo::Parser::simple_parser($parser_conf);
+  my $parser = Parsetexi::parser($parser_conf);
   if ($parser->{'DEBUG'}) {
     print STDERR "GDT $translation_result\n";
   }
@@ -411,10 +418,67 @@ sub gdt($$;$$)
   } else {
     $tree = $parser->parse_texi_line($translation_result);
   }
+  #warn '--------------------------------------', "\n";
+  #warn Texinfo::Parser::_print_tree ($tree);
+  $tree = _substitute ($tree, $context);
+  #warn Texinfo::Parser::_print_tree ($tree);
+  #warn '--------------------------------------', "\n";
   return $tree;
 }
 
+sub _substitute_element_array ($$);
+sub _substitute_element_array ($$) {
+  my $array = shift; my $context = shift;
 
+  # "the push @{$current->{'contents'}}, {}; prevents a trailing
+  # text to be merged, to avoid having the value tree modified."
+
+  # It's not necessary now, but we are yet to update the test
+  # results to remove the extra {}'s.
+
+  @{$array} = map {
+    if ($_->{'cmdname'} and $_->{'cmdname'} eq 'value'
+      and $_->{'type'}
+    ) {
+      my $name = $_->{'type'};
+      $name =~ s/^_//;
+      if (ref($context->{$name}) eq 'HASH') {
+        ( $context->{$name} , {} );
+      } elsif (ref($context->{$name}) eq 'ARRAY') {
+        ( @{$context->{$name}} , {} );
+      }
+    } else {
+      _substitute($_, $context);
+      ( $_ );
+    }
+  } @{$array};
+}
+
+sub _substitute ($$);
+sub _substitute ($$) {
+  my $tree = shift; my $context = shift;
+
+  if ($tree->{'contents'}) {
+    _substitute_element_array ($tree->{'contents'}, $context);
+  }
+
+  if ($tree->{'args'}) {
+    _substitute_element_array ($tree->{'args'}, $context);
+  }
+
+  # Used for @email and @url
+  if ($tree->{'extra'} and $tree->{'extra'}{'brace_command_contents'}) {
+    for my $arg (@{$tree->{'extra'}{'brace_command_contents'}}) {
+      if ($arg) {
+        _substitute_element_array ($arg, $context);
+      }
+    }
+  }
+
+  return $tree;
+}
+
+
 1;
 
 __END__


===================================================================


Integration with rest of Perl code:

Passing data as Perl code to be evaluated in very slow, which limits the 
speed-up that is obtained. One way round this would be to access the 
parse tree in the Perl code through an API. This would be done in two 
stages:

* Convert makeinfo backends to use a Perl stub for the C API. The C 
parser will not be used at this point.
* Substitute the XS API implementation for the Perl API when the C 
parser is good enough.

