scheme-gnunet/doc/service-communication.tm

<TeXmacs|2.1>

<project|scheme-gnunet.tm>

<style|tmmanual>

<\body>
  To connect with a GNUnet service<index|services> \V this applies to both
  the C and Scheme implementation, the GNUnet service must bind a local
  domain socket<\footnote>
    The C implementation supports Internet sockets as well.
  </footnote> somewhere on the file system and the client (possibly another
  service) must connect to it.<space|1em>Connections to a service can be made
  with the <scm|connect/fibers><index|connect/fibers> procedure from
  <scm|(gnu gnunet mq-impl stream)><index|(gnu gnunet mq-impl stream)>, like
  this:

  <\scm-code>
    (define mq (connect/fibers config "nse" handlers error-handler))
  </scm-code>

  <section|Asynchronuously connecting><index|connecting to services>

  This is an asynchronuous operation: it will \<#2018\>complete\<#2019\>
  immediately and the connection will actually be formed in the
  background.<space|1em>When the connection has actually be formed, the
  <scm|error-handler><index|error-handler> is called with the symbol
  <scm|connection:connected><index|connection:connected>.<space|1em>To
  demonstrate, the following code asynchronuously connects to the NSE
  service, and prints the text <scm|"connected!"> when the connection has
  actually been formed.

  <\scm-code>
    ;; XXX test this, explain 'config' ...

    (define (error-handler error . args)

    \ \ (case error

    \ \ \ \ ((connection:connected)

    \ \ \ \ \ (format #t "connected!~%"))

    \ \ \ \ (else (format #t "unknown error: ~a ~a~%" error args))))

    \;

    (define mq

    \ \ (connect/fibers config "nse" (message-handlers) error-handler))
  </scm-code>

  <section|Message handler>

  <index|message handler>When a message is received by the message queue, the
  corresponding message handler is invoked.<space|1em>Message handlers can be
  constructed with the <scm|message-handler><index|message-handler> macro and
  the <scm|make-message-handler><index|make-message-handler> procedure from
  <scm|(gnu gnunet mq handler)>, as follows:

  <\scm-code>
    (import (gnu gnunet mq handler)

    \ \ \ \ \ \ \ \ (gnu extractor enum)

    \ \ \ \ \ \ \ \ (gnu gnunet message protocols)

    \ \ \ \ \ \ \ \ (gnu gnunet util struct)

    \ \ \ \ \ \ \ \ (gnu gnunet utils bv-slice)

    \ \ \ \ \ \ \ \ (gnu gnunet netstruct syntactic))

    \;

    (define handler/syntactic

    \ \ (message-handler

    \ \ \ (type (symbol-value message-type msg:util:dummy))

    \ \ \ ((interpose code) code)

    \ \ \ ((well-formed? slice)

    \ \ \ \ (= (slice-length slice)

    \ \ \ \ \ \ \ (sizeof /:message-header '())))

    \ \ \ ((handle! slice)

    \ \ \ \ (pk 'message: slice))))

    \;

    (define handler/procedural

    \ \ (make-message-handler

    \ \ \ (symbol-value message-type msg:util:dummy)

    \ \ \ (lambda (thunk) (thunk))

    \ \ \ (lambda (slice)

    \ \ \ \ \ (= (slice-length slice)

    \ \ \ \ \ \ \ \ (sizeof /:message-header '())))

    \ \ \ (lambda (slice)

    \ \ \ \ \ (pk 'message: slice))))
  </scm-code>

  As illustrated in the example code above, a message handler has four
  components: the <with|font-shape|italic|type> of message<subindex|message
  type|of handler> it handles, an <with|font-shape|italic|interposer><index|interposer>
  which will be explained later, the <with|font-shape|italic|verifier><index|verifier>
  deciding if a message is well-formed and the
  <with|font-shape|italic|handler procedure><index|handler procedure>.

  The verifier is passed a bytevector slice with the message and should
  return <scm|#true> if the message is well-formed and <scm|#false> if it
  isn't.<space|1em>It may assume that the length of the slice corresponds to
  the length <em|in> the message header and is at least the length <em|of>
  the message header and that the type in the message header corresponds to
  the type of the message handler.<space|1em>Messages will only be passed to
  the handler procedue if the verifiers returns <scm|#true>.

  The handler procedure is passed a bytevector slice with the message, but
  only if the verifier considers it well-formed.<space|1em>The handler
  procedure and verifier are run from the
  <with|font-shape|italic|interposer>.<space|1em>The interposer is passed a
  thunk to execute and may e.g. install exception handlers and parameterise
  parameters.<space|1em>It can change the current input, output and error
  ports for example.

  <todo|document the message type database, various procedures>

  <section|Message type database><label|sec:message type><subindex|message
  type|database>

  The module <scm|(gnu gnunet message protocols)><index|(gnu gnunet message
  protocols)> has a mapping of symbolic names of every message type known to
  scheme-GNUnet to their numeric value.<space|1em>To use it, the macro
  <scm|symbol-value><index|symbol-value> from <scm|(gnu extractor
  enum)><index|(gnu extractor enum)> is required and possibly
  <scm|value-\<gtr\>index><index|value-\<gtr\>index> as well.<space|1em>To
  determine the numeric value of the message type <scm|msg:nse:estimate>, one
  would write:

  <\scm-code>
    (define numeric-type

    \ \ (value-\<gtr\>index (symbol-value message-type msg:nse:estimate)))
  </scm-code>

  <todo|other various enum procedures for introspection, documentation,
  <text-dots>?>

  <todo|how to define new message types>

  <section|Sending messages>

  Messages can be sent with the <scm|send-message!><index|send-message!>
  procedure, which can be called as <scm|(send-message! <var|mq>
  <var|message> #:priority <var|priority>)>, where <var|mq> is the message
  queue and <var|message> is the message to send as a readable bytevector
  slice. This is an asynchronuous operation, so this procedure can return
  before the service has processed the message.

  <label|mq-prio-prefs>Depending on the transport, it might be possible for
  messages to be lost or received out-of-order. Some transports allow to
  explicitely allow messages to be lost or received out-of-order and would by
  default retransmit lost messages and reorder out-of-order messages; this
  behaviour can to a degree be controlled by setting the
  <dfn|priority-preference> flags.

  These flags are not absolute, e.g. even if reliable transmission is
  requested, it is possible that the transport fail to transmit the message.
  The exact behaviour is transport-dependent!

  <\description>
    <item*|<scm|pref:unreliable>>Unreliable delivery is acceptable.

    <item*|<scm|pref:low-latency>>Low latency is desired, this cannot be
    meaningfully combined with <scm|pref:cork-allowed>.

    <item*|<scm|pref:cork-allowed>>The transmission of a message can be
    delayed to combine this message with other messages into a larger
    transmission with less per-message overhead.

    <item*|<scm|pref:good-throughput>>High bandwith is desired; the method
    chosen for transmission should focus on overall throughput.

    <item*|<scm|pref:out-of-order>>Out-of-order delivery is acceptable.
  </description>

  These flags can be combined into a numeric value with the macro
  <scm|prio-prefs> from <scm|(gnu gnunet mq prio-prefs)>; the following code
  defines <var|x> as the numeric value of the flags <scm|pref:unreliable> and
  <scm|pref:out-of-order>:

  <\scm>
    (import (gnu gnunet mq prio-prefs))

    (define x (prio-prefs pref:unreliable pref:out-of-order))
  </scm>

  This numeric priority-preference can be passsed to <scm|send-message!> as
  the optional <var|priority> keyword argument of <scm|send-message!>. The
  transport of <scm|connect/fibers> is always reliable and in-order.
  <todo|notify-sent! callbacks><todo|cancellation><todo|queue size limits,
  <scm|%suspicious-length>>

  <section|Error handler><index|error handler>

  The message queue implementation usually just sends and receives messages,
  but some exceptional situations cannot be communicated with
  <scm|send-message!> or <scm|inject-message!>.<space|1em>For those, there is
  the <scm|inject-error!><index|inject-error!> procedure.<space|1em>This
  variadic procedure accepts a message queue to inject the error into, a
  <with|font-shape|italic|key><index|key> (usually a symbol) describing the
  exceptional situation and rest arguments.<space|1em>It calls the
  <with|font-shape|italic|error handler> of the message queue with the key
  and rest arguments.<space|1em>The following errors can currently be
  reported by the built-in message queue implementations:

  <\explain>
    <scm|connection:connected><index|connection:connected>
  <|explain>
    The connection to the server has been established.
  </explain>

  <\explain>
    <scm|connection:interrupted><index|connection:interrupted>
  </explain|The message queue has been closed before the connection to the
  server could be established.>

  <\explain>
    <scm|input:regular-end-of-file><index|input:regular-end-of-file>
  <|explain>
    The connection has been closed by the server.

    For people wondering about what happens if a connection becomes
    half-duplex: GNUnet does not have a notion of half-duplex message
    streams.<space|1em>If it is detected the underlying stream became
    half-duplex anyways, it will be treated as closed by scheme-GNUnet,
    resulting in this error.<space|1em>However, note that currently broken
    pipes cannot be reliably detected.
  </explain>

  <\explain>
    <scm|input:premature-end-of-file><index|input:premature-end-of-file>
  </explain|The connection was closed by the server while a message was still
  being read.<space|1em>This can happen if the server was stopped while it
  was still sending the rest of the message.>

  <\explain>
    <scm|input:overly-small> <var|type> <var|size><index|input:overly-small>
  </explain|The message size in the header was smaller than the minimal
  message size.<space|1em>Sometimes, but not always, the message type
  <var|type> and message size <var|size> are available (as exact
  naturals).<space|1em>When they are not available, <var|type> and <var|size>
  are <scm|#false> instead.<space|1em>This can only happen if the server or
  connection to the server is buggy.>

  <\explain>
    <scm|logic:no-handler> <var|type> . <var|rest><index|logic:no-handler>
  <|explain>
    The received message of type <var|type> (as an integer) does not have a
    corresponding message handler.<space|1em><var|rest> is currently
    unspecified.
  </explain>

  <\explain>
    <scm|logic:ill-formed> <var|type> . <var|rest><index|logic:ill-formed>
  </explain|The received message of type (as an integer) is ill-formed
  according to the message handler.<space|1em><var|rest> is currently
  unspecified.>

  Consider automatically reconnecting after
  <scm|<scm|input:regular-end-of-file>> and
  <scm|<scm|input:premature-end-of-file>>, to allow the server to restart
  without having to manually restart every individual
  application.<space|1em>To report errors, see the section
  <reference|sec:error reporting> Error reporting.

  <section|Ordering of injected errors and messages and sent messages>

  This section describes how injected errors and messages and sent messages
  are ordered with respect to each other in the default message queue
  implementation.<space|1em>Messages are handled or corresponding
  <scm|logic:no-handler> or <scm|logic:ill-formed> errors are injected in the
  order that the messages are received.<space|1em>Before messages are read,
  <scm|connection:connected> is injected.<space|1em>This error is injected at
  most once.

  <em|Soon> after all messages are read (and therefore
  <with|font-shape|italic|soon> after all handled messages or corresponding
  errors), the error <scm|input:regular-end-of-file>,
  <scm|input:overly-small> or <scm|input:premature-end-of-file> is
  injected.<space|1em>Only one of those errors can be injected for the entire
  lifetime of the message queue.

  Be aware that <em|soon> is not <em|immediate> here!<space|1em>For example,
  it is possible for a message to be received, the port closed, a message
  queued for sending, the closing of the port being detected by the write
  fiber, <scm|input:regular-end-of-file> being injected from the write fiber
  and the read fiber handling the received message, and the read fiber
  exiting because the port is closed, in that order.

  Messages are sent (and received on the other side) in the order they were
  enqueued for sending.<space|1em>Likewise, the notify-sent callback of
  enqueued messages are called in order.<space|1em>If the notify-sent
  callback is called, it is before the message is received by the other
  side.<space|1em>The message and its notify-sent callback are only received
  by the other side and called after the message has been injected and
  <scm|connection:connected> has been injected.<space|1em>It is possible for
  the notify-sent callback to be called without the message being received by
  the other side, e.g. if the port was closed during the notify-sent
  callback.

  If a message is received by the other side, all previously-sent messages
  have be received before.<space|1em>If a notify-sent callback is invoked,
  all notify-sent callbacks of previous messages have been invoked before,
  except the messages that are eventually cancelled.

  The errors <scm|logic:no-handler> and <scm|logic:ill-formed> are not fatal:
  later messages can still be read and handled.<space|1em>If
  <scm|connection:interrupted> is injected, no other errors are ever
  injected, whether in the past or in the future.<space|1em>This error can
  only be injected once.

  <todo|I/O errors>

  <todo|envelopes>

  <section|Disconnecting><index|disconnecting>

  A message queue can be closed with the <scm|close-queue!><index|close-queue!>
  procedure from <scm|(gnu gnunet mq)>.<space|1em>In the default message
  queue implementation, this asynchronuously closes the port and stops
  associated fibers.<space|1em>Closing ports when they won't be used anymore
  is important for limiting resource consumption, especially for servers that
  can have many connections.<space|1em>Closing message queues is an
  idempotent operation: closing a message queue twice is the same as closing
  it once.<space|1em> If a message queue is closed before a connection could
  be formed, <scm|connection:interrupted><index|connection:interrupted> is
  injected instead of <scm|connection:connected> and
  <scm|connection:regular-end-of-file>.

  <section|Error reporting><label|sec:error reporting>

  <index|error reporting>Errors can be reported with the procedure
  <scm|report-error> from the module <scm|(gnu gnunet mq
  error-reporting)><index|(gnu gnunet mq error-reporting)>.<space|1em>It can
  be called as <scm|(report-error key argument ...)>, e.g. <scm|(report-error
  'logic:no-handler 3)><index|report-error>.<space|1em>By default, it reports
  the error to the current error port.<space|1em>If this is not desired, the
  output can be sent to another port by setting the parameter
  <scm|textual-error-reporting-port><index|textual-error-reporting-port>.<space|1em>If
  textual error reporting is not desired, the parameter
  <scm|error-reporter><index|error-reporter> can be set to a procedure with
  the same interface as <scm|report-error>.<space|1em>Such a procedure could
  e.g. open a GUI dialog, sent the message to the system logger or ignore the
  error.

  Error messages are translated for the current locale.<todo|TODO actually
  call bindtextdomain>
</body>

<\initial>
  <\collection>
    <associate|page-medium|paper>
    <associate|save-aux|false>
  </collection>
</initial>