== What's in the HAl instantiation branch

=== quick summary

- a backwards-compatible extension of the C HAL API and halcmd and to support
 creation and deletion of components at run time
- instcomp, an extended version of the comp component generator to
 support this API
- several migrated and new components
- demo components for the new features

=== in a nutshell - what can I do with it:

==== before you had to know in advance how many copies of a comp you need:

 loadrt or2 count=2

and that was it, you need one more - go back to the loadrt statement
and fix it, wherever that was

==== after: create instances as needed

(for the lutn component, see the man page for lut5 - it's just a more
flexible version of lut5: variable number of pins, and instantiable -
note each instance can have different functions and number of pins):

 # create 2 instances with 2 inputs each and the or function
 # wherever needed
 # syntax:
 # newinst <compname> <instance name> <per-instance parameters...>
 newinst lutn or2.0  inputs=2 function=0xe
 newinst lutn or2.1  inputs=2 function=0xe

 # the functs exported are named after the instance name 
 addf or2.0 <threadname>
 addf or2.1 <threadname>

 # NB: the 'loadrt lutn' command will happen automatically behind the
 # scenes if not yet loaded
 # and if you do not need it any more, destroy the instance

 delinst or2.1

 # poof, or2.1 is gone!

or2.0 will continue to work as before.

== API change: thread functions are now int, not void

Thread functions in instcomp-generated components *must return 0* for now; so far they were defined 'void' and did not need to return a value.

This change has been made to enable future functionality. A return value is a way to signal the calling layer about success/failure, and there are good use case for this: in particular error signaling from the funct to the calling thread, and more mundane features generic statistics accumulation in the calling layer (per-thread). While this is currently not used, we decided to make the switch now because it is a large change anyway and this one is minor in relation.

All hal/i_components have been modified to adhere to this new convention.


== Other user-visible changes

- there is a new named HAL object, called "instance", visible at the
  halcmd ("show inst"), HAL C API, and HAL Cython API levels.
- An instance is owned by a component, and has a name and a
  size. Technically it is a blob of memory with a name, attached to a
  component.
- instances are relevant only for RT components. User components have
  no use for the concept, and hence do not support it.
- components, as well as instances may own pins, params and functs.
- legacy components work unchanged as before and do not use instances.
- newly written components, modified components (to use the instance
  API) and components processed by Mick's 'instcomp' will support instances.
- instances, as well as normal components, can have parameters.
- instance and component parameters are distinct and cannot be mixed.
- component parameters are applied at 'loadrt <compname> <params>' as before.
- to create an instance of a component, use the halcmd 'newinst'
  command (syntax: 'newinst <compname> <unique instance name> [any
  instance paramters])
- instance parameters apply at instantiation time, and only for that
  particular instance, other than
  component parameters, which apply at loadrt time.
- to delete an instance of a component, use the halcmd 'delinst'
  command.
- deleting an instance of a component will leave other instances untouched.
- exiting a component will destroy all instances the component has,
  including all pins, params and functs.
- instance names MUST be unique.
- Usage of pins, params, functs is no different between legacy
  components and instances.


legacy vs instantiable: cd field in show comp

=== Overview of changes relevant for developers

- Previously pins, params and functs were owned by components. The API
functions hence had a 'int comp_id' parameter.
- Now, pins, params and functs can be owned EITHER by a component, OR
by an instance (which in turn is always owned by a component).
- All C API signatures have been changed to reflect this fact: where
formerly there was a 'comp_id' parameter, it is now called an
'owner_id', which can refer EITHER to a component or an instance.
- A legacy component so far was initialized by hal_init("compname").
- An instantiable component is initialized by the new hal_xinit() call,
which takes several parameters besides the name. To be instantiable, a
component MUST export a constructor function. It MAY export a
destructor function. A component which uses hal_xinit() but does not
export a constructor (NULL) behaves like a legacy component.
- Coding an instantiable component ("icomp") must follow a certain protocol: see
the examples under src/hal/icomp-example to get the idea.
- Once an icomp is instantiated with "newinst", its constructor
function will be called. The constructor is expected to check the
instance parameters, and once valide, create the named instance with
hal_inst_create().
- The constructor is also responsible for initializing any pins,
params and functs which are owned by this instance. The pin+param
memory must lie in the blob of memory returned by hal_inst_create().
- component parameters are declared as before with the RTAPI_MP_*
macros.
- instance parameters are declare with the new RTAPI_IP_INT and
RTAPI_MP_STRING macros. Other types as well as arrays are currently not supported (and
unlikely to be needed).

=== instantiable component example

Some examples can be found in hal/icomp-example:

- lutn.c is an actually useful instantiable variant of lut5.c
- icomp.c shows all variations of the instantiation API, not all of
   which might be needed for every component

=== HAL userfunct objects

userfuncts are a similar to normal functs, but cannot be addf'd to a thread.

So what they are good for then? well, they can be called from userland via halcmd, or the C or Cython RTAPI API's.

userfuncts are the basic feature on which the whole HAL instantiation API rests.
hal_lib is now a component and exports internal userfuncts (newinst and delinst)
to support the instantiation API.


ufdemo.c shows the API usage, as well as the extended thread API.

Show time!

 $ realtime start
 $ halcmd -f -k

 halcmd: loadrt ufdemo

 <stdin>:1: Realtime module 'ufdemo' loaded

 halcmd: show comp

 Loaded HAL Components:
     ID  Type Flags Inst Name                                      PID   State
  32770  RT            0 ufdemo                                    RT    ready, u1:0 u2:0
     71  User          0 halcmd1774                                1774  ready, u1:0 u2:0
     70  uHAL   i      0 hal_lib1774                               1774  ready, u1:0 u2:0
  32769  rHAL   i      0 hal_lib                                   0     ready, u1:0 u2:0

 halcmd: show funct

 Exported Functions:
   Comp   Inst CodeAddr  Arg       FP   Users Type    Name
  32769        b6e5fe6b  00000000  NO       0 user    delinst
  32769        b6e5fc13  00000000  NO       0 user    newinst
  32770        b753d83c  b753db7b  NO       0 user    ufdemo.demo-funct
  32770        b753d82d  b753db36  NO       0 thread  ufdemo.legacy-funct
  32770        b753d832  b753db5a  NO       0 xthread ufdemo.xthread-funct

 halcmd: call ufdemo.demo-funct foo bar baz
 <stdin>:4: function 'ufdemo.demo-funct' returned 3
 halcmd:

relevant part of linuxcnc.log:

 Mar 12 21:25:28 nwheezy msgd:0: hal_lib:1768:rt RTAPI: initing module HAL_ufdemo
 Mar 12 21:25:28 nwheezy msgd:0: hal_lib:1768:rt RTAPI: module 'HAL_ufdemo' loaded, ID: 32770
 Mar 12 21:25:28 nwheezy msgd:0: hal_lib:1768:rt hal_xinit:246 HAL: legacy component 'ufdemo' id=32770 initialized
 Mar 12 21:25:28 nwheezy msgd:0: hal_lib:1768:rt hal_export_xfunctfv:70 HAL: exporting function 'ufdemo.legacy-funct' type 0
 Mar 12 21:25:28 nwheezy msgd:0: hal_lib:1768:rt hal_pin_new:121 HAL: creating pin 'ufdemo.legacy-funct.time'
 Mar 12 21:25:28 nwheezy msgd:0: hal_lib:1768:rt hal_param_new:135 HAL: creating parameter 'ufdemo.legacy-funct.tmax'
 Mar 12 21:25:28 nwheezy msgd:0: hal_lib:1768:rt hal_param_new:135 HAL: creating parameter  'ufdemo.legacy-funct.tmax-increased' 
 Mar 12 21:25:28 nwheezy msgd:0: hal_lib:1768:rt hal_export_xfunctfv:70 HAL: exporting function 'ufdemo.xthread-funct' type 1
 Mar 12 21:25:28 nwheezy msgd:0: hal_lib:1768:rt hal_pin_new:121 HAL: creating pin 'ufdemo.xthread-funct.time'
 Mar 12 21:25:28 nwheezy msgd:0: hal_lib:1768:rt hal_param_new:135 HAL: creating parameter 'ufdemo.xthread-funct.tmax'
 Mar 12 21:25:28 nwheezy msgd:0: hal_lib:1768:rt hal_param_new:135 HAL: creating parameter 'ufdemo.xthread-funct.tmax-increased'
 Mar 12 21:25:28 nwheezy msgd:0: hal_lib:1768:rt hal_export_xfunctfv:70 HAL: exporting function 'ufdemo.demo-funct' type 2
 Mar 12 21:25:28 nwheezy msgd:0: rtapi_app:1768:user ufdemo: loaded from ufdemo.so
 Mar 12 21:25:53 nwheezy msgd:0: hal_lib:1768:rt ufdemo: userfunct 'ufdemo.demo-funct' called, arg='u-instance-data' argc=3
 Mar 12 21:25:53 nwheezy msgd:0: hal_lib:1768:rt     argv[0] = "foo"
 Mar 12 21:25:53 nwheezy msgd:0: hal_lib:1768:rt     argv[1] = "bar"
 Mar 12 21:25:53 nwheezy msgd:0: hal_lib:1768:rt     argv[2] = "baz"

=== thread API extensions

This branch also contains work on a generalized thread API, which was
required for the userfunct feature to work.

The problems with the old API were:

- very limited exposure of runtime data (e.g. actual invocation times
  are already computed in the calling code, but not exposed to thread
  functions except the rather useless period setup parameter)
- support for userfuncts was needed

For an overview of the new thread API, see src/hal/hal_priv.h .

Legacy code using the hal_export_funct API will continue to run unchanged.


== Easier Debugging of RT, msgd, haltalk, webtalk: NOSIGHDLR


Due to robust signal handling in these demons, it is hard to attach
these processes with gdb and debug. Also, on a crash the generated
core is sometimes not usable due to reasons which I do not
understand yet.

This branch introduces an environment variable
NOSIGHDLR which - if set - will disable signal trapping in these
demons.

This makes attaching with gdb -p <process-id>, as well as
detaching much simpler, and the processes do not exit on
detach.


The actual value of NOSIGHDLR is irrelevant.  There are
corresponding command line options '--nosighdlr/-G' which achieve
the same effect on a per-process basis.

Note this means default   signal handling, that is, the processes will exit without any
further log message, and maybe create a core.

To debug rtapi or msgd, it's best to use the posix flavor, which will not complain
about RT violations due to the delays caused by debugging. it does
not matter for haltalk and webtalk because these are RT-unaware.

Initiating a debuggable RT session hence amounts to:

  $ export DEBUG=5 FLAVOR=posix NOSIGHDLR=1
  $ realtime start # or linuxcnc...

To debug rtapi, use 

  $ sudo gdb -p `pidof rtapi:0`

To debug msgd,  use 

  $ sudo gdb -p `pidof msgd:0`

Debugging RT thread functions now works perfectly fine in userland flavors.
