netdisco/lib/App/Netdisco/Manual/WritingWorkers.pod

=head1 NAME

App::Netdisco::Manual::WritingWorkers - Developer Documentation on Worker Plugins

=head1 Introduction

L<App::Netdisco>'s plugin system allows users to write I<workers> to gather
information from network devices using different I<transports> and store
results in the database.

For example, transports might be SNMP, SSH, or HTTPS. Workers might be
combining those transports with application protocols such as SNMP, NETCONF
(OpenConfig with XML), RESTCONF (OpenConfig with JSON), eAPI, or even CLI
scraping. The combination of transport and protocol is known as a I<driver>.

Workers can be restricted to certain vendor platforms using familiar ACL
syntax. They are also attached to specific actions in Netdisco's backend
operation (discover, macsuck, etc).

See L<App::Netdisco::Worker::Plugin> for more information about worker
plugins.

=head1 Developing Workers

A worker is Perl code which is run. Therefore it can do anything you like, but
typically it will make a connection to a device, gather some data, and store
it in Netdisco's database.

App::Netdisco plugins must load the L<App::Netdisco::Worker::Plugin> module.
This exports a helper subroutine to register the worker. Here's the
boilerplate code for our example plugin module:

 package App::Netdisco::Worker::Plugin::Discover::Wireless::UniFi;
 
 use Dancer ':syntax';
 use App::Netdisco::Worker::Plugin;
 use aliased 'App::Netdisco::Worker::Status';
 
 # worker registration code goes here, ** see below **
 
 true;

=head1 Registering a Worker

Use the C<register_worker> helper from L<App::Netdisco::Worker::Plugin> to
register a worker:

 register_worker( $coderef );
 # or
 register_worker( \%workerconf, $coderef );

For example (using the second form):

 register_worker({
   driver => 'unifiapi',
 }, sub { "worker code here" });

The C<%workerconf> hashref is optional, and described below. The C<$coderef>
is the main body of your worker. Your worker is run in a L<Try::Tiny>
statement to catch errors, and passed the following arguments:

 $coderef->($job, \%workerconf);

The C<$job> is an instance of L<App::Netdisco::Backend::Job>. Note that this
class has a C<device> slot which may be filled, depending on the action, and
if the device is not yet discovered then the row will not yet be in storage.
The C<\%workerconf> hashref is the set of configuration parameters you used
to declare the worker (documented below).

=head2 Package Naming Convention

The package name used where the worker is declared is significant. Let's look
at the boilerplate example again:

 package App::Netdisco::Worker::Plugin::Discover::Wireless::UniFi;

The package name B<must> contain C<Plugin::> and the namespace component after
that becomes the action. For example workers registered in the above package
will be run during the I<discover> backend action (that is, during a
C<discover> job). You can replace C<Discover> with other actions such as
C<Macsuck>, C<Arpnip>, C<Expire>, and C<Nbtstat>, or create your own.

The component after the action is known as the I<phase> (C<Wireless> in this
example), and is the way to override a Netdisco built-in worker, by using the
same name (plus an entry in C<%workerconf>, see below). Otherwise you can use
any valid Perl bareword for the phase.

Workers may also be registered directly to the action (C<Discover>, in this
example), without any phase. This is used for very early bootstrapping code
(such as first inserting a device into the database so it can be used by
subsequent phases) or for very simple, generic actions (such as C<netdisco-do
psql>).

=head2 C<%workerconf> Options

=over 4

=item ACL Options

Workers may have C<only> and C<no> parameters configured which use the
standard ACL syntax described in L<the settings
guide|App::Netdisco::Manual::Configuration>. The C<only> directive is
especially useful as it can restrict a worker to a given device platform or
operating system (for example Cisco IOS XR for the C<restconf> driver).

=item C<driver> (string)

The driver is a label associated with a group of workers and typically refers
to the combination of transport and application protocol. Examples include
C<snmp>, C<netconf>, C<restconf>, C<eapi>, and C<cli>. The convention is for
driver names to be lowercase.

Users will bind authentication configuration settings to drivers in their
configuration. If no driver is specified when registering a worker, it will be
run for every device and phase (such as during Expire jobs).

=item C<primary> (boolean)

When multiple workers are registered for the same phase, they will all be run.
However there is a special "I<primary>" slot for each phase in which only one
worker (the first that succeeds) is used. Most of Netdisco's built-in worker
code is registered in this way, so to override it you can use the same package
namespace and set C<primary> to be C<true>.

=back

=head1 Worker Execution and Return Code

Workers are configured as an ordered list. They are grouped by C<action> and
C<phase> (as in Package Naming Convention, above).

Workers defined in C<extra_worker_plugins> are run before those in
C<worker_plugins> so you have an opportunity to override built-in workers by
adding them to C<extra_worker_plugins> and setting C<primary> to C<true> in
the worker configuration.

The return code of the worker is significant for those configured with
C<primary> as C<true>: when the worker returns true, no other C<primary> hooks
are run for that phase. You should always use the aliased
L<App::Netdisco::Worker::Status> helper (loaded as in the boilerplate code
above) when returning a value, such as:

 return Status->done('everything is good');
 # or
 return Status->error('something went wrong');
 # or
 return Status->defer('this device cannot be processed right now');

Remember that a worker is only run if it matches the hardware platform of the
target device and the user's configuration, and is not also excluded by the
user's configuration. This filtering takes place before inspecting C<primary>.

=head1 Accessing Transports

From your worker you will want to connect to a device to gather data. This is
done using a transport protocol session (SNMP, SSH, etc). Transports are
singleton objects instantiated on demand, so they can be shared among a set of
workers that are accessing the same device.

See the documentation for each transport to find out how to access it:

=over 4

=item *

L<App::Netdisco::Transport::SNMP>

=back

=head1 Database Connections

The Netdisco database is available via the C<netdisco> schema key, as below.
You can also use the C<external_databases> configuration item to set up
connections to other databases.

 # plugin package
 use Dancer::Plugin::DBIC;
 my $set =
   schema('netdisco')->resultset('Devices')
                     ->search({vendor => 'cisco'});

=head1 Review of Terminology

In summary, Worker code is defined in a package namespace specifying the
Action and Phase, and registered as a plugin with configuration which may
specify the Driver and whether it is in the Primary slot. Access Control Lists
determine which Workers are permitted to run, and when. Here are more complete
definitions:

=over 4

=item C<action>

The highest level grouping of workers, corresponding to a Netdisco command
such as C<discover> or C<macsuck>. Workers can be registered at this level to
do really early bootstrapping work.

=item C<phase>

The next level down from C<action> for grouping workers. Phases have arbitrary
names and are visited in the order defined in the C<extra_worker_plugins>
setting list, followed by the C<worker_plugins> setting list. Workers are
usually registered at this level.

=item C<worker>

A lump of code you write which does a single clearly defined task. The package
namespace of the worker identifies the action and optionally the phase.
Workers are typically registered with some configuration settings.

=item C<driver>

A label associated with a group of workers which refers to a combination of
transport and application protocol used to connect to and communicate with the
target device. Users attach authentication configuration to specific drivers.

=item C<primary> (defaults to C<false>)

Indicates that the worker will only be run if no other C<primary> worker for
this phase has already succeeded. In this way, you can override Netdisco code
by setting this option and returning true from your worker.

=back

=cut