Files
netdisco/lib/App/Netdisco/Worker/Plugin.pm
Oliver Gorwits 826e1db39d API Endpoints to submit arpnip and macsuck results (#942)
* Add macsuck worker to collect various PortAccessEntity (NAC) attributes

* Incorporate PAE feedback on #937

 * missing Result/Device.pm column added
 * pae_is... columns instead of pae_capabilities
 * moved most code to Util/PortAccessEntity.pm so the update can
   be done in discover and macsuck

* Refactor PAE attributes during discover as separate Plugin

* PortAccessEntity: don't use device->dns in log string

* Fix "Experimental keys on scalar is now forbidden" test failure

* Revamp pae_control and add missing attribute

 - device.pae_control (text) is now device.pae_is_enabled (bool)
 - also store pae_authconfig_port_control (port mode auto/force(un)Auth)

* Fix "Experimental keys on scalar is now forbidden" test failure

 - ... again because of botched merge
 - at least perlgolfed away a set of curly braces

* Update PortAccessEntity.pm

* Incorporate @ollyg PR feedback

* allow actions without transport to run when there are also no creds

* initial refactor for separate gather, process, store phases for macsuck

* factor out the vlan sanity check

* additional help with log of action workers

* cleanup logic in check macsuck

* refactor to make main phases only

* some fixes

* implement file slurp. amazingly the whole thing works

* remove outdated noop from test

* treat error as critical, use cancel to suppress further drivers

* big refactor to share mac sanity code to both paths

* fix inverted logic on vlan sanity filter

* some code tidy

* fix error in default value

* fix for vlan 0 nodes input from cli

* ensure imported MACs are IEEE format

* add api endpoint, no useful return status yet

* exit status if error from nodes PUT

* suppress other networked workers when direct workers are active

* better log showing worker

* fix status recording to get first error or last done message

* implement arpnip API PUT

* avoid package redeclaration error

* make sure write API methods require admin status

* add doc for passing JSON data to arpnip and macsuck

* update manifest

* remove option to do jobs in web handler; all by queue now

* use job entry timestamp for offline queued jobs

* fix store username and IP on api PUT

* never de-duplicate user-submitted jobs; never reset DeviceSkip for offline jobs

* myworker no longer needed

* make logic cleaner

Co-authored-by: Christian Ramseyer <ramseyer@netnea.com>
2022-11-25 15:24:23 +00:00

152 lines
5.3 KiB
Perl

package App::Netdisco::Worker::Plugin;
use Dancer ':syntax';
use Dancer::Plugin;
use App::Netdisco::Util::Permission qw/check_acl_no check_acl_only/;
use aliased 'App::Netdisco::Worker::Status';
use Scope::Guard 'guard';
use Storable 'dclone';
register 'register_worker' => sub {
my ($self, $first, $second) = plugin_args(@_);
my $workerconf = (ref $first eq 'HASH' ? $first : {});
my $code = (ref $first eq 'CODE' ? $first : $second);
return error "bad param to register_worker"
unless ((ref sub {} eq ref $code) and (ref {} eq ref $workerconf));
my $package = (caller)[0];
if ($package =~ m/Plugin::(\w+)(?:::(\w+))?/) {
$workerconf->{action} = lc($1);
$workerconf->{namespace} = lc($2) if $2;
}
return error "failed to parse action in '$package'"
unless $workerconf->{action};
( $workerconf->{title} ||= lc($package) ) =~ s/.+plugin:://;
$workerconf->{phase} ||= 'user';
$workerconf->{namespace} ||= '_base_';
$workerconf->{priority} ||= (exists $workerconf->{driver}
? (setting('driver_priority')->{$workerconf->{driver}} || 0) : 0);
my $worker = sub {
my $job = shift or die 'missing job param';
# use DDP; p $workerconf;
debug sprintf '-> run worker %s/%s "%s"',
@$workerconf{qw/phase priority title/};
if ($job->is_cancelled) {
return $job->add_status( Status->info('skip: job is cancelled') );
}
if ($job->is_offline
and $workerconf->{phase} eq 'main'
and $workerconf->{priority} > 0
and $workerconf->{priority} < setting('driver_priority')->{'direct'}) {
return $job->add_status( Status->info('skip: networked worker but job is running offline') );
}
# check to see if this namespace has already passed at higher priority
# and also update job's record of namespace and priority
return $job->add_status( Status->info('skip: namespace passed at higher priority') )
if $job->namespace_passed($workerconf);
# support part-actions via action::namespace
if ($job->only_namespace and $workerconf->{phase} ne 'check') {
# skip namespaces not the requested ::namespace
return unless $workerconf->{namespace} eq lc( $job->only_namespace )
# apart from discover::properties which needs to run, so that's early
# phase for unknown devices, but not ::hooks/early (if implemented)
or (($job->only_namespace ne 'hooks') and ($workerconf->{phase} eq 'early')
and ($job->device and not $job->device->in_storage));
}
my @newuserconf = ();
my @userconf = @{ dclone (setting('device_auth') || []) };
# worker might be vendor/platform specific
if (ref $job->device) {
my $no = (exists $workerconf->{no} ? $workerconf->{no} : undef);
my $only = (exists $workerconf->{only} ? $workerconf->{only} : undef);
return $job->add_status( Status->info('skip: acls restricted') )
if ($no and check_acl_no($job->device, $no))
or ($only and not check_acl_only($job->device, $only));
# reduce device_auth by driver and action filters
foreach my $stanza (@userconf) {
next if exists $stanza->{driver} and exists $workerconf->{driver}
and (($stanza->{driver} || '') ne ($workerconf->{driver} || ''));
# filter here rather than in Runner as runner does not know namespace
next if exists $stanza->{action}
and not _find_matchaction($workerconf, lc($stanza->{action}));
push @newuserconf, dclone $stanza;
}
# per-device action but no device creds available
return $job->add_status( Status->info('skip: driver or action not applicable') )
if 0 == scalar @newuserconf
and $workerconf->{priority} > 0
and $workerconf->{priority} < setting('driver_priority')->{'direct'};
}
# back up and restore device_auth
my $guard = guard { set(device_auth => \@userconf) };
set(device_auth => \@newuserconf);
# use DDP; p @newuserconf;
# run worker
$code->($job, $workerconf);
};
# store the built worker as Worker.pm will build the dispatch order later on
push @{ vars->{'workers'}->{$workerconf->{action}}
->{$workerconf->{phase}}
->{$workerconf->{namespace}}
->{$workerconf->{priority}} }, $worker;
};
sub _find_matchaction {
my ($conf, $action) = @_;
return true if !defined $action;
$action = [$action] if ref [] ne ref $action;
foreach my $f (@$action) {
return true if
$f eq $conf->{action} or $f eq "$conf->{action}::$conf->{namespace}";
}
return false;
}
register_plugin;
true;
=head1 NAME
App::Netdisco::Worker::Plugin - Netdisco Workers
=head1 Introduction
L<App::Netdisco>'s plugin system allows users to write I<workers> to gather
information from network devices using different I<transports> and store
results in the database.
For example, transports might be SNMP, SSH, or HTTPS. Workers might be
combining those transports with application protocols such as SNMP, NETCONF
(OpenConfig with XML), RESTCONF (OpenConfig with JSON), eAPI, or even CLI
scraping. The combination of transport and protocol is known as a I<driver>.
Workers can be restricted to certain vendor platforms using familiar ACL
syntax. They are also attached to specific actions in Netdisco's backend
operation (discover, macsuck, etc).
See L<https://github.com/netdisco/netdisco/wiki/Backend-Plugins> for details.
=cut