Merge the backend worker plugins branch og-coreplugins

Squashed commit of the following:

commit 86d0f61d0b
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Thu Nov 16 22:26:32 2017 +0000

    fix typo

commit 5aff19621c
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Thu Nov 16 22:10:18 2017 +0000

    fix use of snmp_connect_ip which does not work for SNMPv3

commit 68a56d35bb
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Thu Nov 16 20:50:16 2017 +0000

    no need for Array::Iterator even though it was cute

commit 71ee869c02
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Nov 15 22:14:47 2017 +0000

    additional doc examples

commit 620b3fe544
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Nov 15 22:09:05 2017 +0000

    stash workers within poller instance, and load plugins explicitly

commit 2431365583
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Nov 13 22:17:11 2017 +0000

    better fix for duplicate module entity index

commit a400b26704
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Nov 13 22:14:42 2017 +0000

    add ignore interfaces for HPE routers

commit 1502ec1966
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Nov 13 22:08:02 2017 +0000

    bug fixes after testing on a real network

commit 840b6b4069
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Nov 12 20:38:35 2017 +0000

    add tests

commit 2de36c69ba
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Nov 12 00:14:21 2017 +0000

    some reengineering to support proper testing

commit c5f138fe62
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Nov 11 14:43:53 2017 +0000

    correct algorithm on finalise status, correct logging

commit 98442a2308
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Thu Nov 9 21:49:45 2017 +0000

    bug fixes

commit e0c6615c87
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Nov 8 20:29:33 2017 +0000

    fix bugs

commit 1eeaba441d
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Nov 7 22:30:55 2017 +0000

    finish refactor to new desired behaviour (buggy?)

commit 7edfe88f25
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Nov 6 22:50:51 2017 +0000

    fix to work, and correct namespace check

commit 25907d3544
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Nov 6 21:26:01 2017 +0000

    move status tracking and checking inside job instance

commit 4436150bf4
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Nov 5 20:54:28 2017 +0000

    remove global rubbish

commit 28b016e713
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Nov 4 23:31:51 2017 +0000

    fix docs

commit 650f6c719b
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Nov 4 23:22:12 2017 +0000

    tidy line

commit 10f78d5dbe
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Nov 4 23:06:20 2017 +0000

    add priority and namespace to support fancy worker overrides

commit b9f9816d09
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Oct 11 18:33:46 2017 +0100

    release 2.036012_001

commit c33bf204a4
Merge: 5b7ce3f7 d3d81eb6
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Oct 11 18:30:23 2017 +0100

    Merge branch 'master' into og-coreplugins

commit 5b7ce3f797
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Oct 9 15:46:09 2017 +0100

    cannot Sereal::Encode DBIC row

commit 0a575f02ba
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Oct 9 14:07:56 2017 +0100

    fix bug in job->device init

commit 207476950d
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Oct 9 14:03:37 2017 +0100

    default causes no attr to be created?!

commit 912f2fa91f
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Oct 8 18:43:51 2017 +0100

    better debug logging

commit dfeb9d9ddc
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Oct 8 18:40:02 2017 +0100

    make device_auth have driver setting for snmp entries

commit 460c0c0ee9
Merge: 3ccd107b 98423445
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Oct 8 18:08:58 2017 +0100

    Merge branch 'master' into og-coreplugins

commit 3ccd107bd4
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Oct 7 14:13:58 2017 +0100

    fix bug in device->has_layer

commit a4b9bf2036
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Oct 7 13:58:52 2017 +0100

    netdisco-do show takes a param for method in -p

commit 4389cd0459
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Oct 7 13:36:06 2017 +0100

    fix to only check last poll on devices in storage

commit 58d0fbddda
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Oct 7 13:21:13 2017 +0100

    do not run discover parts if properties failed to complete

commit b52aaaf1a1
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Oct 7 13:08:46 2017 +0100

    fix typo

commit 41be926921
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Oct 7 13:04:45 2017 +0100

    run all check workers

commit a41d114965
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Oct 7 13:02:46 2017 +0100

    fix driver config

commit b10908a138
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Oct 7 12:43:50 2017 +0100

    use vars() cache between phases

commit 08b34e083d
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Oct 7 11:39:17 2017 +0100

    remove die() calls

commit b8108986fb
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Oct 7 11:31:59 2017 +0100

    phase fixups

commit 273cbbc11b
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Oct 7 09:42:41 2017 +0100

    change stage to phase

commit 256c10bae5
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Oct 7 09:35:14 2017 +0100

    multi worker actions need not return done from all workers

commit ee38bae48a
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Oct 7 09:05:25 2017 +0100

    store result of worker if best for this phase so far

commit 5bddfc73ba
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Oct 7 08:50:31 2017 +0100

    auto debug-log worker return messages

commit 8b660a89c0
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Fri Oct 6 07:48:58 2017 +0100

    bug fixes

commit b58a5816a9
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Fri Oct 6 07:44:20 2017 +0100

    remove unnecessary check phases

commit e44f06364a
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Fri Oct 6 07:18:03 2017 +0100

    fix unknown command check in netdisco-do

commit 3af13f0dfe
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Fri Oct 6 07:15:59 2017 +0100

    introduce noop and refactor checks in all workers

commit 98463c8cad
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Oct 1 10:49:12 2017 +0100

    no need to debug log if there are no hooks in phase

commit 3b32e84312
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Oct 1 08:18:13 2017 +0100

    fiddle about with runner logic to fix exit states

commit 8fdba38ee0
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Fri Sep 29 08:01:42 2017 +0100

    cannot reuse a worker as the job will be already set and the wrong plugins loaded

commit a155d9cb77
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Fri Sep 29 08:01:06 2017 +0100

    should defer when we cannot connect to device

commit 10b5f6cbc4
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Fri Sep 29 08:00:32 2017 +0100

    fix bug in where workerconf acls are checked

commit 2a74e0befa
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Fri Sep 29 07:38:05 2017 +0100

    can pass device instance to check_*

commit 4256b117df
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Fri Sep 29 07:27:14 2017 +0100

    move device_auth build to be with community defaults setting

commit a2de2c1616
Merge: 32be11c3 8dc4b9bc
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Fri Sep 29 07:21:03 2017 +0100

    Merge branch 'master' into og-coreplugins

commit 32be11c3ff
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Thu Sep 21 00:09:29 2017 +0100

    move remaining interactive actions to be plugins

commit 3e41c93f5a
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Sep 20 21:47:50 2017 +0100

    clean snmp handling

commit 30a2d5dd86
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Sep 20 21:00:29 2017 +0100

    make sure check plugins are loaded/run before phases

commit 3454d95a84
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Sep 20 20:53:52 2017 +0100

    capture result on main phase as well

commit 559fa4f93f
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Sep 18 22:46:35 2017 +0100

    build device_auth from communities

commit 1969291719
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Sep 18 22:04:22 2017 +0100

    simplify to remove phases and fewer hooks

commit 6f78032e28
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Thu Sep 14 21:30:03 2017 +0100

    add phase to test worker

commit 6edd2dc879
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Sep 13 21:51:40 2017 +0100

    no need to list all plugins

commit dfaeb34d8c
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Sep 13 20:42:41 2017 +0100

    add reset after messing with snmp context or community index

commit 09214dce92
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Sep 13 20:29:21 2017 +0100

    no need to pass $snmp around

commit 58cd488ccc
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Sep 13 19:22:40 2017 +0100

    refactor layer and pseudo checks

commit 753acc607f
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Sep 13 10:53:12 2017 +0100

    use overloaded $device

commit d5d39289d6
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Sep 13 10:44:31 2017 +0100

    rename init stage to check

commit 1fdb086183
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 12 08:12:12 2017 +0100

    refactor to remove second loop

commit 64a9491115
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Sep 10 16:09:45 2017 +0100

    change to init, first, second stages

commit 5f2da69697
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Sep 9 22:26:04 2017 +0100

    move discover and discoverall to worker plugins

commit c6ebb7cf07
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Sep 9 16:44:32 2017 +0100

    move arpnip and arpwalk to worker plugins

commit 16a79463cb
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Sep 9 16:27:58 2017 +0100

    set snmp driver on macsuck phase workers

commit 9167e02de5
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Sep 9 15:55:53 2017 +0100

    move macsuck and macwalk to worker plugins (macsuck needs snmp scope guard)

commit 68ca85643b
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Sep 9 14:56:15 2017 +0100

    move expire and expirenodes to worker plugins

commit 271ef1a25c
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Sep 9 14:46:00 2017 +0100

    move nbtstat and nbtwalk to worker plugins

commit e7508a9eca
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Sep 6 21:23:54 2017 +0100

    move all netdisco-do action to worker plugins

commit 707fc82b99
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Sep 6 21:01:37 2017 +0100

    remove psql code from netdisco-do and fix detection of misspelled action

commit 411918e3f8
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Sep 6 20:56:26 2017 +0100

    only load worker plugins for the action

commit 1f9740c0e2
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Sep 6 18:30:43 2017 +0100

    shorten hook names

commit a59c23de79
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Sep 6 18:27:34 2017 +0100

    make psql worker primary, add hook debug log

commit 36c70220a2
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 5 22:39:22 2017 +0100

    allow two forms of worker declaration, and update docs

commit a79cb9a9e4
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 5 22:10:53 2017 +0100

    all the bug fixes and a working plugin!!!!!!!!! :-D

commit 04896202e0
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 5 21:39:41 2017 +0100

    refine runner

commit 547fce2f3c
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 5 20:56:21 2017 +0100

    hack the status class to regen if needed

commit cd71a0b7a8
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 5 20:41:05 2017 +0100

    move status update to job class

commit c8e5cea4ed
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 5 20:37:13 2017 +0100

    objectify the running

commit f48004fffa
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 5 19:58:28 2017 +0100

    bug squish

commit 46ece568f6
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 5 19:54:57 2017 +0100

    implement runner?!

commit fc9c60f707
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 5 19:28:38 2017 +0100

    rename ok to is_ok and change slot names to avoid conflict with creators

commit 3ee85383ab
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 5 19:25:41 2017 +0100

    skip worker when action is per-device but no creds

commit 75abdad812
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Sep 4 21:54:37 2017 +0100

    further work on retval handling from workers

commit 4c1fdf4f92
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Sep 4 20:37:53 2017 +0100

    move worker plugin loader to Worker.pm

commit be0c5181a3
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Sep 4 20:35:42 2017 +0100

    move Runner to Worker namespace

commit 1c2cf924bc
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Sep 4 20:33:20 2017 +0100

    worker roles in Role namespace

commit 3099eda393
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Sep 4 20:30:58 2017 +0100

    load workers when runner role is loaded

commit a8c58a7b05
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Sep 3 22:30:28 2017 +0100

    initial broken implementation of the runner

commit 49b5274c33
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Sep 3 19:04:20 2017 +0100

    use run() mixin to exec action

commit e0a666668a
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Sep 3 18:54:44 2017 +0100

    fix pod; set status defaults; stub runner mixin

commit 8eaa33770c
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Sep 3 18:45:00 2017 +0100

    rename Core to Worker and move other packages around

commit 4def0af0b0
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Sep 3 17:58:03 2017 +0100

    better use of new status class

commit 8675bf62c6
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Sep 3 17:27:38 2017 +0100

    fix hook naming and implement primary workers

commit ef1bb81f2b
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Sep 3 17:26:27 2017 +0100

    new backend status class

commit 5f50dfadf1
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Sep 3 16:51:55 2017 +0100

    new Backend package to load core plugins

commit 3baa7a818a
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Sep 3 16:22:29 2017 +0100

    remove unnecessary Worker::Common role

commit 36b4adcc06
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Sep 3 16:17:29 2017 +0100

    disambiguate util/backend package and remove backend prelaod

commit 98bff731bd
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Sep 2 08:25:06 2017 +0100

    settle on a design for hook override, I think

commit fe5c16a16d
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Aug 30 20:37:36 2017 +0100

    rework docs to be more clear and reflect new operation

commit b34ba1977c
Merge: 31d1977f c34ed61d
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Aug 21 21:17:46 2017 +0100

    Merge branch 'master' into og-coreplugins

commit 31d1977f1e
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Aug 14 18:11:42 2017 +0100

    Revert "move expire code to be initial plugin pilot (broken)"

    I think we'll only do the new backend code for jobs with a device.

    This reverts commit 07998b72d9.

commit 61dc80aff8
Merge: 07998b72 ade02db1
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Aug 14 18:10:29 2017 +0100

    Merge branch 'master' into og-coreplugins

commit 07998b72d9
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Aug 5 22:15:00 2017 +0100

    move expire code to be initial plugin pilot (broken)

commit 685ec02108
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Aug 5 22:10:58 2017 +0100

    pass $job to the core worker

commit d6523fe543
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Aug 5 22:01:49 2017 +0100

    $job->device is always a DBIC row

commit ee6deea01b
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Aug 5 18:12:34 2017 +0100

    load plugins

commit fd80096ca2
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Aug 5 16:53:16 2017 +0100

    rename all the things

commit 464c42d1f5
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Aug 2 10:19:16 2017 +0100

    use Scope::Guard to reduce device_auth

commit ec041dafd2
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Aug 1 15:34:37 2017 +0100

    the other way around

commit 33d2fe13bd
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Jul 31 17:57:29 2017 +0100

    fix pod

commit 3faee1cf16
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Jul 31 17:55:10 2017 +0100

    remove need for instance() call

commit c6d0f1c035
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Jul 26 13:51:23 2017 +0100

    add doc note on accessing transports

commit dca4b4fc03
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Jul 26 11:50:10 2017 +0100

    add backend driver documentation

commit 052a2acd79
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Jul 26 10:16:58 2017 +0100

    rename web plugins doc

commit 69c9a6393a
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Jul 26 10:12:42 2017 +0100

    rename args to driverconf

commit 2586a36f8c
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Jul 25 22:41:10 2017 +0100

    new version of core plugin manager with better config and filters

commit 4056831f99
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Jul 25 20:53:56 2017 +0100

    change SNMP to be a cached transport singleton

commit c31030ef70
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Jul 23 13:46:27 2017 +0100

    fixes because Dancer docs are a mess!

commit f65ef90b86
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Jul 22 08:11:36 2017 +0100

    rename snmp_auth to device_auth and include a little doc on transports

commit d61556e1cf
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Jul 22 07:54:26 2017 +0100

    plugin config added

commit de8de56308
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Jul 12 21:38:31 2017 +0100

    initial core plugin implementation
This commit is contained in:
Oliver Gorwits
2017-11-19 13:34:35 +00:00
parent d3d81eb67d
commit 5ff7d6fe47
85 changed files with 4078 additions and 2502 deletions

View File

@@ -0,0 +1,55 @@
package App::Netdisco::Worker::Loader;
use strict;
use warnings;
use Module::Load ();
use Dancer qw/:moose :syntax/;
use Moo::Role;
use namespace::clean;
has [qw/workers_check
workers_early
workers_main
workers_user/] => ( is => 'rw' );
sub load_workers {
my $self = shift;
my $action = $self->job->action or die "missing action\n";
my @core_plugins = @{ setting('worker_plugins') || [] };
my @user_plugins = @{ setting('extra_worker_plugins') || [] };
# load worker plugins for our action
foreach my $plugin (@user_plugins, @core_plugins) {
$plugin =~ s/^X::/+App::NetdiscoX::Worker::Plugin::/;
$plugin = 'App::Netdisco::Worker::Plugin::'. $plugin
if $plugin !~ m/^\+/;
$plugin =~ s/^\+//;
next unless $plugin =~ m/::Plugin::${action}(?:::|$)/i;
$ENV{PLUGIN_LOAD_DEBUG} && debug "loading worker plugin $plugin";
Module::Load::load $plugin;
}
# now vars->{workers} is populated, we set the dispatch order
my $workers = vars->{'workers'}->{$action} || {};
#use DDP; p vars->{'workers'};
foreach my $phase (qw/check early main user/) {
my $pname = "workers_${phase}";
my @wset = ();
foreach my $namespace (sort keys %{ $workers->{$phase} }) {
foreach my $priority (sort {$b <=> $a}
keys %{ $workers->{$phase}->{$namespace} }) {
push @wset, @{ $workers->{$phase}->{$namespace}->{$priority} };
}
}
$self->$pname( \@wset );
}
}
true;

View File

@@ -0,0 +1,164 @@
package App::Netdisco::Worker::Plugin;
use Dancer ':syntax';
use Dancer::Plugin;
use App::Netdisco::Util::Permission qw/check_acl_no check_acl_only/;
use aliased 'App::Netdisco::Worker::Status';
use Scope::Guard 'guard';
register 'register_worker' => sub {
my ($self, $first, $second) = plugin_args(@_);
my $workerconf = (ref $first eq 'HASH' ? $first : {});
my $code = (ref $first eq 'CODE' ? $first : $second);
return error "bad param to register_worker"
unless ((ref sub {} eq ref $code) and (ref {} eq ref $workerconf));
my $package = (caller)[0];
if ($package =~ m/Plugin::(\w+)(?:::(\w+))?/) {
$workerconf->{action} = lc($1);
$workerconf->{namespace} = lc($2) if $2;
}
return error "failed to parse action in '$package'"
unless $workerconf->{action};
$workerconf->{phase} ||= 'user';
$workerconf->{namespace} ||= '_base_';
$workerconf->{priority} ||= (exists $workerconf->{driver}
? (setting('driver_priority')->{$workerconf->{driver}} || 0) : 0);
my $worker = sub {
my $job = shift or die 'missing job param';
# use DDP; p $workerconf;
# update job's record of namespace and priority
# check to see if this namespace has already passed at higher priority
return if $job->namespace_passed($workerconf);
my @newuserconf = ();
my @userconf = @{ setting('device_auth') || [] };
# worker might be vendor/platform specific
if (ref $job->device) {
my $no = (exists $workerconf->{no} ? $workerconf->{no} : undef);
my $only = (exists $workerconf->{only} ? $workerconf->{only} : undef);
return $job->add_status( Status->noop('worker not applicable to this device') )
if ($no and check_acl_no($job->device, $no))
or ($only and not check_acl_only($job->device, $only));
# reduce device_auth by driver and action filters
foreach my $stanza (@userconf) {
next if exists $stanza->{driver} and exists $workerconf->{driver}
and (($stanza->{driver} || '') ne ($workerconf->{driver} || ''));
# filter here rather than in Runner as runner does not know namespace
next if exists $stanza->{action}
and not _find_matchaction($workerconf, lc($stanza->{action}));
push @newuserconf, $stanza;
}
# per-device action but no device creds available
return $job->add_status( Status->noop('worker driver or action not applicable') )
if 0 == scalar @newuserconf;
}
# back up and restore device_auth
my $guard = guard { set(device_auth => \@userconf) };
set(device_auth => \@newuserconf);
# use DDP; p @newuserconf;
# run worker
$code->($job, $workerconf);
};
# store the built worker as Worker.pm will build the dispatch order later on
push @{ vars->{'workers'}->{$workerconf->{action}}
->{$workerconf->{phase}}
->{$workerconf->{namespace}}
->{$workerconf->{priority}} }, $worker;
};
sub _find_matchaction {
my ($conf, $action) = @_;
return true if !defined $action;
$action = [$action] if ref [] ne ref $action;
foreach my $f (@$action) {
return true if
$f eq $conf->{action} or $f eq "$conf->{action}::$conf->{namespace}";
}
return false;
}
register_plugin;
true;
=head1 NAME
App::Netdisco::Worker::Plugin - Netdisco Workers
=head1 Introduction
L<App::Netdisco>'s plugin system allows users to write I<workers> to gather
information from network devices using different I<transports> and store
results in the database.
For example, transports might be SNMP, SSH, or HTTPS. Workers might be
combining those transports with application protocols such as SNMP, NETCONF
(OpenConfig with XML), RESTCONF (OpenConfig with JSON), eAPI, or even CLI
scraping. The combination of transport and protocol is known as a I<driver>.
Workers can be restricted to certain vendor platforms using familiar ACL
syntax. They are also attached to specific actions in Netdisco's backend
operation (discover, macsuck, etc).
=head1 Application Configuration
The C<worker_plugins> and C<extra_worker_plugins> settings list in YAML format
the set of Perl module names which are the plugins to be loaded.
Any change should go into your local C<deployment.yml> configuration file. If
you want to view the default settings, see the C<share/config.yml> file in the
C<App::Netdisco> distribution.
=head1 How to Configure
The C<extra_worker_plugins> setting is empty, and used when you want to add
new plugins and not change the set enabled by default. If you do want to add
to or remove from the default set, then create a version of C<worker_plugins>
instead.
Netdisco prepends "C<App::Netdisco::Worker::Plugin::>" to any entry in the
list. For example, "C<Discover::Wireless::UniFi>" will load the
C<App::Netdisco::Worker::Plugin::Discover::Wireless::UniFi> package.
You can prepend module names with "C<X::>" as shorthand for the "Netdisco
extension" namespace. For example, "C<X::Macsuck::WirelessNodes::UniFi>" will
load the L<App::NetdiscoX::Worker::Plugin::Macsuck::WirelessNodes::UniFi>
module.
If an entry in the list starts with a "C<+>" (plus) sign then Netdisco attemps
to load the module as-is, without prepending anything to the name. This allows
you to have worker plugins in any namespace.
Plugin modules can either ship with the App::Netdisco distribution itself, or
be installed separately. Perl uses the standard C<@INC> path searching
mechanism to load the plugin modules. See the C<include_paths> and
C<site_local_files> settings in order to modify C<@INC> for loading local
plugins.
As an example, if you set C<site_local_files> to be true, set
C<extra_worker_plugins> to be C<'X::MyPluginName'> (the plugin package is
"App::NetdiscoX::Worker::Plugin::MyPluginName") then your plugin lives at:
~netdisco/nd-site-local/lib/App/NetdiscoX/Worker/Plugin/MyPluginName.pm
The order of the entries is significant, workers being executed in the order
which they appear in C<extra_worker_plugins> followed by C<worker_plugins>.
See L<App::Netdisco::Manual::WritingWorkers> for further details.
=cut

View File

@@ -0,0 +1,31 @@
package App::Netdisco::Worker::Plugin::Arpnip;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
use App::Netdisco::Util::Device 'is_arpnipable_now';
register_worker({ phase => 'check' }, sub {
my ($job, $workerconf) = @_;
my $device = $job->device;
return Status->error('arpnip failed: unable to interpret device param')
unless defined $device;
return Status->error("arpnip skipped: $device not yet discovered")
unless $device->in_storage;
return Status->defer("arpnip skipped: $device is pseudo-device")
if $device->is_pseudo;
return Status->defer("arpnip skipped: $device has no layer 3 capability")
unless $device->has_layer(3);
return Status->defer("arpnip deferred: $device is not arpnipable")
unless is_arpnipable_now($device);
return Status->done('arpnip is able to run');
});
true;

View File

@@ -0,0 +1,117 @@
package App::Netdisco::Worker::Plugin::Arpnip::Nodes;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
use App::Netdisco::Transport::SNMP ();
use App::Netdisco::Util::Node 'check_mac';
use App::Netdisco::Util::FastResolver 'hostnames_resolve_async';
use Dancer::Plugin::DBIC 'schema';
use Time::HiRes 'gettimeofday';
use NetAddr::MAC ();
register_worker({ phase => 'main', driver => 'snmp' }, sub {
my ($job, $workerconf) = @_;
my $device = $job->device;
my $snmp = App::Netdisco::Transport::SNMP->reader_for($device)
or return Status->defer("arpnip failed: could not SNMP connect to $device");
# get v4 arp table
my $v4 = get_arps($device, $snmp->at_paddr, $snmp->at_netaddr);
# get v6 neighbor cache
my $v6 = get_arps($device, $snmp->ipv6_n2p_mac, $snmp->ipv6_n2p_addr);
# would be possible just to use now() on updated records, but by using this
# same value for them all, we _can_ if we want add a job at the end to
# select and do something with the updated set (no reason to yet, though)
my $now = 'to_timestamp('. (join '.', gettimeofday) .')';
# update node_ip with ARP and Neighbor Cache entries
store_arp(\%$_, $now) for @$v4;
debug sprintf ' [%s] arpnip - processed %s ARP Cache entries',
$device->ip, scalar @$v4;
store_arp(\%$_, $now) for @$v6;
debug sprintf ' [%s] arpnip - processed %s IPv6 Neighbor Cache entries',
$device->ip, scalar @$v6;
$device->update({last_arpnip => \$now});
return Status->done("Ended arpnip for $device");
});
# get an arp table (v4 or v6)
sub get_arps {
my ($device, $paddr, $netaddr) = @_;
my @arps = ();
while (my ($arp, $node) = each %$paddr) {
my $ip = $netaddr->{$arp};
next unless defined $ip;
next unless check_mac($device, $node);
push @arps, {
node => $node,
ip => $ip,
dns => undef,
};
}
debug sprintf ' resolving %d ARP entries with max %d outstanding requests',
scalar @arps, $ENV{'PERL_ANYEVENT_MAX_OUTSTANDING_DNS'};
my $resolved_ips = hostnames_resolve_async(\@arps);
return $resolved_ips;
}
=head2 store_arp( \%host, $now? )
Stores a new entry to the C<node_ip> table with the given MAC, IP (v4 or v6)
and DNS host name. Host details are provided in a Hash ref:
{
ip => '192.0.2.1',
node => '00:11:22:33:44:55',
dns => 'myhost.example.com',
}
The C<dns> entry is optional. The update will mark old entries for this IP as
no longer C<active>.
Optionally a literal string can be passed in the second argument for the
C<time_last> timestamp, otherwise the current timestamp (C<now()>) is used.
=cut
sub store_arp {
my ($hash_ref, $now) = @_;
$now ||= 'now()';
my $ip = $hash_ref->{'ip'};
my $mac = NetAddr::MAC->new($hash_ref->{'node'});
my $name = $hash_ref->{'dns'};
return if !defined $mac or $mac->errstr;
schema('netdisco')->txn_do(sub {
my $current = schema('netdisco')->resultset('NodeIp')
->search(
{ ip => $ip, -bool => 'active'},
{ columns => [qw/mac ip/] })->update({active => \'false'});
schema('netdisco')->resultset('NodeIp')
->update_or_create(
{
mac => $mac->as_ieee,
ip => $ip,
dns => $name,
active => \'true',
time_last => \$now,
},
{
key => 'primary',
for => 'update',
});
});
}
true;

View File

@@ -0,0 +1,74 @@
package App::Netdisco::Worker::Plugin::Arpnip::Subnets;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
use App::Netdisco::Transport::SNMP ();
use App::Netdisco::Util::Permission 'check_acl_no';
use Dancer::Plugin::DBIC 'schema';
use NetAddr::IP::Lite ':lower';
use Time::HiRes 'gettimeofday';
register_worker({ phase => 'main', driver => 'snmp' }, sub {
my ($job, $workerconf) = @_;
my $device = $job->device;
my $snmp = App::Netdisco::Transport::SNMP->reader_for($device)
or return Status->defer("arpnip failed: could not SNMP connect to $device");
# get directly connected networks
my @subnets = gather_subnets($device);
# TODO: IPv6 subnets
my $now = 'to_timestamp('. (join '.', gettimeofday) .')';
store_subnet($_, $now) for @subnets;
return Status->noop(sprintf ' [%s] arpnip - processed %s Subnet entries',
$device->ip, scalar @subnets);
});
# gathers device subnets
sub gather_subnets {
my $device = shift;
my @subnets = ();
my $snmp = App::Netdisco::Transport::SNMP->reader_for($device)
or return (); # already checked!
my $ip_netmask = $snmp->ip_netmask;
foreach my $entry (keys %$ip_netmask) {
my $ip = NetAddr::IP::Lite->new($entry);
my $addr = $ip->addr;
next if $addr eq '0.0.0.0';
next if check_acl_no($ip, 'group:__LOCAL_ADDRESSES__');
next if setting('ignore_private_nets') and $ip->is_rfc1918;
my $netmask = $ip_netmask->{$addr};
next if $netmask eq '255.255.255.255' or $netmask eq '0.0.0.0';
my $cidr = NetAddr::IP::Lite->new($addr, $netmask)->network->cidr;
debug sprintf ' [%s] arpnip - found subnet %s', $device->ip, $cidr;
push @subnets, $cidr;
}
return @subnets;
}
# update subnets with new networks
sub store_subnet {
my ($subnet, $now) = @_;
schema('netdisco')->txn_do(sub {
schema('netdisco')->resultset('Subnet')->update_or_create(
{
net => $subnet,
last_discover => \$now,
},
{ for => 'update' });
});
}
true;

View File

@@ -0,0 +1,31 @@
package App::Netdisco::Worker::Plugin::Arpwalk;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
use App::Netdisco::JobQueue qw/jq_queued jq_insert/;
use Dancer::Plugin::DBIC 'schema';
register_worker({ phase => 'main' }, sub {
my ($job, $workerconf) = @_;
my %queued = map {$_ => 1} jq_queued('arpnip');
my @devices = schema('netdisco')->resultset('Device')->search({
-or => [ 'vendor' => undef, 'vendor' => { '!=' => 'netdisco' }],
})->has_layer('3')->get_column('ip')->all;
my @filtered_devices = grep {!exists $queued{$_}} @devices;
jq_insert([
map {{
device => $_,
action => 'arpnip',
username => $job->username,
userip => $job->userip,
}} (@filtered_devices)
]);
return Status->done('Queued arpnip job for all devices');
});
true;

View File

@@ -0,0 +1,43 @@
package App::Netdisco::Worker::Plugin::Contact;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
use App::Netdisco::Transport::SNMP;
register_worker({ phase => 'check' }, sub {
return Status->error('Missing device (-d).')
unless defined shift->device;
return Status->done('Contact is able to run');
});
register_worker({ phase => 'main', driver => 'snmp' }, sub {
my ($job, $workerconf) = @_;
my ($device, $data) = map {$job->$_} qw/device extra/;
# snmp connect using rw community
my $snmp = App::Netdisco::Transport::SNMP->writer_for($device)
or return Status->defer("failed to connect to $device to update contact");
my $rv = $snmp->set_contact($data);
if (!defined $rv) {
return Status->error(
"failed to set contact on $device: ". ($snmp->error || ''));
}
# confirm the set happened
$snmp->clear_cache;
my $new_data = ($snmp->contact || '');
if ($new_data ne $data) {
return Status->error("verify of contact failed on $device: $new_data");
}
# update netdisco DB
$device->update({contact => $data});
return Status->done("Updated contact on $device to [$data]");
});
true;

View File

@@ -0,0 +1,24 @@
package App::Netdisco::Worker::Plugin::Delete;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
use App::Netdisco::Util::Device 'delete_device';
register_worker({ phase => 'check' }, sub {
return Status->error('Missing device (-d).')
unless defined shift->device;
return Status->done('Delete is able to run');
});
register_worker({ phase => 'main' }, sub {
my ($job, $workerconf) = @_;
my ($device, $port, $extra) = map {$job->$_} qw/device port extra/;
$port = ($port ? 1 : 0);
delete_device($device, $port, $extra);
return Status->done("Deleted device: $device");
});
true;

View File

@@ -0,0 +1,28 @@
package App::Netdisco::Worker::Plugin::Discover;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
use App::Netdisco::Util::Device 'is_discoverable_now';
register_worker({ phase => 'check' }, sub {
my ($job, $workerconf) = @_;
my $device = $job->device;
return Status->error('discover failed: unable to interpret device param')
unless defined $device;
return Status->error("discover failed: no device param (need -d ?)")
if $device->ip eq '0.0.0.0';
return Status->defer("discover skipped: $device is pseudo-device")
if $device->is_pseudo;
return Status->defer("discover deferred: $device is not discoverable")
unless is_discoverable_now($device);
return Status->done('Discover is able to run.');
});
true;

View File

@@ -0,0 +1,80 @@
package App::Netdisco::Worker::Plugin::Discover::CanonicalIP;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
use App::Netdisco::Transport::SNMP ();
use App::Netdisco::Util::Permission 'check_acl_only';
use App::Netdisco::Util::DNS 'ipv4_from_hostname';
use Dancer::Plugin::DBIC 'schema';
register_worker({ phase => 'main', driver => 'snmp' }, sub {
my ($job, $workerconf) = @_;
my $device = $job->device;
return unless $device->in_storage;
my $snmp = App::Netdisco::Transport::SNMP->reader_for($device)
or return Status->defer("discover failed: could not SNMP connect to $device");
my $old_ip = $device->ip;
my $new_ip = $old_ip;
my $revofname = ipv4_from_hostname($snmp->name);
if (setting('reverse_sysname') and $revofname) {
if (App::Netdisco::Transport::SNMP->test_connection( $new_ip )) {
$new_ip = $revofname;
}
else {
debug sprintf ' [%s] device - cannot renumber to %s - SNMP connect failed',
$old_ip, $revofname;
}
}
if (scalar @{ setting('device_identity') }) {
my @idmaps = @{ setting('device_identity') };
my $devips = $device->device_ips->order_by('alias');
ALIAS: while (my $alias = $devips->next) {
next if $alias->alias eq $old_ip;
foreach my $map (@idmaps) {
next unless ref {} eq ref $map;
foreach my $key (sort keys %$map) {
# lhs matches device, rhs matches device_ip
if (check_acl_only($device, $key)
and check_acl_only($alias, $map->{$key})) {
if (App::Netdisco::Transport::SNMP->test_connection( $alias->alias )) {
$new_ip = $alias->alias;
last ALIAS;
}
else {
debug sprintf ' [%s] device - cannot renumber to %s - SNMP connect failed',
$old_ip, $alias->alias;
}
}
}
}
} # ALIAS
}
return if $new_ip eq $old_ip;
schema('netdisco')->txn_do(sub {
# delete target device with the same vendor and serial number
schema('netdisco')->resultset('Device')->search({
ip => $new_ip, vendor => $device->vendor, serial => $device->serial,
})->delete;
# if target device exists then this will die
$device->renumber($new_ip)
or die "cannot renumber to: $new_ip"; # rollback
return Status->noop(sprintf ' [%s] device - changed IP to %s (%s)',
$old_ip, $device->ip, ($device->dns || ''));
});
});
true;

View File

@@ -0,0 +1,95 @@
package App::Netdisco::Worker::Plugin::Discover::Entities;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
use App::Netdisco::Transport::SNMP ();
use Dancer::Plugin::DBIC 'schema';
use Encode;
register_worker({ phase => 'main', driver => 'snmp' }, sub {
my ($job, $workerconf) = @_;
my $device = $job->device;
return unless $device->in_storage;
my $snmp = App::Netdisco::Transport::SNMP->reader_for($device)
or return Status->defer("discover failed: could not SNMP connect to $device");
my $e_index = $snmp->e_index;
if (!defined $e_index) {
schema('netdisco')->txn_do(sub {
my $gone = $device->modules->delete;
debug sprintf ' [%s] modules - removed %d chassis modules',
$device->ip, $gone;
$device->modules->update_or_create({
ip => $device->ip,
index => 1,
parent => 0,
name => 'chassis',
class => 'chassis',
pos => -1,
# too verbose and link doesn't work anyway
# description => $device->description,
sw_ver => $device->os_ver,
serial => $device->serial,
model => $device->model,
fru => \'false',
last_discover => \'now()',
});
});
return Status->noop(
sprintf ' [%s] modules - 0 chassis components (added one pseudo for chassis)',
$device->ip);
}
my $e_descr = $snmp->e_descr;
my $e_type = $snmp->e_type;
my $e_parent = $snmp->e_parent;
my $e_name = $snmp->e_name;
my $e_class = $snmp->e_class;
my $e_pos = $snmp->e_pos;
my $e_hwver = $snmp->e_hwver;
my $e_fwver = $snmp->e_fwver;
my $e_swver = $snmp->e_swver;
my $e_model = $snmp->e_model;
my $e_serial = $snmp->e_serial;
my $e_fru = $snmp->e_fru;
# build device modules list for DBIC
my (@modules, %seen_idx);
foreach my $entry (keys %$e_index) {
next if $seen_idx{ $e_index->{$entry} }++;
push @modules, {
index => $e_index->{$entry},
type => $e_type->{$entry},
parent => $e_parent->{$entry},
name => Encode::decode('UTF-8', $e_name->{$entry}),
class => $e_class->{$entry},
pos => $e_pos->{$entry},
hw_ver => Encode::decode('UTF-8', $e_hwver->{$entry}),
fw_ver => Encode::decode('UTF-8', $e_fwver->{$entry}),
sw_ver => Encode::decode('UTF-8', $e_swver->{$entry}),
model => Encode::decode('UTF-8', $e_model->{$entry}),
serial => Encode::decode('UTF-8', $e_serial->{$entry}),
fru => $e_fru->{$entry},
description => Encode::decode('UTF-8', $e_descr->{$entry}),
last_discover => \'now()',
};
}
schema('netdisco')->txn_do(sub {
my $gone = $device->modules->delete;
debug sprintf ' [%s] modules - removed %d chassis modules',
$device->ip, $gone;
$device->modules->populate(\@modules);
return Status->noop(sprintf ' [%s] modules - added %d new chassis modules',
$device->ip, scalar @modules);
});
});
true;

View File

@@ -0,0 +1,338 @@
package App::Netdisco::Worker::Plugin::Discover::Neighbors;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
use App::Netdisco::Transport::SNMP ();
use App::Netdisco::Util::Device
qw/get_device match_devicetype is_discoverable/;
use App::Netdisco::Util::Permission 'check_acl_no';
use App::Netdisco::JobQueue 'jq_insert';
use Dancer::Plugin::DBIC 'schema';
use List::MoreUtils ();
use NetAddr::MAC;
use Encode;
=head2 discover_new_neighbors( )
Given a Device database object, and a working SNMP connection, discover and
store the device's port neighbors information.
Entries in the Topology database table will override any discovered device
port relationships.
The Device database object can be a fresh L<DBIx::Class::Row> object which is
not yet stored to the database.
Any discovered neighbor unknown to Netdisco will have a C<discover> job
immediately queued (subject to the filtering by the C<discover_*> settings).
=cut
register_worker({ phase => 'main', driver => 'snmp' }, sub {
my ($job, $workerconf) = @_;
my $device = $job->device;
return unless $device->in_storage;
my $snmp = App::Netdisco::Transport::SNMP->reader_for($device)
or return Status->defer("discover failed: could not SNMP connect to $device");
my @to_discover = store_neighbors($device);
# only enqueue if device is not already discovered,
# discover_* config permits the discovery
foreach my $neighbor (@to_discover) {
my ($ip, $remote_type) = @$neighbor;
my $device = get_device($ip);
next if $device->in_storage;
if (not is_discoverable($device, $remote_type)) {
debug sprintf
' queue - %s, type [%s] excluded by discover_* config',
$ip, ($remote_type || '');
next;
}
jq_insert({
device => $ip,
action => 'discover',
subaction => 'with-nodes',
});
}
});
=head2 store_neighbors( $device )
returns: C<@to_discover>
Given a Device database object, and a working SNMP connection, discover and
store the device's port neighbors information.
Entries in the Topology database table will override any discovered device
port relationships.
The Device database object can be a fresh L<DBIx::Class::Row> object which is
not yet stored to the database.
A list of discovererd neighbors will be returned as [C<$ip>, C<$type>] tuples.
=cut
sub store_neighbors {
my $device = shift;
my @to_discover = ();
my $snmp = App::Netdisco::Transport::SNMP->reader_for($device)
or return (); # already checked!
# first allow any manually configured topology to be set
set_manual_topology($device);
if (!defined $snmp->has_topo) {
debug sprintf ' [%s] neigh - neighbor protocols are not enabled', $device->ip;
return @to_discover;
}
my $interfaces = $snmp->interfaces;
my $c_if = $snmp->c_if;
my $c_port = $snmp->c_port;
my $c_id = $snmp->c_id;
my $c_platform = $snmp->c_platform;
my $c_cap = $snmp->c_cap;
# v4 and v6 neighbor tables
my $c_ip = ($snmp->c_ip || {});
my %c_ipv6 = %{ ($snmp->can('hasLLDP') and $snmp->hasLLDP)
? ($snmp->lldp_ipv6 || {}) : {} };
# remove keys with undef values, as c_ip does
delete @c_ipv6{ grep { not defined $c_ipv6{$_} } keys %c_ipv6 };
# now combine them, v6 wins
$c_ip = { %$c_ip, %c_ipv6 };
foreach my $entry (sort (List::MoreUtils::uniq( (keys %$c_ip), (keys %$c_cap) ))) {
if (!defined $c_if->{$entry} or !defined $interfaces->{ $c_if->{$entry} }) {
debug sprintf ' [%s] neigh - port for IID:%s not resolved, skipping',
$device->ip, $entry;
next;
}
my $port = $interfaces->{ $c_if->{$entry} };
my $portrow = schema('netdisco')->resultset('DevicePort')
->single({ip => $device->ip, port => $port});
if (!defined $portrow) {
info sprintf ' [%s] neigh - local port %s not in database!',
$device->ip, $port;
next;
}
if (ref $c_ip->{$entry}) {
error sprintf ' [%s] neigh - Error! port %s has multiple neighbors - skipping',
$device->ip, $port;
next;
}
my $remote_ip = $c_ip->{$entry};
my $remote_port = undef;
my $remote_type = Encode::decode('UTF-8', $c_platform->{$entry} || '');
my $remote_id = Encode::decode('UTF-8', $c_id->{$entry});
my $remote_cap = $c_cap->{$entry} || [];
# IP Phone and WAP detection type fixup
if (scalar @$remote_cap or $remote_type) {
my $phone_flag = grep {match_devicetype($_, 'phone_capabilities')}
@$remote_cap;
my $ap_flag = grep {match_devicetype($_, 'wap_capabilities')}
@$remote_cap;
if ($phone_flag or match_devicetype($remote_type, 'phone_platforms')) {
$remote_type = 'IP Phone: '. $remote_type
if $remote_type !~ /ip.phone/i;
}
elsif ($ap_flag or match_devicetype($remote_type, 'wap_platforms')) {
$remote_type = 'AP: '. $remote_type;
}
$portrow->update({remote_type => $remote_type});
}
if ($portrow->manual_topo) {
info sprintf ' [%s] neigh - %s has manually defined topology',
$device->ip, $port;
next;
}
next unless $remote_ip;
# a bunch of heuristics to search known devices if we don't have a
# useable remote IP...
if ($remote_ip eq '0.0.0.0' or
check_acl_no($remote_ip, 'group:__LOCAL_ADDRESSES__')) {
if ($remote_id) {
my $devices = schema('netdisco')->resultset('Device');
my $neigh = $devices->single({name => $remote_id});
info sprintf
' [%s] neigh - bad address %s on port %s, searching for %s instead',
$device->ip, $remote_ip, $port, $remote_id;
if (!defined $neigh) {
my $mac = NetAddr::MAC->new(mac => $remote_id);
if ($mac and not $mac->errstr) {
$neigh = $devices->single({mac => $mac->as_ieee});
}
}
# some HP switches send 127.0.0.1 as remote_ip if no ip address
# on default vlan for HP switches remote_ip looks like
# "myswitchname(012345-012345)"
if (!defined $neigh) {
(my $tmpid = $remote_id) =~ s/.*\(([0-9a-f]{6})-([0-9a-f]{6})\).*/$1$2/;
my $mac = NetAddr::MAC->new(mac => $tmpid);
if ($mac and not $mac->errstr) {
info sprintf
'[%s] neigh - found neighbor %s by MAC %s',
$device->ip, $remote_id, $mac->as_ieee;
$neigh = $devices->single({mac => $mac->as_ieee});
}
}
if (!defined $neigh) {
(my $shortid = $remote_id) =~ s/\..*//;
$neigh = $devices->single({name => { -ilike => "${shortid}%" }});
}
if ($neigh) {
$remote_ip = $neigh->ip;
info sprintf ' [%s] neigh - found %s with IP %s',
$device->ip, $remote_id, $remote_ip;
}
else {
info sprintf ' [%s] neigh - could not find %s, skipping',
$device->ip, $remote_id;
next;
}
}
else {
info sprintf ' [%s] neigh - skipping unuseable address %s on port %s',
$device->ip, $remote_ip, $port;
next;
}
}
# what we came here to do.... discover the neighbor
debug sprintf
' [%s] neigh - adding neighbor %s, type [%s], on %s to discovery queue',
$device->ip, $remote_ip, ($remote_type || ''), $port;
push @to_discover, [$remote_ip, $remote_type];
$remote_port = $c_port->{$entry};
if (defined $remote_port) {
# clean weird characters
$remote_port =~ s/[^\d\/\.,()\w:-]+//gi;
}
else {
info sprintf ' [%s] neigh - no remote port found for port %s at %s',
$device->ip, $port, $remote_ip;
}
$portrow->update({
remote_ip => $remote_ip,
remote_port => $remote_port,
remote_type => $remote_type,
remote_id => $remote_id,
is_uplink => \"true",
manual_topo => \"false",
});
# update master of our aggregate to be a neighbor of
# the master on our peer device (a lot of iffs to get there...).
# & cannot use ->neighbor prefetch because this is the port insert!
if (defined $portrow->slave_of) {
my $peer_device = get_device($remote_ip);
my $master = schema('netdisco')->resultset('DevicePort')->single({
ip => $device->ip,
port => $portrow->slave_of
});
if ($peer_device and $peer_device->in_storage and $master
and not ($portrow->is_master or defined $master->slave_of)) {
my $peer_port = schema('netdisco')->resultset('DevicePort')->single({
ip => $peer_device->ip,
port => $portrow->remote_port,
});
$master->update({
remote_ip => ($peer_device->ip || $remote_ip),
remote_port => ($peer_port ? $peer_port->slave_of : undef ),
is_uplink => \"true",
is_master => \"true",
manual_topo => \"false",
});
}
}
}
return @to_discover;
}
# take data from the topology table and update remote_ip and remote_port
# in the devices table. only use root_ips and skip any bad topo entries.
sub set_manual_topology {
my $device = shift;
my $snmp = App::Netdisco::Transport::SNMP->reader_for($device) or return;
schema('netdisco')->txn_do(sub {
# clear manual topology flags
schema('netdisco')->resultset('DevicePort')
->search({ip => $device->ip})->update({manual_topo => \'false'});
my $topo_links = schema('netdisco')->resultset('Topology')
->search({-or => [dev1 => $device->ip, dev2 => $device->ip]});
debug sprintf ' [%s] neigh - setting manual topology links', $device->ip;
while (my $link = $topo_links->next) {
# could fail for broken topo, but we ignore to try the rest
try {
schema('netdisco')->txn_do(sub {
# only work on root_ips
my $left = get_device($link->dev1);
my $right = get_device($link->dev2);
# skip bad entries
return unless ($left->in_storage and $right->in_storage);
$left->ports
->single({port => $link->port1})
->update({
remote_ip => $right->ip,
remote_port => $link->port2,
remote_type => undef,
remote_id => undef,
is_uplink => \"true",
manual_topo => \"true",
});
$right->ports
->single({port => $link->port2})
->update({
remote_ip => $left->ip,
remote_port => $link->port1,
remote_type => undef,
remote_id => undef,
is_uplink => \"true",
manual_topo => \"true",
});
});
};
}
});
}
true;

View File

@@ -0,0 +1,81 @@
package App::Netdisco::Worker::Plugin::Discover::PortPower;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
use App::Netdisco::Transport::SNMP ();
use Dancer::Plugin::DBIC 'schema';
register_worker({ phase => 'main', driver => 'snmp' }, sub {
my ($job, $workerconf) = @_;
my $device = $job->device;
return unless $device->in_storage;
my $snmp = App::Netdisco::Transport::SNMP->reader_for($device)
or return Status->defer("discover failed: could not SNMP connect to $device");
my $p_watts = $snmp->peth_power_watts;
my $p_status = $snmp->peth_power_status;
if (!defined $p_watts) {
return Status->noop(sprintf ' [%s] power - 0 power modules', $device->ip);
}
# build device module power info suitable for DBIC
my @devicepower;
foreach my $entry (keys %$p_watts) {
push @devicepower, {
module => $entry,
power => $p_watts->{$entry},
status => $p_status->{$entry},
};
}
my $interfaces = $snmp->interfaces;
my $p_ifindex = $snmp->peth_port_ifindex;
my $p_admin = $snmp->peth_port_admin;
my $p_pstatus = $snmp->peth_port_status;
my $p_class = $snmp->peth_port_class;
my $p_power = $snmp->peth_port_power;
# build device port power info suitable for DBIC
my @portpower;
foreach my $entry (keys %$p_ifindex) {
my $port = $interfaces->{ $p_ifindex->{$entry} };
next unless $port;
my ($module) = split m/\./, $entry;
push @portpower, {
port => $port,
module => $module,
admin => $p_admin->{$entry},
status => $p_pstatus->{$entry},
class => $p_class->{$entry},
power => $p_power->{$entry},
};
}
schema('netdisco')->txn_do(sub {
my $gone = $device->power_modules->delete;
debug sprintf ' [%s] power - removed %d power modules',
$device->ip, $gone;
$device->power_modules->populate(\@devicepower);
debug sprintf ' [%s] power - added %d new power modules',
$device->ip, scalar @devicepower;
});
schema('netdisco')->txn_do(sub {
my $gone = $device->powered_ports->delete;
debug sprintf ' [%s] power - removed %d PoE capable ports',
$device->ip, $gone;
$device->powered_ports->populate(\@portpower);
return Status->noop(sprintf ' [%s] power - added %d new PoE capable ports',
$device->ip, scalar @portpower);
});
});
true;

View File

@@ -0,0 +1,245 @@
package App::Netdisco::Worker::Plugin::Discover::Properties;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
use App::Netdisco::Transport::SNMP ();
use App::Netdisco::Util::Permission 'check_acl_no';
use App::Netdisco::Util::FastResolver 'hostnames_resolve_async';
use App::Netdisco::Util::DNS 'hostname_from_ip';
use Dancer::Plugin::DBIC 'schema';
use NetAddr::IP::Lite ':lower';
use Encode;
register_worker({ phase => 'early', driver => 'snmp' }, sub {
my ($job, $workerconf) = @_;
my $device = $job->device;
my $snmp = App::Netdisco::Transport::SNMP->reader_for($device)
or return Status->defer("discover failed: could not SNMP connect to $device");
my $ip_index = $snmp->ip_index;
my $interfaces = $snmp->interfaces;
my $ip_netmask = $snmp->ip_netmask;
# build device aliases suitable for DBIC
my @aliases;
foreach my $entry (keys %$ip_index) {
my $ip = NetAddr::IP::Lite->new($entry)
or next;
my $addr = $ip->addr;
next if $addr eq '0.0.0.0';
next if check_acl_no($ip, 'group:__LOCAL_ADDRESSES__');
next if setting('ignore_private_nets') and $ip->is_rfc1918;
my $iid = $ip_index->{$addr};
my $port = $interfaces->{$iid};
my $subnet = $ip_netmask->{$addr}
? NetAddr::IP::Lite->new($addr, $ip_netmask->{$addr})->network->cidr
: undef;
debug sprintf ' [%s] device - aliased as %s', $device->ip, $addr;
push @aliases, {
alias => $addr,
port => $port,
subnet => $subnet,
dns => undef,
};
}
debug sprintf ' resolving %d aliases with max %d outstanding requests',
scalar @aliases, $ENV{'PERL_ANYEVENT_MAX_OUTSTANDING_DNS'};
my $resolved_aliases = hostnames_resolve_async(\@aliases);
# fake one aliases entry for devices not providing ip_index
push @$resolved_aliases, { alias => $device->ip, dns => $device->dns }
if 0 == scalar @aliases;
# VTP Management Domain -- assume only one.
my $vtpdomains = $snmp->vtp_d_name;
my $vtpdomain;
if (defined $vtpdomains and scalar values %$vtpdomains) {
$device->set_column( vtp_domain => (values %$vtpdomains)[-1] );
}
my $hostname = hostname_from_ip($device->ip);
$device->set_column( dns => $hostname ) if $hostname;
my @properties = qw/
snmp_ver
description uptime name
layers ports mac
ps1_type ps2_type ps1_status ps2_status
fan slots
vendor os os_ver
/;
foreach my $property (@properties) {
$device->set_column( $property => $snmp->$property );
}
$device->set_column( model => Encode::decode('UTF-8', $snmp->model) );
$device->set_column( serial => Encode::decode('UTF-8', $snmp->serial) );
$device->set_column( contact => Encode::decode('UTF-8', $snmp->contact) );
$device->set_column( location => Encode::decode('UTF-8', $snmp->location) );
$device->set_column( snmp_class => $snmp->class );
$device->set_column( last_discover => \'now()' );
schema('netdisco')->txn_do(sub {
my $gone = $device->device_ips->delete;
debug sprintf ' [%s] device - removed %d aliases',
$device->ip, $gone;
$device->update_or_insert(undef, {for => 'update'});
$device->device_ips->populate($resolved_aliases);
debug sprintf ' [%s] device - added %d new aliases',
$device->ip, scalar @aliases;
});
return Status->done("Ended discover for $device");
});
register_worker({ phase => 'early', driver => 'snmp' }, sub {
my ($job, $workerconf) = @_;
my $device = $job->device;
return unless $device->in_storage;
my $snmp = App::Netdisco::Transport::SNMP->reader_for($device)
or return Status->defer("discover failed: could not SNMP connect to $device");
my $interfaces = $snmp->interfaces;
my $i_type = $snmp->i_type;
my $i_ignore = $snmp->i_ignore;
my $i_descr = $snmp->i_description;
my $i_mtu = $snmp->i_mtu;
my $i_speed = $snmp->i_speed;
my $i_mac = $snmp->i_mac;
my $i_up = $snmp->i_up;
my $i_up_admin = $snmp->i_up_admin;
my $i_name = $snmp->i_name;
my $i_duplex = $snmp->i_duplex;
my $i_duplex_admin = $snmp->i_duplex_admin;
my $i_stp_state = $snmp->i_stp_state;
my $i_vlan = $snmp->i_vlan;
my $i_lastchange = $snmp->i_lastchange;
my $agg_ports = $snmp->agg_ports;
# clear the cached uptime and get a new one
my $dev_uptime = $snmp->load_uptime;
if (!defined $dev_uptime) {
error sprintf ' [%s] interfaces - Error! Failed to get uptime from device!',
$device->ip;
return Status->error("discover failed: no uptime from device $device!");
}
# used to track how many times the device uptime wrapped
my $dev_uptime_wrapped = 0;
# use SNMP-FRAMEWORK-MIB::snmpEngineTime if available to
# fix device uptime if wrapped
if (defined $snmp->snmpEngineTime) {
$dev_uptime_wrapped = int( $snmp->snmpEngineTime * 100 / 2**32 );
if ($dev_uptime_wrapped > 0) {
info sprintf ' [%s] interface - device uptime wrapped %d times - correcting',
$device->ip, $dev_uptime_wrapped;
$device->uptime( $dev_uptime + $dev_uptime_wrapped * 2**32 );
}
}
# build device interfaces suitable for DBIC
my %interfaces;
foreach my $entry (keys %$interfaces) {
my $port = $interfaces->{$entry};
if (not $port) {
debug sprintf ' [%s] interfaces - ignoring %s (no port mapping)',
$device->ip, $entry;
next;
}
if (scalar grep {$port =~ m/^$_$/} @{setting('ignore_interfaces') || []}) {
debug sprintf
' [%s] interfaces - ignoring %s (%s) (config:ignore_interfaces)',
$device->ip, $entry, $port;
next;
}
if (exists $i_ignore->{$entry}) {
debug sprintf ' [%s] interfaces - ignoring %s (%s) (%s)',
$device->ip, $entry, $port, $i_type->{$entry};
next;
}
my $lc = $i_lastchange->{$entry} || 0;
if (not $dev_uptime_wrapped and $lc > $dev_uptime) {
info sprintf ' [%s] interfaces - device uptime wrapped (%s) - correcting',
$device->ip, $port;
$device->uptime( $dev_uptime + 2**32 );
$dev_uptime_wrapped = 1;
}
if ($device->is_column_changed('uptime') and $lc) {
if ($lc < $dev_uptime) {
# ambiguous: lastchange could be sysUptime before or after wrap
if ($dev_uptime > 30000 and $lc < 30000) {
# uptime wrap more than 5min ago but lastchange within 5min
# assume lastchange was directly after boot -> no action
}
else {
# uptime wrap less than 5min ago or lastchange > 5min ago
# to be on safe side, assume lastchange after counter wrap
debug sprintf
' [%s] interfaces - correcting LastChange for %s, assuming sysUptime wrap',
$device->ip, $port;
$lc += $dev_uptime_wrapped * 2**32;
}
}
}
$interfaces{$port} = {
port => $port,
descr => $i_descr->{$entry},
up => $i_up->{$entry},
up_admin => $i_up_admin->{$entry},
mac => $i_mac->{$entry},
speed => $i_speed->{$entry},
mtu => $i_mtu->{$entry},
name => Encode::decode('UTF-8', $i_name->{$entry}),
duplex => $i_duplex->{$entry},
duplex_admin => $i_duplex_admin->{$entry},
stp => $i_stp_state->{$entry},
type => $i_type->{$entry},
vlan => $i_vlan->{$entry},
pvid => $i_vlan->{$entry},
is_master => 'false',
slave_of => undef,
lastchange => $lc,
};
}
# must do this after building %interfaces so that we can set is_master
foreach my $sidx (keys %$agg_ports) {
my $slave = $interfaces->{$sidx} or next;
my $master = $interfaces->{ $agg_ports->{$sidx} } or next;
next unless exists $interfaces{$slave} and exists $interfaces{$master};
$interfaces{$slave}->{slave_of} = $master;
$interfaces{$master}->{is_master} = 'true';
}
schema('netdisco')->resultset('DevicePort')->txn_do_locked(sub {
my $gone = $device->ports->delete({keep_nodes => 1});
debug sprintf ' [%s] interfaces - removed %d interfaces',
$device->ip, $gone;
$device->update_or_insert(undef, {for => 'update'});
$device->ports->populate([values %interfaces]);
return Status->noop(sprintf ' [%s] interfaces - added %d new interfaces',
$device->ip, scalar values %interfaces);
});
});
true;

View File

@@ -0,0 +1,95 @@
package App::Netdisco::Worker::Plugin::Discover::VLANs;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
use App::Netdisco::Transport::SNMP ();
use Dancer::Plugin::DBIC 'schema';
register_worker({ phase => 'main', driver => 'snmp' }, sub {
my ($job, $workerconf) = @_;
my $device = $job->device;
return unless $device->in_storage;
my $snmp = App::Netdisco::Transport::SNMP->reader_for($device)
or return Status->defer("discover failed: could not SNMP connect to $device");
my $v_name = $snmp->v_name;
my $v_index = $snmp->v_index;
# build device vlans suitable for DBIC
my %v_seen = ();
my @devicevlans;
foreach my $entry (keys %$v_name) {
my $vlan = $v_index->{$entry};
next unless defined $vlan and $vlan;
++$v_seen{$vlan};
push @devicevlans, {
vlan => $vlan,
description => $v_name->{$entry},
last_discover => \'now()',
};
}
my $i_vlan = $snmp->i_vlan;
my $i_vlan_membership = $snmp->i_vlan_membership;
my $i_vlan_type = $snmp->i_vlan_type;
my $interfaces = $snmp->interfaces;
# build device port vlans suitable for DBIC
my @portvlans = ();
foreach my $entry (keys %$i_vlan_membership) {
my %port_vseen = ();
my $port = $interfaces->{$entry};
next unless defined $port;
my $type = $i_vlan_type->{$entry};
foreach my $vlan (@{ $i_vlan_membership->{$entry} }) {
next unless defined $vlan and $vlan;
next if ++$port_vseen{$vlan} > 1;
my $native = ((defined $i_vlan->{$entry}) and ($vlan eq $i_vlan->{$entry})) ? "t" : "f";
push @portvlans, {
port => $port,
vlan => $vlan,
native => $native,
vlantype => $type,
last_discover => \'now()',
};
next if $v_seen{$vlan};
# also add an unnamed vlan to the device
push @devicevlans, {
vlan => $vlan,
description => (sprintf "VLAN %d", $vlan),
last_discover => \'now()',
};
++$v_seen{$vlan};
}
}
schema('netdisco')->txn_do(sub {
my $gone = $device->vlans->delete;
debug sprintf ' [%s] vlans - removed %d device VLANs',
$device->ip, $gone;
$device->vlans->populate(\@devicevlans);
debug sprintf ' [%s] vlans - added %d new device VLANs',
$device->ip, scalar @devicevlans;
});
schema('netdisco')->txn_do(sub {
my $gone = $device->port_vlans->delete;
debug sprintf ' [%s] vlans - removed %d port VLANs',
$device->ip, $gone;
$device->port_vlans->populate(\@portvlans);
return Status->noop(sprintf ' [%s] vlans - added %d new port VLANs',
$device->ip, scalar @portvlans);
});
});
true;

View File

@@ -0,0 +1,85 @@
package App::Netdisco::Worker::Plugin::Discover::Wireless;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
use App::Netdisco::Transport::SNMP ();
use Dancer::Plugin::DBIC 'schema';
register_worker({ phase => 'main', driver => 'snmp' }, sub {
my ($job, $workerconf) = @_;
my $device = $job->device;
return unless $device->in_storage;
my $snmp = App::Netdisco::Transport::SNMP->reader_for($device)
or return Status->defer("discover failed: could not SNMP connect to $device");
my $ssidlist = $snmp->i_ssidlist;
return unless scalar keys %$ssidlist;
my $interfaces = $snmp->interfaces;
my $ssidbcast = $snmp->i_ssidbcast;
my $ssidmac = $snmp->i_ssidmac;
my $channel = $snmp->i_80211channel;
my $power = $snmp->dot11_cur_tx_pwr_mw;
# build device ssid list suitable for DBIC
my @ssids;
foreach my $entry (keys %$ssidlist) {
(my $iid = $entry) =~ s/\.\d+$//;
my $port = $interfaces->{$iid};
if (not $port) {
debug sprintf ' [%s] wireless - ignoring %s (no port mapping)',
$device->ip, $iid;
next;
}
push @ssids, {
port => $port,
ssid => $ssidlist->{$entry},
broadcast => $ssidbcast->{$entry},
bssid => $ssidmac->{$entry},
};
}
schema('netdisco')->txn_do(sub {
my $gone = $device->ssids->delete;
debug sprintf ' [%s] wireless - removed %d SSIDs',
$device->ip, $gone;
$device->ssids->populate(\@ssids);
debug sprintf ' [%s] wireless - added %d new SSIDs',
$device->ip, scalar @ssids;
});
# build device channel list suitable for DBIC
my @channels;
foreach my $entry (keys %$channel) {
my $port = $interfaces->{$entry};
if (not $port) {
debug sprintf ' [%s] wireless - ignoring %s (no port mapping)',
$device->ip, $entry;
next;
}
push @channels, {
port => $port,
channel => $channel->{$entry},
power => $power->{$entry},
};
}
schema('netdisco')->txn_do(sub {
my $gone = $device->wireless_ports->delete;
debug sprintf ' [%s] wireless - removed %d wireless channels',
$device->ip, $gone;
$device->wireless_ports->populate(\@channels);
return Status->noop(sprintf ' [%s] wireless - added %d new wireless channels',
$device->ip, scalar @channels);
});
});
true;

View File

@@ -0,0 +1,38 @@
package App::Netdisco::Worker::Plugin::Discover::WithNodes;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
use App::Netdisco::JobQueue 'jq_insert';
use Dancer::Plugin::DBIC 'schema';
register_worker({ phase => 'main' }, sub {
my ($job, $workerconf) = @_;
my $device = $job->device;
# if requested, and the device has not yet been
# arpniped/macsucked, queue those jobs now
return unless $device->in_storage
and $job->subaction and $job->subaction eq 'with-nodes';
if (!defined $device->last_macsuck and $device->has_layer(2)) {
jq_insert({
device => $device->ip,
action => 'macsuck',
username => $job->username,
userip => $job->userip,
});
}
if (!defined $device->last_arpnip and $device->has_layer(3)) {
jq_insert({
device => $device->ip,
action => 'arpnip',
username => $job->username,
userip => $job->userip,
});
}
});
true;

View File

@@ -0,0 +1,31 @@
package App::Netdisco::Worker::Plugin::DiscoverAll;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
use App::Netdisco::JobQueue qw/jq_queued jq_insert/;
use Dancer::Plugin::DBIC 'schema';
register_worker({ phase => 'main' }, sub {
my ($job, $workerconf) = @_;
my %queued = map {$_ => 1} jq_queued('discover');
my @devices = schema('netdisco')->resultset('Device')->search({
-or => [ 'vendor' => undef, 'vendor' => { '!=' => 'netdisco' }],
})->get_column('ip')->all;
my @filtered_devices = grep {!exists $queued{$_}} @devices;
jq_insert([
map {{
device => $_,
action => 'discover',
username => $job->username,
userip => $job->userip,
}} (@filtered_devices)
]);
return Status->done('Queued discover job for all devices');
});
true;

View File

@@ -0,0 +1,57 @@
package App::Netdisco::Worker::Plugin::Expire;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
use Dancer::Plugin::DBIC 'schema';
use App::Netdisco::Util::Statistics 'update_stats';
register_worker({ phase => 'main' }, sub {
my ($job, $workerconf) = @_;
if (setting('expire_devices') and setting('expire_devices') > 0) {
schema('netdisco')->txn_do(sub {
schema('netdisco')->resultset('Device')->search({
-or => [ 'vendor' => undef, 'vendor' => { '!=' => 'netdisco' }],
last_discover => \[q/< (now() - ?::interval)/,
(setting('expire_devices') * 86400)],
})->delete();
});
}
if (setting('expire_nodes') and setting('expire_nodes') > 0) {
schema('netdisco')->txn_do(sub {
schema('netdisco')->resultset('Node')->search({
time_last => \[q/< (now() - ?::interval)/,
(setting('expire_nodes') * 86400)],
})->delete();
});
}
if (setting('expire_nodes_archive') and setting('expire_nodes_archive') > 0) {
schema('netdisco')->txn_do(sub {
schema('netdisco')->resultset('Node')->search({
-not_bool => 'active',
time_last => \[q/< (now() - ?::interval)/,
(setting('expire_nodes_archive') * 86400)],
})->delete();
});
}
if (setting('expire_jobs') and setting('expire_jobs') > 0) {
schema('netdisco')->txn_do(sub {
schema('netdisco')->resultset('Admin')->search({
entered => \[q/< (now() - ?::interval)/,
(setting('expire_jobs') * 86400)],
})->delete();
});
}
# now update stats
update_stats();
return Status->done('Checked expiry and updated stats');
});
true;

View File

@@ -0,0 +1,30 @@
package App::Netdisco::Worker::Plugin::ExpireNodes;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
use Dancer::Plugin::DBIC 'schema';
register_worker({ phase => 'check' }, sub {
return Status->error('Missing device (-d).')
unless defined shift->device;
return Status->done('ExpireNodes is able to run');
});
register_worker({ phase => 'main' }, sub {
my ($job, $workerconf) = @_;
schema('netdisco')->txn_do(sub {
schema('netdisco')->resultset('Node')->search({
switch => $job->device->ip,
($job->port ? (port => $job->port) : ()),
})->delete(
($job->extra ? () : ({ archive_nodes => 1 }))
);
});
return Status->done('Expired nodes for '. $job->device->ip);
});
true;

View File

@@ -0,0 +1,14 @@
package App::Netdisco::Worker::Plugin::Graph;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
use App::Netdisco::Util::Graph ();
register_worker({ phase => 'main' }, sub {
App::Netdisco::Util::Graph::graph();
return Status->done('Generated graph data');
});
true;

View File

@@ -0,0 +1,43 @@
package App::Netdisco::Worker::Plugin::Location;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
use App::Netdisco::Transport::SNMP;
register_worker({ phase => 'check' }, sub {
return Status->error('Missing device (-d).')
unless defined shift->device;
return Status->done('Location is able to run');
});
register_worker({ phase => 'main', driver => 'snmp' }, sub {
my ($job, $workerconf) = @_;
my ($device, $data) = map {$job->$_} qw/device extra/;
# snmp connect using rw community
my $snmp = App::Netdisco::Transport::SNMP->writer_for($device)
or return Status->defer("failed to connect to $device to update location");
my $rv = $snmp->set_location($data);
if (!defined $rv) {
return Status->error(
"failed to set location on $device: ". ($snmp->error || ''));
}
# confirm the set happened
$snmp->clear_cache;
my $new_data = ($snmp->location || '');
if ($new_data ne $data) {
return Status->error("verify of location failed on $device: $new_data");
}
# update netdisco DB
$device->update({location => $data});
return Status->done("Updated location on $device to [$data]");
});
true;

View File

@@ -0,0 +1,31 @@
package App::Netdisco::Worker::Plugin::Macsuck;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
use App::Netdisco::Util::Device 'is_macsuckable_now';
register_worker({ phase => 'check' }, sub {
my ($job, $workerconf) = @_;
my $device = $job->device;
return Status->error('macsuck failed: unable to interpret device param')
unless defined $device;
return Status->error("macsuck skipped: $device not yet discovered")
unless $device->in_storage;
return Status->defer("macsuck skipped: $device is pseudo-device")
if $device->is_pseudo;
return Status->defer("arpnip skipped: $device has no layer 2 capability")
unless $device->has_layer(2);
return Status->defer("macsuck deferred: $device is not macsuckable")
unless is_macsuckable_now($device);
return Status->done('Macsuck is able to run.');
});
true;

View File

@@ -0,0 +1,429 @@
package App::Netdisco::Worker::Plugin::Macsuck::Nodes;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
use App::Netdisco::Transport::SNMP ();
use App::Netdisco::Util::Permission 'check_acl_no';
use App::Netdisco::Util::PortMAC 'get_port_macs';
use App::Netdisco::Util::Device 'match_devicetype';
use App::Netdisco::Util::Node 'check_mac';
use App::Netdisco::Util::SNMP 'snmp_comm_reindex';
use Dancer::Plugin::DBIC 'schema';
use Time::HiRes 'gettimeofday';
use Scope::Guard 'guard';
register_worker({ phase => 'main', driver => 'snmp' }, sub {
my ($job, $workerconf) = @_;
my $device = $job->device;
my $snmp = App::Netdisco::Transport::SNMP->reader_for($device)
or return Status->defer("macsuck failed: could not SNMP connect to $device");
# would be possible just to use now() on updated records, but by using this
# same value for them all, we can if we want add a job at the end to
# select and do something with the updated set (see set archive, below)
my $now = 'to_timestamp('. (join '.', gettimeofday) .')';
my $total_nodes = 0;
# cache the device ports to save hitting the database for many single rows
my $device_ports = {map {($_->port => $_)}
$device->ports(undef, {prefetch => {neighbor_alias => 'device'}})->all};
my $port_macs = get_port_macs();
my $interfaces = $snmp->interfaces;
# get forwarding table data via basic snmp connection
my $fwtable = walk_fwtable($device, $interfaces, $port_macs, $device_ports);
# ...then per-vlan if supported
my @vlan_list = get_vlan_list($device);
{
my $guard = guard { snmp_comm_reindex($snmp, $device, 0) };
foreach my $vlan (@vlan_list) {
snmp_comm_reindex($snmp, $device, $vlan);
my $pv_fwtable =
walk_fwtable($device, $interfaces, $port_macs, $device_ports, $vlan);
$fwtable = {%$fwtable, %$pv_fwtable};
}
}
# now it's time to call store_node for every node discovered
# on every port on every vlan on this device.
# reverse sort allows vlan 0 entries to be included only as fallback
foreach my $vlan (reverse sort keys %$fwtable) {
foreach my $port (keys %{ $fwtable->{$vlan} }) {
debug sprintf ' [%s] macsuck - port %s vlan %s : %s nodes',
$device->ip, $port, $vlan, scalar keys %{ $fwtable->{$vlan}->{$port} };
# make sure this port is UP in netdisco (unless it's a lag master,
# because we can still see nodes without a functioning aggregate)
$device_ports->{$port}->update({up_admin => 'up', up => 'up'})
if not $device_ports->{$port}->is_master;
foreach my $mac (keys %{ $fwtable->{$vlan}->{$port} }) {
# remove vlan 0 entry for this MAC addr
delete $fwtable->{0}->{$_}->{$mac}
for keys %{ $fwtable->{0} };
++$total_nodes;
store_node($device->ip, $vlan, $port, $mac, $now);
}
}
}
debug sprintf ' [%s] macsuck - %s updated forwarding table entries',
$device->ip, $total_nodes;
# a use for $now ... need to archive dissapeared nodes
my $archived = 0;
if (setting('node_freshness')) {
$archived = schema('netdisco')->resultset('Node')->search({
switch => $device->ip,
time_last => \[ "< ($now - ?::interval)",
setting('node_freshness') .' minutes' ],
})->update({ active => \'false' });
}
debug sprintf ' [%s] macsuck - removed %d fwd table entries to archive',
$device->ip, $archived;
$device->update({last_macsuck => \$now});
return Status->done("Ended macsuck for $device");
});
=head2 store_node( $ip, $vlan, $port, $mac, $now? )
Writes a fresh entry to the Netdisco C<node> database table. Will mark old
entries for this data as no longer C<active>.
All four fields in the tuple are required. If you don't know the VLAN ID,
Netdisco supports using ID "0".
Optionally, a fifth argument can be the literal string passed to the time_last
field of the database record. If not provided, it defauls to C<now()>.
=cut
sub store_node {
my ($ip, $vlan, $port, $mac, $now) = @_;
$now ||= 'now()';
$vlan ||= 0;
schema('netdisco')->txn_do(sub {
my $nodes = schema('netdisco')->resultset('Node');
my $old = $nodes->search(
{ mac => $mac,
# where vlan is unknown, need to archive on all other vlans
($vlan ? (vlan => $vlan) : ()),
-bool => 'active',
-not => {
switch => $ip,
port => $port,
},
})->update( { active => \'false' } );
# new data
$nodes->update_or_create(
{
switch => $ip,
port => $port,
vlan => $vlan,
mac => $mac,
active => \'true',
oui => substr($mac,0,8),
time_last => \$now,
(($old != 0) ? (time_recent => \$now) : ()),
},
{
key => 'primary',
for => 'update',
}
);
});
}
# return a list of vlan numbers which are OK to macsuck on this device
sub get_vlan_list {
my $device = shift;
my $snmp = App::Netdisco::Transport::SNMP->reader_for($device)
or return (); # already checked!
return () unless $snmp->cisco_comm_indexing;
my (%vlans, %vlan_names);
my $i_vlan = $snmp->i_vlan || {};
my $trunks = $snmp->i_vlan_membership || {};
my $i_type = $snmp->i_type || {};
# get list of vlans in use
while (my ($idx, $vlan) = each %$i_vlan) {
# hack: if vlan id comes as 1.142 instead of 142
$vlan =~ s/^\d+\.//;
# VLANs are ports interfaces capture VLAN, but don't count as in use
# Port channels are also 'propVirtual', but capture while checking
# trunk VLANs below
if (exists $i_type->{$idx} and $i_type->{$idx} eq 'propVirtual') {
$vlans{$vlan} ||= 0;
}
else {
++$vlans{$vlan};
}
foreach my $t_vlan (@{$trunks->{$idx}}) {
++$vlans{$t_vlan};
}
}
unless (scalar keys %vlans) {
debug sprintf ' [%s] macsuck - no VLANs found.', $device->ip;
return ();
}
my $v_name = $snmp->v_name || {};
# get vlan names (required for config which filters by name)
while (my ($idx, $name) = each %$v_name) {
# hack: if vlan id comes as 1.142 instead of 142
(my $vlan = $idx) =~ s/^\d+\.//;
# just in case i_vlan is different to v_name set
# capture the VLAN, but it's not in use on a port
$vlans{$vlan} ||= 0;
$vlan_names{$vlan} = $name;
}
debug sprintf ' [%s] macsuck - VLANs: %s', $device->ip,
(join ',', sort keys %vlans);
my @ok_vlans = ();
foreach my $vlan (sort keys %vlans) {
my $name = $vlan_names{$vlan} || '(unnamed)';
if (ref [] eq ref setting('macsuck_no_vlan')) {
my $ignore = setting('macsuck_no_vlan');
if ((scalar grep {$_ eq $vlan} @$ignore) or
(scalar grep {$_ eq $name} @$ignore)) {
debug sprintf
' [%s] macsuck VLAN %s - skipped by macsuck_no_vlan config',
$device->ip, $vlan;
next;
}
}
if (ref [] eq ref setting('macsuck_no_devicevlan')) {
my $ignore = setting('macsuck_no_devicevlan');
my $ip = $device->ip;
if ((scalar grep {$_ eq "$ip:$vlan"} @$ignore) or
(scalar grep {$_ eq "$ip:$name"} @$ignore)) {
debug sprintf
' [%s] macsuck VLAN %s - skipped by macsuck_no_devicevlan config',
$device->ip, $vlan;
next;
}
}
if (setting('macsuck_no_unnamed') and $name eq '(unnamed)') {
debug sprintf
' [%s] macsuck VLAN %s - skipped by macsuck_no_unnamed config',
$device->ip, $vlan;
next;
}
if ($vlan == 0 or $vlan > 4094) {
debug sprintf ' [%s] macsuck - invalid VLAN number %s',
$device->ip, $vlan;
next;
}
# check in use by a port on this device
if (!$vlans{$vlan} && !setting('macsuck_all_vlans')) {
debug sprintf
' [%s] macsuck VLAN %s/%s - not in use by any port - skipping.',
$device->ip, $vlan, $name;
next;
}
push @ok_vlans, $vlan;
}
return @ok_vlans;
}
# walks the forwarding table (BRIDGE-MIB) for the device and returns a
# table of node entries.
sub walk_fwtable {
my ($device, $interfaces, $port_macs, $device_ports, $comm_vlan) = @_;
my $skiplist = {}; # ports through which we can see another device
my $cache = {};
my $snmp = App::Netdisco::Transport::SNMP->reader_for($device)
or return $cache; # already checked!
my $fw_mac = $snmp->fw_mac;
my $fw_port = $snmp->fw_port;
my $fw_vlan = $snmp->qb_fw_vlan;
my $bp_index = $snmp->bp_index;
# to map forwarding table port to device port we have
# fw_port -> bp_index -> interfaces
while (my ($idx, $mac) = each %$fw_mac) {
my $bp_id = $fw_port->{$idx};
next unless check_mac($device, $mac);
unless (defined $bp_id) {
debug sprintf
' [%s] macsuck %s - %s has no fw_port mapping - skipping.',
$device->ip, $mac, $idx;
next;
}
my $iid = $bp_index->{$bp_id};
unless (defined $iid) {
debug sprintf
' [%s] macsuck %s - port %s has no bp_index mapping - skipping.',
$device->ip, $mac, $bp_id;
next;
}
my $port = $interfaces->{$iid};
unless (defined $port) {
debug sprintf
' [%s] macsuck %s - iid %s has no port mapping - skipping.',
$device->ip, $mac, $iid;
next;
}
if (exists $skiplist->{$port}) {
debug sprintf
' [%s] macsuck %s - seen another device thru port %s - skipping.',
$device->ip, $mac, $port;
next;
}
# this uses the cached $ports resultset to limit hits on the db
my $device_port = $device_ports->{$port};
unless (defined $device_port) {
debug sprintf
' [%s] macsuck %s - port %s is not in database - skipping.',
$device->ip, $mac, $port;
next;
}
my $vlan = $fw_vlan->{$idx} || $comm_vlan || '0';
# check to see if the port is connected to another device
# and if we have that device in the database.
# we have several ways to detect "uplink" port status:
# * a neighbor was discovered using CDP/LLDP
# * a mac addr is seen which belongs to any device port/interface
# * (TODO) admin sets is_uplink_admin on the device_port
# allow to gather MACs on upstream port for some kinds of device that
# do not expose MAC address tables via SNMP. relies on prefetched
# neighbors otherwise it would kill the DB with device lookups.
my $neigh_cannot_macsuck = eval { # can fail
check_acl_no(($device_port->neighbor || "0 but true"), 'macsuck_unsupported') ||
match_devicetype($device_port->remote_type, 'macsuck_unsupported_type') };
if ($device_port->is_uplink) {
if ($neigh_cannot_macsuck) {
debug sprintf
' [%s] macsuck %s - port %s neighbor %s without macsuck support',
$device->ip, $mac, $port,
(eval { $device_port->neighbor->ip }
|| ($device_port->remote_ip
|| $device_port->remote_id || '?'));
# continue!!
}
elsif (my $neighbor = $device_port->neighbor) {
debug sprintf
' [%s] macsuck %s - port %s has neighbor %s - skipping.',
$device->ip, $mac, $port, $neighbor->ip;
next;
}
elsif (my $remote = $device_port->remote_ip) {
debug sprintf
' [%s] macsuck %s - port %s has undiscovered neighbor %s',
$device->ip, $mac, $port, $remote;
# continue!!
}
elsif (not setting('macsuck_bleed')) {
debug sprintf
' [%s] macsuck %s - port %s is detected uplink - skipping.',
$device->ip, $mac, $port;
$skiplist->{$port} = [ $vlan, $mac ] # remember for later
if exists $port_macs->{$mac};
next;
}
}
if (exists $port_macs->{$mac}) {
my $switch_ip = $port_macs->{$mac};
if ($device->ip eq $switch_ip) {
debug sprintf
' [%s] macsuck %s - port %s connects to self - skipping.',
$device->ip, $mac, $port;
next;
}
debug sprintf ' [%s] macsuck %s - port %s is probably an uplink',
$device->ip, $mac, $port;
$device_port->update({is_uplink => \'true'});
# neighbor exists and Netdisco can speak to it, so we don't want
# its MAC address. however don't add to skiplist as that would
# clear all other MACs on the port.
next if $neigh_cannot_macsuck;
# when there's no CDP/LLDP, we only want to gather macs at the
# topology edge, hence skip ports with known device macs.
if (not setting('macsuck_bleed')) {
debug sprintf ' [%s] macsuck %s - adding port %s to skiplist',
$device->ip, $mac, $port;
$skiplist->{$port} = [ $vlan, $mac ]; # remember for later
next;
}
}
# possibly move node to lag master
if (defined $device_port->slave_of
and exists $device_ports->{$device_port->slave_of}) {
$port = $device_port->slave_of;
$device_ports->{$port}->update({is_uplink => \'true'});
}
++$cache->{$vlan}->{$port}->{$mac};
}
# restore MACs of neighbor devices.
# this is when we have a "possible uplink" detected but we still want to
# record the single MAC of the neighbor device so it works in Node search.
foreach my $port (keys %$skiplist) {
my ($vlan, $mac) = @{ $skiplist->{$port} };
delete $cache->{$_}->{$port} for keys %$cache; # nuke nodes on all VLANs
++$cache->{$vlan}->{$port}->{$mac};
}
return $cache;
}
true;

View File

@@ -0,0 +1,82 @@
package App::Netdisco::Worker::Plugin::Macsuck::WirelessNodes;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
use App::Netdisco::Transport::SNMP ();
use Dancer::Plugin::DBIC 'schema';
use Time::HiRes 'gettimeofday';
register_worker({ phase => 'main', driver => 'snmp' }, sub {
my ($job, $workerconf) = @_;
my $device = $job->device;
my $snmp = App::Netdisco::Transport::SNMP->reader_for($device)
or return Status->defer("macsuck failed: could not SNMP connect to $device");
my $now = 'to_timestamp('. (join '.', gettimeofday) .')';
my $cd11_txrate = $snmp->cd11_txrate;
return unless $cd11_txrate and scalar keys %$cd11_txrate;
if (setting('store_wireless_clients')) {
debug sprintf ' [%s] macsuck - gathering wireless client info',
$device->ip;
}
else {
return Status->noop(sprintf ' [%s] macsuck - dot11 info available but skipped due to config',
$device->ip);
}
my $cd11_rateset = $snmp->cd11_rateset();
my $cd11_uptime = $snmp->cd11_uptime();
my $cd11_sigstrength = $snmp->cd11_sigstrength();
my $cd11_sigqual = $snmp->cd11_sigqual();
my $cd11_mac = $snmp->cd11_mac();
my $cd11_port = $snmp->cd11_port();
my $cd11_rxpkt = $snmp->cd11_rxpkt();
my $cd11_txpkt = $snmp->cd11_txpkt();
my $cd11_rxbyte = $snmp->cd11_rxbyte();
my $cd11_txbyte = $snmp->cd11_txbyte();
my $cd11_ssid = $snmp->cd11_ssid();
while (my ($idx, $txrates) = each %$cd11_txrate) {
my $rates = $cd11_rateset->{$idx};
my $mac = $cd11_mac->{$idx};
next unless defined $mac; # avoid null entries
# there can be more rows in txrate than other tables
my $txrate = defined $txrates->[$#$txrates]
? int($txrates->[$#$txrates])
: undef;
my $maxrate = defined $rates->[$#$rates]
? int($rates->[$#$rates])
: undef;
my $ssid = $cd11_ssid->{$idx} || 'unknown';
schema('netdisco')->txn_do(sub {
schema('netdisco')->resultset('NodeWireless')
->search({ 'me.mac' => $mac, 'me.ssid' => $ssid })
->update_or_create({
txrate => $txrate,
maxrate => $maxrate,
uptime => $cd11_uptime->{$idx},
rxpkt => $cd11_rxpkt->{$idx},
txpkt => $cd11_txpkt->{$idx},
rxbyte => $cd11_rxbyte->{$idx},
txbyte => $cd11_txbyte->{$idx},
sigqual => $cd11_sigqual->{$idx},
sigstrength => $cd11_sigstrength->{$idx},
time_last => \$now,
}, {
order_by => [qw/mac ssid/],
for => 'update',
});
});
}
});
true;

View File

@@ -0,0 +1,31 @@
package App::Netdisco::Worker::Plugin::Macwalk;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
use App::Netdisco::JobQueue qw/jq_queued jq_insert/;
use Dancer::Plugin::DBIC 'schema';
register_worker({ phase => 'main' }, sub {
my ($job, $workerconf) = @_;
my %queued = map {$_ => 1} jq_queued('macsuck');
my @devices = schema('netdisco')->resultset('Device')->search({
-or => [ 'vendor' => undef, 'vendor' => { '!=' => 'netdisco' }],
})->has_layer('2')->get_column('ip')->all;
my @filtered_devices = grep {!exists $queued{$_}} @devices;
jq_insert([
map {{
device => $_,
action => 'macsuck',
username => $job->username,
userip => $job->userip,
}} (@filtered_devices)
]);
return Status->done('Queued macsuck job for all devices');
});
true;

View File

@@ -0,0 +1,14 @@
package App::Netdisco::Worker::Plugin::Monitor;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
use App::Netdisco::Util::NodeMonitor ();
register_worker({ phase => 'main' }, sub {
App::Netdisco::Util::NodeMonitor::monitor();
return Status->done('Generated monitor data');
});
true;

View File

@@ -0,0 +1,21 @@
package App::Netdisco::Worker::Plugin::Nbtstat;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
use App::Netdisco::Util::Device 'is_macsuckable';
register_worker({ phase => 'check' }, sub {
my ($job, $workerconf) = @_;
return Status->error('nbtstat failed: unable to interpret device param')
unless defined $job->device;
return Status->defer(sprintf 'nbtstat deferred: %s is not macsuckable', $job->device->ip)
unless is_macsuckable($job->device);
return Status->done('Nbtstat is able to run.');
});
true;

View File

@@ -0,0 +1,50 @@
package App::Netdisco::Worker::Plugin::Nbtstat::Core;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
use App::Netdisco::Util::Nbtstat qw/nbtstat_resolve_async store_nbt/;
use App::Netdisco::Util::Node 'is_nbtstatable';
use Dancer::Plugin::DBIC 'schema';
use Time::HiRes 'gettimeofday';
register_worker({ phase => 'main' }, sub {
my ($job, $workerconf) = @_;
my $host = $job->device->ip;
# get list of nodes on device
my $interval = (setting('nbtstat_max_age') || 7) . ' day';
my $rs = schema('netdisco')->resultset('NodeIp')->search({
-bool => 'me.active',
-bool => 'nodes.active',
'nodes.switch' => $host,
'me.time_last' => \[ '>= now() - ?::interval', $interval ],
},{
join => 'nodes',
columns => 'ip',
distinct => 1,
})->ip_version(4);
my @ips = map {+{'ip' => $_}}
grep { is_nbtstatable( $_ ) }
$rs->get_column('ip')->all;
# Unless we have IPs don't bother
if (scalar @ips) {
my $now = 'to_timestamp('. (join '.', gettimeofday) .')';
my $resolved_nodes = nbtstat_resolve_async(\@ips);
# update node_nbt with status entries
foreach my $result (@$resolved_nodes) {
if (defined $result->{'nbname'}) {
store_nbt($result, $now);
}
}
}
return Status->done("Ended nbtstat for $host");
});
true;

View File

@@ -0,0 +1,31 @@
package App::Netdisco::Worker::Plugin::Nbtwalk;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
use App::Netdisco::JobQueue qw/jq_queued jq_insert/;
use Dancer::Plugin::DBIC 'schema';
register_worker({ phase => 'main' }, sub {
my ($job, $workerconf) = @_;
my %queued = map {$_ => 1} jq_queued('nbtstat');
my @devices = schema('netdisco')->resultset('Device')->search({
-or => [ 'vendor' => undef, 'vendor' => { '!=' => 'netdisco' }],
})->has_layer('2')->get_column('ip')->all;
my @filtered_devices = grep {!exists $queued{$_}} @devices;
jq_insert([
map {{
device => $_,
action => 'nbtstat',
username => $job->username,
userip => $job->userip,
}} (@filtered_devices)
]);
return Status->done('Queued nbtstat job for all devices');
});
true;

View File

@@ -0,0 +1,76 @@
package App::Netdisco::Worker::Plugin::PortControl;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
use App::Netdisco::Transport::SNMP;
use App::Netdisco::Util::Port ':all';
register_worker({ phase => 'check' }, sub {
my ($job, $workerconf) = @_;
return Status->error('Missing device (-d).') unless defined $job->device;
return Status->error('Missing port (-p).') unless defined $job->port;
return Status->error('Missing status (-e).') unless defined $job->subaction;
vars->{'port'} = get_port($job->device, $job->port)
or return Status->error(sprintf "Unknown port name [%s] on device %s",
$job->port, $job->device);
my $vlan_reconfig_check = vlan_reconfig_check(vars->{'port'});
return Status->error("Cannot alter vlan: $vlan_reconfig_check")
if $vlan_reconfig_check;
return Status->done('PortControl is able to run');
});
register_worker({ phase => 'main', driver => 'snmp' }, sub {
my ($job, $workerconf) = @_;
my ($device, $pn) = map {$job->$_} qw/device port/;
# need to remove "-other" which appears for power/portcontrol
(my $sa = $job->subaction) =~ s/-\w+//;
$job->subaction($sa);
if ($sa eq 'bounce') {
$job->subaction('down');
my $status = _action($job);
return $status if $status->not_ok;
$job->subaction('up');
}
return _action($job);
});
sub _action {
my $job = shift;
my ($device, $pn, $data) = map {$job->$_} qw/device port subaction/;
# snmp connect using rw community
my $snmp = App::Netdisco::Transport::SNMP->writer_for($device)
or return Status->defer("failed to connect to $device to update up_admin");
my $iid = get_iid($snmp, vars->{'port'})
or return Status->error("Failed to get port ID for [$pn] from $device");
my $rv = $snmp->set_i_up_admin($data, $iid);
if (!defined $rv) {
return Status->error(sprintf 'Failed to set [%s] up_admin to [%s] on $device: %s',
$pn, $data, ($snmp->error || ''));
}
# confirm the set happened
$snmp->clear_cache;
my $state = ($snmp->i_up_admin($iid) || '');
if (ref {} ne ref $state or $state->{$iid} ne $data) {
return Status->error("Verify of [$pn] up_admin failed on $device");
}
# update netdisco DB
vars->{'port'}->update({up_admin => $data});
return Status->done("Updated [$pn] up_admin on [$device] to [$data]");
}
true;

View File

@@ -0,0 +1,54 @@
package App::Netdisco::Worker::Plugin::PortName;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
use App::Netdisco::Transport::SNMP;
use App::Netdisco::Util::Port ':all';
register_worker({ phase => 'check' }, sub {
my ($job, $workerconf) = @_;
return Status->error('Missing device (-d).') unless defined $job->device;
return Status->error('Missing port (-p).') unless defined $job->port;
return Status->error('Missing name (-e).') unless defined $job->subaction;
vars->{'port'} = get_port($job->device, $job->port)
or return Status->error(sprintf "Unknown port name [%s] on device %s",
$job->port, $job->device);
return Status->done('PortName is able to run');
});
register_worker({ phase => 'main', driver => 'snmp' }, sub {
my ($job, $workerconf) = @_;
my ($device, $pn, $data) = map {$job->$_} qw/device port extra/;
# snmp connect using rw community
my $snmp = App::Netdisco::Transport::SNMP->writer_for($device)
or return Status->defer("failed to connect to $device to update alias");
my $iid = get_iid($snmp, vars->{'port'})
or return Status->error("Failed to get port ID for [$pn] from $device");
my $rv = $snmp->set_i_alias($data, $iid);
if (!defined $rv) {
return Status->error(sprintf 'Failed to set [%s] alias to [%s] on $device: %s',
$pn, $data, ($snmp->error || ''));
}
# confirm the set happened
$snmp->clear_cache;
my $state = ($snmp->i_alias($iid) || '');
if (ref {} ne ref $state or $state->{$iid} ne $data) {
return Status->error("Verify of [$pn] alias failed on $device");
}
# update netdisco DB
vars->{'port'}->update({name => $data});
return Status->done("Updated [$pn] alias on [$device] to [$data]");
});
true;

View File

@@ -0,0 +1,69 @@
package App::Netdisco::Worker::Plugin::Power;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
use App::Netdisco::Transport::SNMP;
use App::Netdisco::Util::Port ':all';
register_worker({ phase => 'check' }, sub {
my ($job, $workerconf) = @_;
return Status->error('Missing device (-d).') unless defined $job->device;
return Status->error('Missing port (-p).') unless defined $job->port;
return Status->error('Missing status (-e).') unless defined $job->subaction;
vars->{'port'} = get_port($job->device, $job->port)
or return Status->error(sprintf "Unknown port name [%s] on device %s",
$job->port, $job->device);
my $vlan_reconfig_check = vlan_reconfig_check(vars->{'port'});
return Status->error("Cannot alter vlan: $vlan_reconfig_check")
if $vlan_reconfig_check;
return Status->error("No PoE service on port [$pn] on device $device")
unless vars->{'port'}->power;
return Status->done('Power is able to run');
});
register_worker({ phase => 'main', driver => 'snmp' }, sub {
my ($job, $workerconf) = @_;
my ($device, $pn) = map {$job->$_} qw/device port/;
# munge data
(my $data = $job->subaction) =~ s/-\w+//; # remove -other
$data = 'true' if $data =~ m/^(on|yes|up)$/;
$data = 'false' if $data =~ m/^(off|no|down)$/;
# snmp connect using rw community
my $snmp = App::Netdisco::Transport::SNMP->writer_for($device)
or return Status->defer("failed to connect to $device to update vlan");
my $powerid = get_powerid($snmp, vars->{'port'})
or return Status->error("failed to get power ID for [$pn] from $device");
my $rv = $snmp->set_peth_port_admin($data, $powerid);
if (!defined $rv) {
return Status->error(sprintf 'failed to set [%s] power to [%s] on [%s]: %s',
$pn, $data, $device, ($snmp->error || ''));
}
# confirm the set happened
$snmp->clear_cache;
my $state = ($snmp->peth_port_admin($powerid) || '');
if (ref {} ne ref $state or $state->{$powerid} ne $data) {
return Status->error("Verify of [$pn] power failed on $device");
}
# update netdisco DB
vars->{'port'}->power->update({
admin => $data,
status => ($data eq 'false' ? 'disabled' : 'searching'),
});
return Status->done("Updated [$pn] power status on $device to [$data]");
});
true;

View File

@@ -0,0 +1,39 @@
package App::Netdisco::Worker::Plugin::Psql;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
register_worker({ phase => 'main' }, sub {
my ($job, $workerconf) = @_;
my ($device, $port, $extra) = map {$job->$_} qw/device port extra/;
my $name = ($ENV{NETDISCO_DBNAME} || setting('database')->{name} || 'netdisco');
my $host = setting('database')->{host};
my $user = setting('database')->{user};
my $pass = setting('database')->{pass};
my $portnum = undef;
if ($host and $host =~ m/([^;]+);port=(\d+)/) {
$host = $1;
$portnum = $2;
}
$ENV{PGHOST} = $host if $host;
$ENV{PGPORT} = $portnum if defined $portnum;
$ENV{PGDATABASE} = $name;
$ENV{PGUSER} = $user;
$ENV{PGPASSWORD} = $pass;
$ENV{PGCLIENTENCODING} = 'UTF8';
if ($extra) {
system('psql', '-c', $extra);
}
else {
system('psql');
}
return Status->done('psql session closed.');
});
true;

View File

@@ -0,0 +1,39 @@
package App::Netdisco::Worker::Plugin::Renumber;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
use NetAddr::IP qw/:rfc3021 :lower/;
use App::Netdisco::Util::Device qw/get_device renumber_device/;
register_worker({ phase => 'check' }, sub {
return Status->error('Missing device (-d).')
unless defined shift->device;
my $new_ip = NetAddr::IP->new($job->extra);
unless ($new_ip and $new_ip->addr ne '0.0.0.0') {
return Status->error("Bad host or IP: ".($job->extra || '0.0.0.0'));
}
return Status->done('Renumber is able to run');
});
register_worker({ phase => 'main' }, sub {
my ($job, $workerconf) = @_;
my ($device, $port, $extra) = map {$job->$_} qw/device port extra/;
my $old_ip = $device->ip;
my $new_ip = NetAddr::IP->new($extra);
my $new_dev = get_device($new_ip->addr);
if ($new_dev and $new_dev->in_storage and ($new_dev->ip ne $device->ip)) {
return Status->error(sprintf "Already know new device as: %s.", $new_dev->ip);
}
renumber_device($device, $new_ip);
return Status->done(sprintf 'Renumbered device %s to %s (%s).',
$device->ip, $new_ip, ($device->dns || ''));
});
true;

View File

@@ -0,0 +1,37 @@
package App::Netdisco::Worker::Plugin::Show;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
use Data::Printer ();
use App::Netdisco::Transport::SNMP;
register_worker({ phase => 'check' }, sub {
return Status->error('Missing device (-d).')
unless defined shift->device;
return Status->done('Show is able to run');
});
register_worker({ phase => 'main', driver => 'snmp' }, sub {
my ($job, $workerconf) = @_;
my ($device, $port, $extra) = map {$job->$_} qw/device port extra/;
$extra ||= 'interfaces'; my $class = undef;
($class, $extra) = split(/::([^:]+)$/, $extra);
if ($class and $extra) {
$class = 'SNMP::Info::'.$class;
}
else {
$extra = $class;
undef $class;
}
my $i = App::Netdisco::Transport::SNMP->reader_for($device, $class);
Data::Printer::p($i->$extra($port));
return Status->done(
sprintf "Showed %s response from %s", $extra, $device->ip);
});
true;

View File

@@ -0,0 +1,14 @@
package App::Netdisco::Worker::Plugin::Stats;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
use App::Netdisco::Util::Statistics ();
register_worker({ phase => 'main' }, sub {
App::Netdisco::Util::Statistics::update_stats();
return Status->done('Updated statistics');
});
true;

View File

@@ -0,0 +1,30 @@
package App::Netdisco::Worker::Plugin::Vlan;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
use App::Netdisco::Util::Port ':all';
register_worker({ phase => 'check' }, sub {
my ($job, $workerconf) = @_;
my ($device, $pn, $data) = map {$job->$_} qw/device port extra/;
return Status->error('Missing device (-d).') if !defined $device;
return Status->error('Missing port (-p).') if !defined $pn;
return Status->error('Missing vlan (-e).') if !defined $data;
vars->{'port'} = get_port($device, $pn)
or return Status->error("Unknown port name [$pn] on device $device");
my $port_reconfig_check = port_reconfig_check(vars->{'port'});
return Status->error("Cannot alter port: $port_reconfig_check")
if $port_reconfig_check;
my $vlan_reconfig_check = vlan_reconfig_check(vars->{'port'});
return Status->error("Cannot alter vlan: $vlan_reconfig_check")
if $vlan_reconfig_check;
return Status->done("Vlan is able to run.");
});
true;

View File

@@ -0,0 +1,61 @@
package App::Netdisco::Worker::Plugin::Vlan::Native;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
use App::Netdisco::Transport::SNMP;
use App::Netdisco::Util::Port ':all';
register_worker({ phase => 'early', driver => 'snmp' }, sub {
my ($job, $workerconf) = @_;
my ($device, $pn) = map {$job->$_} qw/device port/;
# snmp connect using rw community
my $snmp = App::Netdisco::Transport::SNMP->writer_for($device)
or return Status->defer("failed to connect to $device to update vlan/pvid");
vars->{'iid'} = get_iid($snmp, vars->{'port'})
or return Status->error("Failed to get port ID for [$pn] from $device");
return Status->noop("Vlan set can continue.");
});
register_worker({ phase => 'main', driver => 'snmp' }, sub {
return unless defined vars->{'iid'};
_action($job, 'pvid');
return _action($job, 'vlan');
}
sub _action {
my ($job, $slot) = @_;
my ($device, $pn, $data) = map {$job->$_} qw/device port extra/;
my $getter = "i_${slot}";
my $setter = "set_i_${slot}";
# snmp connect using rw community
my $snmp = App::Netdisco::Transport::SNMP->writer_for($device)
or return Status->defer("failed to connect to $device to update $slot");
my $rv = $snmp->$setter($data, vars->{'iid'});
if (!defined $rv) {
return Status->error(sprintf 'Failed to set [%s] %s to [%s] on $device: %s',
$pn, $slot, $data, ($snmp->error || ''));
}
# confirm the set happened
$snmp->clear_cache;
my $state = ($snmp->$getter(vars->{'iid'}) || '');
if (ref {} ne ref $state or $state->{ vars->{'iid'} } ne $data) {
return Status->error("Verify of [$pn] $slot failed on $device");
}
# update netdisco DB
vars->{'port'}->update({$slot => $data});
return Status->done("Updated [$pn] $slot on [$device] to [$data]");
});
true;

View File

@@ -0,0 +1,85 @@
package App::Netdisco::Worker::Runner;
use Dancer qw/:moose :syntax/;
use App::Netdisco::Util::Device 'get_device';
use App::Netdisco::Util::Permission qw/check_acl_no check_acl_only/;
use aliased 'App::Netdisco::Worker::Status';
use Try::Tiny;
use Module::Load ();
use Scope::Guard 'guard';
use Moo::Role;
use namespace::clean;
with 'App::Netdisco::Worker::Loader';
has 'job' => ( is => 'rw' );
# mixin code to run workers loaded via plugins
sub run {
my ($self, $job) = @_;
die 'cannot reuse a worker' if $self->job;
die 'bad job to run()'
unless ref $job eq 'App::Netdisco::Backend::Job';
$self->job($job);
$job->device( get_device($job->device) );
$self->load_workers();
# finalise job status when we exit
my $statusguard = guard { $job->finalise_status };
my @newuserconf = ();
my @userconf = @{ setting('device_auth') || [] };
# reduce device_auth by only/no
if (ref $job->device) {
foreach my $stanza (@userconf) {
my $no = (exists $stanza->{no} ? $stanza->{no} : undef);
my $only = (exists $stanza->{only} ? $stanza->{only} : undef);
next if $no and check_acl_no($job->device, $no);
next if $only and not check_acl_only($job->device, $only);
push @newuserconf, $stanza;
}
# per-device action but no device creds available
return $job->add_status( Status->defer('deferred job with no device creds') )
if 0 == scalar @newuserconf;
}
# back up and restore device_auth
my $configguard = guard { set(device_auth => \@userconf) };
set(device_auth => \@newuserconf);
# run check phase and if there are workers then one MUST be successful
$self->run_workers('workers_check');
return if not $job->check_passed;
# run other phases
$self->run_workers("workers_${_}") for qw/early main user/;
}
sub run_workers {
my $self = shift;
my $job = $self->job or die error 'no job in worker job slot';
my $set = shift
or return $job->add_status( Status->error('missing set param') );
return unless ref [] eq ref $self->$set and 0 < scalar @{ $self->$set };
(my $phase = $set) =~ s/^workers_//;
$job->enter_phase($phase);
foreach my $worker (@{ $self->$set }) {
try { $job->add_status( $worker->($job) ) }
catch {
debug "=> $_" if $_;
$job->add_status( Status->error($_) );
};
}
}
true;

View File

@@ -0,0 +1,105 @@
package App::Netdisco::Worker::Status;
use strict;
use warnings;
use Dancer qw/:moose :syntax !error/;
use Moo;
use namespace::clean;
has 'status' => (
is => 'rw',
default => undef,
);
has [qw/log phase/] => (
is => 'rw',
default => '',
);
=head1 INTRODUCTION
The status can be:
=over 4
=item * C<done>
At C<check> phase, indicates the action may continue. At other phases,
indicates the worker has completed without error or has no work to do.
=item * C<error>
Indicates that there is an error condition. Also used to quit a worker without
side effects that C<done> and C<defer> have.
=item * C<defer>
Quits a worker. If the final recorded outcome for a device is C<defer> several
times in a row, then it may be skipped from further jobs.
=back
=head1 METHODS
=head2 done, error, defer
Shorthand for new() with setting param, accepts log as arg.
=cut
sub _make_new {
my ($self, $status, $log) = @_;
die unless $status;
my $new = (ref $self ? $self : $self->new());
$new->log($log);
$new->status($status);
return $new;
}
sub error { shift->_make_new('error', @_) }
sub done { shift->_make_new('done', @_) }
sub defer { shift->_make_new('defer', @_) }
=head2 noop
Simply logs a message at debug level if passed, and returns true. Used for
consistency with other Status class methods but really does nothing.
=cut
sub noop {
debug $_[1] if $_[1];
return true;
}
=head2 is_ok
Returns true if status is C<done>.
=cut
sub is_ok { return $_[0]->status eq 'done' }
=head2 not_ok
Returns true if status is C<error> or C<defer>.
=cut
sub not_ok { return (not $_[0]->is_ok) }
=head2 level
A numeric constant for the status, to allow comparison.
=cut
sub level {
my $self = shift;
return (($self->status eq 'done') ? 3
: ($self->status eq 'defer') ? 2 : 1);
}
1;