135/netdisco
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

add discover/refresh, scheduler jobs, netdisco-do

Browse Source 
New Poller daemon worker can discover and refresh devices.
  New Scheduler daemon worker replaces the cron jobs with config.
  New netdisco-do script allows running a job one-off from CLI.

Squashed commit of the following:

commit fa25f36e14
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Apr 10 22:43:47 2013 +0100

    fix HTTP port at 5000

commit 202ea4a84c
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Apr 10 22:33:03 2013 +0100

    bug fixes in discover

commit 925d9e4d6b
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Apr 10 21:51:44 2013 +0100

    add mini app for one-time jobs

commit d3a6c08a9d
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Apr 10 21:46:55 2013 +0100

    better name for subaction

commit 4adf473b20
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Apr 10 20:15:18 2013 +0100

    add logging of db add/del

commit 8aacafedaa
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Apr 10 19:49:00 2013 +0100

    copy all remaining messages from netdisco 1

commit 3e1156df1f
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Apr 8 22:09:40 2013 +0100

    alter some log levels and messages

commit e7ea92920f
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Apr 8 20:32:33 2013 +0100

    store wireless ssid and port info to DB

commit d1d16938a1
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Fri Apr 5 08:52:59 2013 +0100

    update packaging for new files and deps

commit 965990786f
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Thu Apr 4 22:22:50 2013 +0100

    implementation of find_neighbors

commit 03c4d8ef09
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Thu Apr 4 20:42:11 2013 +0100

    add discoverall and discover_neighbors poller jobs

commit df68ff0890
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Thu Apr 4 19:52:37 2013 +0100

    implementation of store_modules

commit c2ac19e647
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Thu Apr 4 19:32:14 2013 +0100

    implementation of store_power

commit b7fb8c64a0
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Thu Apr 4 18:48:09 2013 +0100

    implementation of store_vlans

commit b8ddbd1eca
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Thu Apr 4 17:08:06 2013 +0100

    implementation of store_wireless (without storing, yet)

commit 2a14057481
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Thu Apr 4 16:47:21 2013 +0100

    implementation of store_interfaces (without wireless)

commit d5b2b71d34
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Thu Apr 4 15:12:19 2013 +0100

    only start Manager if there are pollers or interactives

commit f4a3dac760
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Thu Apr 4 15:03:10 2013 +0100

    change sub names so as not to collide with Dancer

commit a8f0894986
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Thu Apr 4 14:47:28 2013 +0100

    implementation of refresh, discover, and store_device

commit 4c2e3cf82d
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Thu Apr 4 11:38:15 2013 +0100

    make get_device return a new result object

commit e6ac131658
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Thu Apr 4 10:38:22 2013 +0100

    better POD section name

commit 6c5b6bbbee
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Thu Apr 4 10:37:46 2013 +0100

    implement separate snmp_connect and snmp_connect_rw methods

commit 62c8e19063
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Apr 1 20:06:29 2013 +0100

    fix for unique constraint on job queue for locally queued jobs

commit ebb65996e6
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Apr 1 20:00:36 2013 +0100

    add refresh poller job

commit 05928e8cf6
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Apr 1 19:31:39 2013 +0100

    Refactor Util namespace

    Squashed commit of the following:

    commit 789c528fcf
    Author: Oliver Gorwits <oliver@cpan.org>
    Date:   Mon Apr 1 19:31:07 2013 +0100

        update manifest and fix typo

    commit b95d0951f2
    Author: Oliver Gorwits <oliver@cpan.org>
    Date:   Mon Apr 1 19:22:41 2013 +0100

        refactor ::Util namespace

    commit a8dde50343
    Author: Oliver Gorwits <oliver@cpan.org>
    Date:   Sun Mar 31 13:45:27 2013 +0100

        no need to search for device - IP should already be exact

commit b42daee4c1
Merge: 6e52762 95bb8fc
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Mar 27 21:00:09 2013 +0000

    Merge branch 'master' into og-poller

commit 6e527629a2
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Mar 26 23:39:23 2013 +0000

    fixes and log messages

commit cfcb7a956f
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Mar 26 22:57:06 2013 +0000

    bug fixes

commit 48f779a8d0
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Mar 26 22:42:16 2013 +0000

    add config for scheduled tasks

commit 2f6efcb312
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Mar 26 22:15:04 2013 +0000

    create poller worker and add poller type stubs

commit 52b28b0ab8
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Mar 26 22:04:00 2013 +0000

    code tidy

commit 96db66739f
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Mar 25 22:35:11 2013 +0000

    more insane but more working version of the job queue constraint

commit cb25216f40
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Mar 24 20:22:11 2013 +0000

    make scheduler start automatic based on housekeeping setting existing

commit 0acbe8abd3
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Mar 24 19:45:24 2013 +0000

    add scheduler based on Algorithm::Cron

commit 49d136b57a
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Mar 24 18:48:10 2013 +0000

    add unique constraint on admin/job queue
This commit is contained in:
Oliver Gorwits
2013-04-10 22:45:33 +01:00
parent 95bb8fc06c
commit d21f9e8e19
35 changed files with 2191 additions and 354 deletions
Download Patch File Download Diff File Expand all files Collapse all files

27
Netdisco/bin/netdisco-daemon-fg
View File

@@ -33,7 +33,7 @@ mkdir $tmp_dir if ! -d $tmp_dir;
my $mce = MCE->new(
spawn_delay => 0.15,
job_delay => 0.15,
job_delay => 1.15,
tmp_dir => $tmp_dir,
user_func => sub { $_[0]->worker_body },
on_post_exit => \&restart_worker,
@@ -41,17 +41,23 @@ my $mce = MCE->new(
)->run();
sub build_tasks_list {
my $tasks = [{
max_workers => 1,
user_begin => worker_factory('Manager'),
}];
# NB MCE does not like max_workers => 0
my $tasks = [];
set(daemon_pollers => 2)
if !defined setting('daemon_pollers');
set(daemon_interactives => 2)
if !defined setting('daemon_interactives');
# XXX MCE does not like max_workers => 0
push @$tasks, {
max_workers => 1,
user_begin => worker_factory('Manager'),
} if setting('daemon_pollers') or setting('daemon_interactives');
push @$tasks, {
max_workers => 1,
user_begin => worker_factory('Scheduler'),
} if setting('housekeeping');
push @$tasks, {
max_workers => setting('daemon_pollers'),
@@ -63,8 +69,13 @@ sub build_tasks_list {
user_begin => worker_factory('Interactive'),
} if setting('daemon_interactives');
info sprintf "MCE will load %s tasks: 1 Manager, %s Poller, %s Interactive",
(1+ scalar @$tasks), (setting('daemon_pollers') || 0), (setting('daemon_interactives') || 0);
info sprintf "MCE will load %s tasks: %s Manager, %s Scheduler, %s Poller, %s Interactive",
(scalar @$tasks),
((setting('daemon_pollers') or setting('daemon_interactives')) ? 1 : 0),
(setting('housekeeping') ? 1 : 0),
(setting('daemon_pollers') || 0),
(setting('daemon_interactives') || 0);
return $tasks;
}

108
Netdisco/bin/netdisco-do Executable file
View File

@@ -0,0 +1,108 @@
#!/usr/bin/env perl
use FindBin;
FindBin::again();
use Path::Class 'dir';
BEGIN {
# stuff useful locations into @INC
unshift @INC,
dir($FindBin::RealBin)->parent->subdir('lib')->stringify,
dir($FindBin::RealBin, 'lib')->stringify;
}
# for netdisco app config
use App::Netdisco;
use Dancer qw/:moose :script/;
use Dancer::Plugin::DBIC 'schema';
info "App::Netdisco version $App::Netdisco::VERSION loaded.";
use Try::Tiny;
use Getopt::Long;
Getopt::Long::Configure ("bundling");
my ($device, $port, $extra, $debug);
my $result = GetOptions(
'device|d=s' => \$device,
'port|p=s' => \$port,
'extra|e=s' => \$extra,
'debug|D' => \$debug,
) or exit(1);
# reconfigure logging to use console
my $CONFIG = config();
$CONFIG->{logger} = 'console';
$CONFIG->{log} = ($debug ? 'debug' : 'info');
Dancer::Logger->init('console', $CONFIG);
# check requested action
my $action = shift @ARGV;
my $PERMITTED_ACTIONS = qr/(?:discover|discover_neighbors)/;
if (!length $action) {
error 'error: missing action!';
exit (1);
}
if ($action !~ m/^$PERMITTED_ACTIONS$/) {
error sprintf 'error: netdisco-do cannot [%s]', $action;
exit (1);
}
if (!length $device) {
error 'error: missing device!';
exit (1);
}
# create worker (placeholder object for the role methods)
{
package MyWorker;
use Moo;
with 'App::Netdisco::Daemon::Worker::Poller::Discover';
}
my $worker = MyWorker->new();
# static configuration for the in-memory local job queue
setting('plugins')->{DBIC}->{daemon} = {
dsn => 'dbi:SQLite:dbname=:memory:',
options => {
AutoCommit => 1,
RaiseError => 1,
sqlite_use_immediate_transaction => 1,
},
schema_class => 'App::Netdisco::Daemon::DB',
};
schema('daemon')->deploy;
# what job are we asked to do?
my $job = schema('daemon')->resultset('Admin')->new_result({
job => 0,
action => $action,
device => $device,
port => $port,
subaction => $extra,
});
# belt and braces check before we go ahead
if (not $worker->can( $action )) {
error sprintf 'error: %s is not a valid action for netdisco-do', $action;
exit (1);
}
# do job
my ($status, $log);
try {
info sprintf '%s: started at %s', $action, scalar localtime;
($status, $log) = $worker->$action($job);
}
catch {
$status = 'error';
$log = "error running job: $_";
};
info sprintf '%s: finished at %s', $action, scalar localtime;
info sprintf '%s: status %s: %s', $action, $status, $log;
exit ($status eq 'done' ? 0 : 1);