we had situations where the manager would start workers on the same job,
either because of race conditions or because at the time of queueing it wasn't
known that the jobs were targeting the same device (due to device aliases).
this commit removes duplicate jobs, reduces the need for locking on the job
queue, and makes use of lldpRemChassisId to try to deduplicate jobs before
they are started. in effect we have several goes to prevent duplicate jobs:
1. at neighbor discovery time we try to skip queueing same lldpRemChassisId
2. at job selection we 'error out' jobs with same profile as job selected
3. at job selection we check for running job with same profile as selected
4. the job manager process also checks for duplicate job profiles
5. at job lock we abort if the job was 'errored out'
all together this seems to work well. a test on a large university network of
303 devices (four core routers and the rest edge routers, runing VRF with many
duplicate identities), ~1200 subnets, ~50k hosts, resulted in no DB deadlock
or contention and a complete discover+arpnip+macsuck (909 jobs) in ~3 minutes
(with ~150 duplicate jobs identified and skipped).
Squashed commit of the following:
commit 86d0f61d0b
Author: Oliver Gorwits <oliver@cpan.org>
Date: Thu Nov 16 22:26:32 2017 +0000
fix typo
commit 5aff19621c
Author: Oliver Gorwits <oliver@cpan.org>
Date: Thu Nov 16 22:10:18 2017 +0000
fix use of snmp_connect_ip which does not work for SNMPv3
commit 68a56d35bb
Author: Oliver Gorwits <oliver@cpan.org>
Date: Thu Nov 16 20:50:16 2017 +0000
no need for Array::Iterator even though it was cute
commit 71ee869c02
Author: Oliver Gorwits <oliver@cpan.org>
Date: Wed Nov 15 22:14:47 2017 +0000
additional doc examples
commit 620b3fe544
Author: Oliver Gorwits <oliver@cpan.org>
Date: Wed Nov 15 22:09:05 2017 +0000
stash workers within poller instance, and load plugins explicitly
commit 2431365583
Author: Oliver Gorwits <oliver@cpan.org>
Date: Mon Nov 13 22:17:11 2017 +0000
better fix for duplicate module entity index
commit a400b26704
Author: Oliver Gorwits <oliver@cpan.org>
Date: Mon Nov 13 22:14:42 2017 +0000
add ignore interfaces for HPE routers
commit 1502ec1966
Author: Oliver Gorwits <oliver@cpan.org>
Date: Mon Nov 13 22:08:02 2017 +0000
bug fixes after testing on a real network
commit 840b6b4069
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sun Nov 12 20:38:35 2017 +0000
add tests
commit 2de36c69ba
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sun Nov 12 00:14:21 2017 +0000
some reengineering to support proper testing
commit c5f138fe62
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sat Nov 11 14:43:53 2017 +0000
correct algorithm on finalise status, correct logging
commit 98442a2308
Author: Oliver Gorwits <oliver@cpan.org>
Date: Thu Nov 9 21:49:45 2017 +0000
bug fixes
commit e0c6615c87
Author: Oliver Gorwits <oliver@cpan.org>
Date: Wed Nov 8 20:29:33 2017 +0000
fix bugs
commit 1eeaba441d
Author: Oliver Gorwits <oliver@cpan.org>
Date: Tue Nov 7 22:30:55 2017 +0000
finish refactor to new desired behaviour (buggy?)
commit 7edfe88f25
Author: Oliver Gorwits <oliver@cpan.org>
Date: Mon Nov 6 22:50:51 2017 +0000
fix to work, and correct namespace check
commit 25907d3544
Author: Oliver Gorwits <oliver@cpan.org>
Date: Mon Nov 6 21:26:01 2017 +0000
move status tracking and checking inside job instance
commit 4436150bf4
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sun Nov 5 20:54:28 2017 +0000
remove global rubbish
commit 28b016e713
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sat Nov 4 23:31:51 2017 +0000
fix docs
commit 650f6c719b
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sat Nov 4 23:22:12 2017 +0000
tidy line
commit 10f78d5dbe
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sat Nov 4 23:06:20 2017 +0000
add priority and namespace to support fancy worker overrides
commit b9f9816d09
Author: Oliver Gorwits <oliver@cpan.org>
Date: Wed Oct 11 18:33:46 2017 +0100
release 2.036012_001
commit c33bf204a4
Merge: 5b7ce3f7d3d81eb6
Author: Oliver Gorwits <oliver@cpan.org>
Date: Wed Oct 11 18:30:23 2017 +0100
Merge branch 'master' into og-coreplugins
commit 5b7ce3f797
Author: Oliver Gorwits <oliver@cpan.org>
Date: Mon Oct 9 15:46:09 2017 +0100
cannot Sereal::Encode DBIC row
commit 0a575f02ba
Author: Oliver Gorwits <oliver@cpan.org>
Date: Mon Oct 9 14:07:56 2017 +0100
fix bug in job->device init
commit 207476950d
Author: Oliver Gorwits <oliver@cpan.org>
Date: Mon Oct 9 14:03:37 2017 +0100
default causes no attr to be created?!
commit 912f2fa91f
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sun Oct 8 18:43:51 2017 +0100
better debug logging
commit dfeb9d9ddc
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sun Oct 8 18:40:02 2017 +0100
make device_auth have driver setting for snmp entries
commit 460c0c0ee9
Merge: 3ccd107b98423445
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sun Oct 8 18:08:58 2017 +0100
Merge branch 'master' into og-coreplugins
commit 3ccd107bd4
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sat Oct 7 14:13:58 2017 +0100
fix bug in device->has_layer
commit a4b9bf2036
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sat Oct 7 13:58:52 2017 +0100
netdisco-do show takes a param for method in -p
commit 4389cd0459
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sat Oct 7 13:36:06 2017 +0100
fix to only check last poll on devices in storage
commit 58d0fbddda
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sat Oct 7 13:21:13 2017 +0100
do not run discover parts if properties failed to complete
commit b52aaaf1a1
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sat Oct 7 13:08:46 2017 +0100
fix typo
commit 41be926921
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sat Oct 7 13:04:45 2017 +0100
run all check workers
commit a41d114965
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sat Oct 7 13:02:46 2017 +0100
fix driver config
commit b10908a138
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sat Oct 7 12:43:50 2017 +0100
use vars() cache between phases
commit 08b34e083d
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sat Oct 7 11:39:17 2017 +0100
remove die() calls
commit b8108986fb
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sat Oct 7 11:31:59 2017 +0100
phase fixups
commit 273cbbc11b
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sat Oct 7 09:42:41 2017 +0100
change stage to phase
commit 256c10bae5
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sat Oct 7 09:35:14 2017 +0100
multi worker actions need not return done from all workers
commit ee38bae48a
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sat Oct 7 09:05:25 2017 +0100
store result of worker if best for this phase so far
commit 5bddfc73ba
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sat Oct 7 08:50:31 2017 +0100
auto debug-log worker return messages
commit 8b660a89c0
Author: Oliver Gorwits <oliver@cpan.org>
Date: Fri Oct 6 07:48:58 2017 +0100
bug fixes
commit b58a5816a9
Author: Oliver Gorwits <oliver@cpan.org>
Date: Fri Oct 6 07:44:20 2017 +0100
remove unnecessary check phases
commit e44f06364a
Author: Oliver Gorwits <oliver@cpan.org>
Date: Fri Oct 6 07:18:03 2017 +0100
fix unknown command check in netdisco-do
commit 3af13f0dfe
Author: Oliver Gorwits <oliver@cpan.org>
Date: Fri Oct 6 07:15:59 2017 +0100
introduce noop and refactor checks in all workers
commit 98463c8cad
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sun Oct 1 10:49:12 2017 +0100
no need to debug log if there are no hooks in phase
commit 3b32e84312
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sun Oct 1 08:18:13 2017 +0100
fiddle about with runner logic to fix exit states
commit 8fdba38ee0
Author: Oliver Gorwits <oliver@cpan.org>
Date: Fri Sep 29 08:01:42 2017 +0100
cannot reuse a worker as the job will be already set and the wrong plugins loaded
commit a155d9cb77
Author: Oliver Gorwits <oliver@cpan.org>
Date: Fri Sep 29 08:01:06 2017 +0100
should defer when we cannot connect to device
commit 10b5f6cbc4
Author: Oliver Gorwits <oliver@cpan.org>
Date: Fri Sep 29 08:00:32 2017 +0100
fix bug in where workerconf acls are checked
commit 2a74e0befa
Author: Oliver Gorwits <oliver@cpan.org>
Date: Fri Sep 29 07:38:05 2017 +0100
can pass device instance to check_*
commit 4256b117df
Author: Oliver Gorwits <oliver@cpan.org>
Date: Fri Sep 29 07:27:14 2017 +0100
move device_auth build to be with community defaults setting
commit a2de2c1616
Merge: 32be11c38dc4b9bc
Author: Oliver Gorwits <oliver@cpan.org>
Date: Fri Sep 29 07:21:03 2017 +0100
Merge branch 'master' into og-coreplugins
commit 32be11c3ff
Author: Oliver Gorwits <oliver@cpan.org>
Date: Thu Sep 21 00:09:29 2017 +0100
move remaining interactive actions to be plugins
commit 3e41c93f5a
Author: Oliver Gorwits <oliver@cpan.org>
Date: Wed Sep 20 21:47:50 2017 +0100
clean snmp handling
commit 30a2d5dd86
Author: Oliver Gorwits <oliver@cpan.org>
Date: Wed Sep 20 21:00:29 2017 +0100
make sure check plugins are loaded/run before phases
commit 3454d95a84
Author: Oliver Gorwits <oliver@cpan.org>
Date: Wed Sep 20 20:53:52 2017 +0100
capture result on main phase as well
commit 559fa4f93f
Author: Oliver Gorwits <oliver@cpan.org>
Date: Mon Sep 18 22:46:35 2017 +0100
build device_auth from communities
commit 1969291719
Author: Oliver Gorwits <oliver@cpan.org>
Date: Mon Sep 18 22:04:22 2017 +0100
simplify to remove phases and fewer hooks
commit 6f78032e28
Author: Oliver Gorwits <oliver@cpan.org>
Date: Thu Sep 14 21:30:03 2017 +0100
add phase to test worker
commit 6edd2dc879
Author: Oliver Gorwits <oliver@cpan.org>
Date: Wed Sep 13 21:51:40 2017 +0100
no need to list all plugins
commit dfaeb34d8c
Author: Oliver Gorwits <oliver@cpan.org>
Date: Wed Sep 13 20:42:41 2017 +0100
add reset after messing with snmp context or community index
commit 09214dce92
Author: Oliver Gorwits <oliver@cpan.org>
Date: Wed Sep 13 20:29:21 2017 +0100
no need to pass $snmp around
commit 58cd488ccc
Author: Oliver Gorwits <oliver@cpan.org>
Date: Wed Sep 13 19:22:40 2017 +0100
refactor layer and pseudo checks
commit 753acc607f
Author: Oliver Gorwits <oliver@cpan.org>
Date: Wed Sep 13 10:53:12 2017 +0100
use overloaded $device
commit d5d39289d6
Author: Oliver Gorwits <oliver@cpan.org>
Date: Wed Sep 13 10:44:31 2017 +0100
rename init stage to check
commit 1fdb086183
Author: Oliver Gorwits <oliver@cpan.org>
Date: Tue Sep 12 08:12:12 2017 +0100
refactor to remove second loop
commit 64a9491115
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sun Sep 10 16:09:45 2017 +0100
change to init, first, second stages
commit 5f2da69697
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sat Sep 9 22:26:04 2017 +0100
move discover and discoverall to worker plugins
commit c6ebb7cf07
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sat Sep 9 16:44:32 2017 +0100
move arpnip and arpwalk to worker plugins
commit 16a79463cb
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sat Sep 9 16:27:58 2017 +0100
set snmp driver on macsuck phase workers
commit 9167e02de5
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sat Sep 9 15:55:53 2017 +0100
move macsuck and macwalk to worker plugins (macsuck needs snmp scope guard)
commit 68ca85643b
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sat Sep 9 14:56:15 2017 +0100
move expire and expirenodes to worker plugins
commit 271ef1a25c
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sat Sep 9 14:46:00 2017 +0100
move nbtstat and nbtwalk to worker plugins
commit e7508a9eca
Author: Oliver Gorwits <oliver@cpan.org>
Date: Wed Sep 6 21:23:54 2017 +0100
move all netdisco-do action to worker plugins
commit 707fc82b99
Author: Oliver Gorwits <oliver@cpan.org>
Date: Wed Sep 6 21:01:37 2017 +0100
remove psql code from netdisco-do and fix detection of misspelled action
commit 411918e3f8
Author: Oliver Gorwits <oliver@cpan.org>
Date: Wed Sep 6 20:56:26 2017 +0100
only load worker plugins for the action
commit 1f9740c0e2
Author: Oliver Gorwits <oliver@cpan.org>
Date: Wed Sep 6 18:30:43 2017 +0100
shorten hook names
commit a59c23de79
Author: Oliver Gorwits <oliver@cpan.org>
Date: Wed Sep 6 18:27:34 2017 +0100
make psql worker primary, add hook debug log
commit 36c70220a2
Author: Oliver Gorwits <oliver@cpan.org>
Date: Tue Sep 5 22:39:22 2017 +0100
allow two forms of worker declaration, and update docs
commit a79cb9a9e4
Author: Oliver Gorwits <oliver@cpan.org>
Date: Tue Sep 5 22:10:53 2017 +0100
all the bug fixes and a working plugin!!!!!!!!! :-D
commit 04896202e0
Author: Oliver Gorwits <oliver@cpan.org>
Date: Tue Sep 5 21:39:41 2017 +0100
refine runner
commit 547fce2f3c
Author: Oliver Gorwits <oliver@cpan.org>
Date: Tue Sep 5 20:56:21 2017 +0100
hack the status class to regen if needed
commit cd71a0b7a8
Author: Oliver Gorwits <oliver@cpan.org>
Date: Tue Sep 5 20:41:05 2017 +0100
move status update to job class
commit c8e5cea4ed
Author: Oliver Gorwits <oliver@cpan.org>
Date: Tue Sep 5 20:37:13 2017 +0100
objectify the running
commit f48004fffa
Author: Oliver Gorwits <oliver@cpan.org>
Date: Tue Sep 5 19:58:28 2017 +0100
bug squish
commit 46ece568f6
Author: Oliver Gorwits <oliver@cpan.org>
Date: Tue Sep 5 19:54:57 2017 +0100
implement runner?!
commit fc9c60f707
Author: Oliver Gorwits <oliver@cpan.org>
Date: Tue Sep 5 19:28:38 2017 +0100
rename ok to is_ok and change slot names to avoid conflict with creators
commit 3ee85383ab
Author: Oliver Gorwits <oliver@cpan.org>
Date: Tue Sep 5 19:25:41 2017 +0100
skip worker when action is per-device but no creds
commit 75abdad812
Author: Oliver Gorwits <oliver@cpan.org>
Date: Mon Sep 4 21:54:37 2017 +0100
further work on retval handling from workers
commit 4c1fdf4f92
Author: Oliver Gorwits <oliver@cpan.org>
Date: Mon Sep 4 20:37:53 2017 +0100
move worker plugin loader to Worker.pm
commit be0c5181a3
Author: Oliver Gorwits <oliver@cpan.org>
Date: Mon Sep 4 20:35:42 2017 +0100
move Runner to Worker namespace
commit 1c2cf924bc
Author: Oliver Gorwits <oliver@cpan.org>
Date: Mon Sep 4 20:33:20 2017 +0100
worker roles in Role namespace
commit 3099eda393
Author: Oliver Gorwits <oliver@cpan.org>
Date: Mon Sep 4 20:30:58 2017 +0100
load workers when runner role is loaded
commit a8c58a7b05
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sun Sep 3 22:30:28 2017 +0100
initial broken implementation of the runner
commit 49b5274c33
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sun Sep 3 19:04:20 2017 +0100
use run() mixin to exec action
commit e0a666668a
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sun Sep 3 18:54:44 2017 +0100
fix pod; set status defaults; stub runner mixin
commit 8eaa33770c
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sun Sep 3 18:45:00 2017 +0100
rename Core to Worker and move other packages around
commit 4def0af0b0
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sun Sep 3 17:58:03 2017 +0100
better use of new status class
commit 8675bf62c6
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sun Sep 3 17:27:38 2017 +0100
fix hook naming and implement primary workers
commit ef1bb81f2b
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sun Sep 3 17:26:27 2017 +0100
new backend status class
commit 5f50dfadf1
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sun Sep 3 16:51:55 2017 +0100
new Backend package to load core plugins
commit 3baa7a818a
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sun Sep 3 16:22:29 2017 +0100
remove unnecessary Worker::Common role
commit 36b4adcc06
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sun Sep 3 16:17:29 2017 +0100
disambiguate util/backend package and remove backend prelaod
commit 98bff731bd
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sat Sep 2 08:25:06 2017 +0100
settle on a design for hook override, I think
commit fe5c16a16d
Author: Oliver Gorwits <oliver@cpan.org>
Date: Wed Aug 30 20:37:36 2017 +0100
rework docs to be more clear and reflect new operation
commit b34ba1977c
Merge: 31d1977fc34ed61d
Author: Oliver Gorwits <oliver@cpan.org>
Date: Mon Aug 21 21:17:46 2017 +0100
Merge branch 'master' into og-coreplugins
commit 31d1977f1e
Author: Oliver Gorwits <oliver@cpan.org>
Date: Mon Aug 14 18:11:42 2017 +0100
Revert "move expire code to be initial plugin pilot (broken)"
I think we'll only do the new backend code for jobs with a device.
This reverts commit 07998b72d9.
commit 61dc80aff8
Merge: 07998b72ade02db1
Author: Oliver Gorwits <oliver@cpan.org>
Date: Mon Aug 14 18:10:29 2017 +0100
Merge branch 'master' into og-coreplugins
commit 07998b72d9
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sat Aug 5 22:15:00 2017 +0100
move expire code to be initial plugin pilot (broken)
commit 685ec02108
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sat Aug 5 22:10:58 2017 +0100
pass $job to the core worker
commit d6523fe543
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sat Aug 5 22:01:49 2017 +0100
$job->device is always a DBIC row
commit ee6deea01b
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sat Aug 5 18:12:34 2017 +0100
load plugins
commit fd80096ca2
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sat Aug 5 16:53:16 2017 +0100
rename all the things
commit 464c42d1f5
Author: Oliver Gorwits <oliver@cpan.org>
Date: Wed Aug 2 10:19:16 2017 +0100
use Scope::Guard to reduce device_auth
commit ec041dafd2
Author: Oliver Gorwits <oliver@cpan.org>
Date: Tue Aug 1 15:34:37 2017 +0100
the other way around
commit 33d2fe13bd
Author: Oliver Gorwits <oliver@cpan.org>
Date: Mon Jul 31 17:57:29 2017 +0100
fix pod
commit 3faee1cf16
Author: Oliver Gorwits <oliver@cpan.org>
Date: Mon Jul 31 17:55:10 2017 +0100
remove need for instance() call
commit c6d0f1c035
Author: Oliver Gorwits <oliver@cpan.org>
Date: Wed Jul 26 13:51:23 2017 +0100
add doc note on accessing transports
commit dca4b4fc03
Author: Oliver Gorwits <oliver@cpan.org>
Date: Wed Jul 26 11:50:10 2017 +0100
add backend driver documentation
commit 052a2acd79
Author: Oliver Gorwits <oliver@cpan.org>
Date: Wed Jul 26 10:16:58 2017 +0100
rename web plugins doc
commit 69c9a6393a
Author: Oliver Gorwits <oliver@cpan.org>
Date: Wed Jul 26 10:12:42 2017 +0100
rename args to driverconf
commit 2586a36f8c
Author: Oliver Gorwits <oliver@cpan.org>
Date: Tue Jul 25 22:41:10 2017 +0100
new version of core plugin manager with better config and filters
commit 4056831f99
Author: Oliver Gorwits <oliver@cpan.org>
Date: Tue Jul 25 20:53:56 2017 +0100
change SNMP to be a cached transport singleton
commit c31030ef70
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sun Jul 23 13:46:27 2017 +0100
fixes because Dancer docs are a mess!
commit f65ef90b86
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sat Jul 22 08:11:36 2017 +0100
rename snmp_auth to device_auth and include a little doc on transports
commit d61556e1cf
Author: Oliver Gorwits <oliver@cpan.org>
Date: Sat Jul 22 07:54:26 2017 +0100
plugin config added
commit de8de56308
Author: Oliver Gorwits <oliver@cpan.org>
Date: Wed Jul 12 21:38:31 2017 +0100
initial core plugin implementation
