Commit Graph
100 Commits
Author SHA1 Message Date
Oliver Gorwits 647b78534b Revert "device skips should not be used on priority jobs"
We still must respect per backend skipping

This reverts commit 7b9e7f4deb.
2017-06-12 10:42:34 +01:00
Oliver Gorwits 7b9e7f4deb device skips should not be used on priority jobs 2017-06-11 14:45:59 +01:00
Oliver Gorwits 2eb0b30812 do not select pseudo devices for poller jobs 2017-06-06 08:08:52 +01:00
Oliver Gorwits 275e01ff59 include last_defer timestamp when deferring a job 2017-06-06 07:56:50 +01:00
Oliver Gorwits 2ff39d131d reset deferrals counter on successful SNMP connect 2017-06-06 07:04:33 +01:00
Oliver Gorwits 4d0b9542d3 shorten timeout on dns for timeoutdevices report 2017-06-06 06:27:29 +01:00
Oliver Gorwits f42ab6ff73 also move PID file here 2017-05-31 19:05:16 +01:00
Oliver Gorwits 2a42a1b5a1 default 50 snmp failures 2017-05-30 21:44:19 +01:00
Oliver Gorwits b0eb652141 testing release 2.035999_007 2017-05-30 21:27:40 +01:00
Oliver Gorwits 585a442b64 move old PID file to new name 2017-05-30 21:23:20 +01:00
Oliver Gorwits 3aa34f12dc add DNS to SNMP Connect Failures report 2017-05-30 21:01:35 +01:00
Oliver Gorwits a553a86f51 Merge branch 'master' of github.com:netdisco/netdisco 2017-05-30 20:42:25 +01:00
Oliver Gorwits 5d7483ab7b testing release 2.035999_006 2017-05-30 16:03:53 +01:00
Oliver Gorwits 4ada3ee160 fix @_ handling 2017-05-30 13:07:19 +01:00
Oliver Gorwits 271ffa80fd deviceskip only if device defiend 2017-05-30 12:39:11 +01:00
Oliver Gorwits 0b0c6f1e94 testing release 2.035999_005 2017-05-30 12:05:16 +01:00
Oliver Gorwits 574977b6a1 only do device_skip for device jobs 2017-05-30 12:04:22 +01:00
Oliver Gorwits 10ce6402ad strange scoping fix 2017-05-30 12:02:22 +01:00
Oliver Gorwits 540cdef6ff testing release 2.035999_004 2017-05-30 11:40:45 +01:00
Oliver Gorwits d007d72015 check_acl call fixes 2017-05-30 11:38:34 +01:00
Oliver Gorwits e0d2517387 quieten links update 2017-05-30 11:31:36 +01:00
Oliver Gorwits 49995a8fa7 minor pod fixup 2017-05-29 17:26:37 +01:00
Oliver Gorwits b27c4342d6 testing release 2.035999_003 2017-05-29 17:21:16 +01:00
Oliver Gorwits e0f592d799 reorder release notes 2017-05-29 17:21:08 +01:00
Oliver Gorwits 8aae2d72c0 typo fixup 2017-05-29 17:16:53 +01:00
Oliver Gorwits efe669c0e6 support list or dict for device_identity 2017-05-29 16:11:39 +01:00
Oliver Gorwits 5daf1805c7 rename prime_skiplist to warm_thrusters 2017-05-28 18:48:58 +01:00
Oliver Gorwits 9ac2662182 add option to disable max_deferrals 2017-05-28 09:31:57 +01:00
Oliver Gorwits 666e0a593d fix only lookup fqdn once 2017-05-28 09:30:27 +01:00
Oliver Gorwits 5f58cfbbe5 change device_identity to be list of mappings 2017-05-28 09:14:08 +01:00
Oliver Gorwits eb56364d98 add note on SNMP Connect Failure tracking 2017-05-27 16:26:31 +01:00
Oliver Gorwits edcce0cfae tidy release notes 2017-05-27 16:24:50 +01:00
Oliver Gorwits 601a943190 update changelog 2017-05-27 16:23:04 +01:00
Oliver Gorwits 234056cb66 testing release 2.035999_002 2017-05-27 15:48:56 +01:00
Oliver Gorwits c7c4c47fa0 update release notes 2017-05-27 15:46:50 +01:00
Oliver Gorwits f8d75442e6 add documentation for device_identity 2017-05-27 15:04:38 +01:00
Oliver Gorwits c4598df24b remove implicit op:and from device_identity lhs check 2017-05-27 12:28:34 +01:00
Oliver Gorwits 60a00efea5 add documentation for host_groups 2017-05-27 11:01:21 +01:00
Oliver Gorwits 02b168ff9c update plugin-writing docs for site_local_files setting 2017-05-27 10:04:05 +01:00
Oliver Gorwits 6932b13013 add max_deferrals documentation and other doc fixes 2017-05-27 09:30:04 +01:00
Oliver Gorwits 9a72d7e74a Avoid lock/defer of jobs deined via ACL
This commit adds a table 'device_skip' that is used to restrict job queue
searches to avoid jobs that are not permitted on this backend via *_no ACLs,
or jobs on devices that have previously encountered multiple SNMP timeouts.

When the backend loads or a device is added, a row is added to the table if
that device should not be polled on this backend (together with the job
actions which are to be skipped/denied). When a device SNMP connect fails a
counter in the same row (or a new row) is incremented.

There is also a new report 'SNMP Connect Failures' to show the devices with
non-zero SNMP connect failure counters. A configurable limit in the setting
'max_deferrals' is used to set the threshold of no longer polling the device.

To reset the deferrals/failures count, restart the Netdisco backend (which
regenerates 'device_skip' cache entries).

Squashed commit of the following:

commit b5e32c219d
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue May 23 20:55:14 2017 +0100

    show all failed connections in report

commit ffce3cee84
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue May 23 20:12:39 2017 +0100

    only resolve fqdn once

commit cc4f680f01
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue May 23 20:10:20 2017 +0100

    Revert "only resolve fqdn once"

    This reverts commit 3d136a54de.

commit d8d082b30e
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue May 23 20:09:05 2017 +0100

    a report to show SNMP failures

commit 3d136a54de
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue May 23 19:37:58 2017 +0100

    only resolve fqdn once

commit 4550b8a84c
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue May 23 17:27:43 2017 +0100

    skipover now implicit from deferrals/actionset; fix sql where logic with better correlation

commit b51edbccd2
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue May 23 16:11:29 2017 +0100

    only abort lock if action matches badactions

commit 415559b24f
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue May 23 13:56:42 2017 +0100

    set skipover true when adding to actionset

commit 1086f2c467
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue May 23 13:50:56 2017 +0100

    fix empty actionset

commit 31962580b8
Merge: 9b2e993e 6808133b
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue May 23 13:25:08 2017 +0100

    Merge branch 'og-device_skip' of github.com:netdisco/netdisco into og-device_skip

commit 6808133bdb
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue May 23 13:19:54 2017 +0100

    in-job checks for acls are required for netdisco-do foreground actions

commit 3944dd7813
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue May 23 13:18:30 2017 +0100

    avoid extra device lookup

commit 9b2e993e0f
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue May 23 12:31:36 2017 +0100

    also delete device_skip rows when deleting device

commit b55854e91d
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue May 23 11:34:27 2017 +0100

    actions in device_skip table are now an array/set

commit 5e126eef07
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue May 23 09:36:33 2017 +0100

    typo

commit 44266f2767
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue May 23 09:14:25 2017 +0100

    *able checks within jobs should not be necessary with skiplist

commit e7c22e7d11
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue May 23 08:58:57 2017 +0100

    increment deferrals field when job is deferred

commit 88ae9c00ba
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue May 23 08:40:27 2017 +0100

    turn connect fail into defer

commit eac1857043
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue May 23 08:26:59 2017 +0100

    rename failures column to be deferrals

commit 96ed444bbb
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon May 22 22:52:51 2017 +0100

    set up list of jobs the backend instance should skip

commit 3a0019296d
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon May 22 22:01:50 2017 +0100

    separate out is_*able last_* checks

commit cf8589aba2
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun May 21 22:35:38 2017 +0100

    change from ignore to skip name

commit ed193356f8
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun May 21 14:52:33 2017 +0100

    device_ignore table to track devices to skip in polling
2017-05-27 08:50:08 +01:00
Oliver Gorwits 47a5f40efe bug fix 2017-05-20 16:15:56 +01:00
Oliver Gorwits 845c1a20ff testing release 2.035999_001 2017-05-17 07:42:50 +01:00
Oliver Gorwits 8e07636a14 topology import script runs discover for each device (M. Bauer) 2017-05-17 07:37:05 +01:00
Oliver Gorwits da04522199 add site_local_files feature for easy lib/template override 2017-05-14 21:12:19 +01:00
Oliver Gorwits 824ec751e2 efficiency gains in set_canonical_ip 2017-05-14 20:50:03 +01:00
Oliver Gorwits 3816aad829 support for device identity steering via device_identity 2017-05-12 20:19:04 +01:00
Oliver Gorwits 849e0ddfd9 support Netdisco device_ip instances in match thing 2017-05-12 19:22:58 +01:00
Oliver Gorwits 5e55c60ee8 fail safe on _no acl matching 2017-05-12 18:25:57 +01:00
Oliver Gorwits 4df2bc45ea use check_acl for dns->no acl 2017-05-10 21:13:03 +01:00
Oliver Gorwits 156f2abfa7 docs fixes for new acl handling 2017-05-10 20:58:33 +01:00
Oliver Gorwits 762ce952d1 tidy up calls to check_{node|device}_{no|only} to use only check_acl* 2017-05-10 18:54:38 +01:00
Oliver Gorwits b20028bb2a test scalar promoted to list 2017-05-08 08:39:51 +01:00
Oliver Gorwits 95598b5468 basic implementation of named acls (need to tidy up calls to check_acl) 2017-05-07 13:34:21 +01:00
Oliver Gorwits 2c41b2925d template_paths config to allow easier override of web templates 2017-05-06 17:59:50 +01:00
Oliver Gorwits 273b49b9ce cleanup dist and meta 2017-05-06 17:52:42 +01:00
Oliver Gorwits 3af2bc2fec typo 2017-05-06 16:52:51 +01:00
Oliver Gorwits d0472a9ddf #309 missing Device Port VLAN Mismatch CSV template 2017-05-06 16:50:34 +01:00
Oliver Gorwits d74ccac4f6 rename *-daemon apps to be *-backend
Squashed commit of the following:

commit 39b438aa4b
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat May 6 16:40:11 2017 +0100

    add release notes

commit ca4ea90d35
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat May 6 16:32:06 2017 +0100

    update distmeta

commit 4e35b904b0
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat May 6 16:30:22 2017 +0100

    rename files from Daemon to Backend

commit 86a605ba68
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat May 6 16:26:43 2017 +0100

    rename daemon to backend in code

commit ffe8fc180f
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat May 6 16:15:57 2017 +0100

    add daemon files which exec to backend equivalents

commit 53e041594e
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat May 6 15:32:49 2017 +0100

    rename netdisco-daemon to netdisco-backend
2017-05-06 16:40:48 +01:00
Oliver Gorwits db9d98b69b Move tests to /xt as they are RELEASE tests
Squashed commit of the following:

commit 606d572db561d8de659bce6ac96252f8a1d7ee29
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat May 6 14:56:07 2017 +0100

    tests should be release tests, so move to xt/

commit 7673f3ee1e
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat May 6 14:19:19 2017 +0100

    allow check_acl to accept Device or NetAddr::IP instance

commit c31059bc01
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat May 6 14:19:00 2017 +0100

    update docs

commit deaeab2670
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat May 6 14:18:27 2017 +0100

    SNMP only stanza has access to full check_acl features

commit 4a44fa5863
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon May 1 18:49:38 2017 +0100

    add AND operator and negation support to ACLs
2017-05-06 15:16:43 +01:00
Oliver Gorwits 03f41f1177 Enhance the ACL options to include AND and negation
Squashed commit of the following:

commit 7673f3ee1e
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat May 6 14:19:19 2017 +0100

    allow check_acl to accept Device or NetAddr::IP instance

commit c31059bc01
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat May 6 14:19:00 2017 +0100

    update docs

commit deaeab2670
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat May 6 14:18:27 2017 +0100

    SNMP only stanza has access to full check_acl features

commit 4a44fa5863
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon May 1 18:49:38 2017 +0100

    add AND operator and negation support to ACLs
2017-05-06 15:16:04 +01:00
Oliver Gorwits 3654468913 Add SSL development library to Release Notes 2017-04-29 16:28:19 +01:00
Oliver Gorwits 46d054ffd1 release 2.035006 2017-04-29 09:29:29 +01:00
Oliver Gorwits 67f067fdb2 Add SSL development library to OS base requirements install doc 2017-04-29 09:28:50 +01:00
Oliver Gorwits 511c59d171 release 2.035005 2017-04-29 09:13:03 +01:00
Oliver Gorwits 645289248e Add note to docs about reinstall after OS upgrade 2017-04-29 09:08:44 +01:00
Oliver Gorwits dc36f1b036 Do not attempt to UTF-8 decode OUI retrieved by curl/wget 2017-04-29 09:02:33 +01:00
Oliver Gorwits c182beeebf update docs for new oui and mib urls 2017-04-27 07:53:21 +01:00
Oliver Gorwits 3ec84161ae New report for Port VLAN Mismatches (M. Bernstein) 2017-04-25 12:26:09 +01:00
Oliver Gorwits fd4678a68b release 2.035004 2017-04-25 10:53:57 +01:00
Oliver Gorwits ff4c999836 Fix for relocated DB schema files 2017-04-25 10:36:18 +01:00
Oliver Gorwits 1d458aec61 release 2.035003 2017-04-24 14:43:54 +01:00
Oliver Gorwits f439241c9f fix for MIB download through proxy 2017-04-24 14:43:03 +01:00
Oliver Gorwits cb9a24ed51 release 2.035002 2017-04-24 12:47:34 +01:00
Oliver Gorwits 48cb6dd4c1 Fix DB schema files location with Module::Build 2017-04-24 12:41:15 +01:00
Oliver Gorwits 96a3313fb7 explicit mention to use File::Copy 2017-04-20 08:43:41 +01:00
Oliver Gorwits 7feb0c5e85 release 2.035001 2017-04-19 22:18:11 +01:00
Oliver Gorwits a9f01aaa54 reindex DB (I did not realise this was being indexed! oof.) 2017-04-19 22:16:13 +01:00
Oliver Gorwits 16fa8ecc23 and mention NXOS in SSHCollector list 2017-04-19 22:00:40 +01:00
Oliver Gorwits b23dc2d551 release 2.035000 2017-04-19 21:56:33 +01:00
Oliver Gorwits 608d1b86b2 #298 NXOS SSHCollector and note in docs about VRFs 2017-04-19 21:48:45 +01:00
Oliver Gorwits d1a7761fda #302 Device searching now searches on module serial numbers 2017-04-19 21:30:47 +01:00
Oliver Gorwits ca3ed44ca5 fix changelog 2017-04-19 19:48:48 +01:00
Oliver Gorwits d23b32500f relocate repo files so ND2 is the only code 2017-04-14 23:08:55 +01:00
Oliver Gorwits 9a016ea6ba version 2.034003
migrate from Module::Install to Module::Build
2017-04-14 22:53:22 +01:00
Oliver Gorwits b7724bc16c #296 modification to add min_runtime optional sleep 2017-04-14 21:24:02 +01:00
Oliver Gorwits 6524ceefae use File::Slurper instead of File::Slurp to better handle UTF8 in oui.txt 2017-04-14 21:01:49 +01:00
Oliver Gorwits 79b1520126 use github release feature 2017-04-14 20:38:21 +01:00
Oliver Gorwits 7a2fa272b6 move anything using db into the db update part 2017-04-14 20:38:00 +01:00
Oliver Gorwits 2406244f4c Move to Github hosted IEEE OUI data 2017-04-14 08:56:12 +01:00
Oliver Gorwits b0263b0767 Add another community FreeBSD install guide 2017-02-27 15:07:35 +00:00
Oliver Gorwits 4d070ee754 Add GAIA Embedded SSH collector (not the same as VSX, apparently) 2017-02-27 15:04:29 +00:00
Oliver Gorwits 724ff68946 #296 Fix occasional empty macsuck when run in daemon 2017-02-27 14:39:09 +00:00
Oliver Gorwits f533a994f2 no need for M::I or META files in the dist 2017-01-06 14:35:23 +00:00
Oliver Gorwits d5afb73cee release 2.034002 2017-01-06 14:33:53 +00:00
Oliver Gorwits 33a37ffe06 update M::I 2017-01-06 14:33:27 +00:00
Oliver Gorwits 927bbc0cce release 2.034001 2016-11-20 17:50:55 +00:00
Oliver Gorwits a1b1928ffb Add note on SLES 11 SP4 install workaround (C. Ramseyer) 2016-11-20 17:30:58 +00:00
Oliver Gorwits d311d1a8cc Add Linux SNMP, LLDP setup docs (S. Hobson) 2016-10-03 22:41:29 +01:00
Oliver Gorwits 6c17fe65b8 community data now deleted with devices 2016-10-03 22:18:47 +01:00