Files
netdisco/share/config.yml
Oliver Gorwits 9a72d7e74a Avoid lock/defer of jobs deined via ACL
This commit adds a table 'device_skip' that is used to restrict job queue
searches to avoid jobs that are not permitted on this backend via *_no ACLs,
or jobs on devices that have previously encountered multiple SNMP timeouts.

When the backend loads or a device is added, a row is added to the table if
that device should not be polled on this backend (together with the job
actions which are to be skipped/denied). When a device SNMP connect fails a
counter in the same row (or a new row) is incremented.

There is also a new report 'SNMP Connect Failures' to show the devices with
non-zero SNMP connect failure counters. A configurable limit in the setting
'max_deferrals' is used to set the threshold of no longer polling the device.

To reset the deferrals/failures count, restart the Netdisco backend (which
regenerates 'device_skip' cache entries).

Squashed commit of the following:

commit b5e32c219d
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue May 23 20:55:14 2017 +0100

    show all failed connections in report

commit ffce3cee84
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue May 23 20:12:39 2017 +0100

    only resolve fqdn once

commit cc4f680f01
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue May 23 20:10:20 2017 +0100

    Revert "only resolve fqdn once"

    This reverts commit 3d136a54de.

commit d8d082b30e
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue May 23 20:09:05 2017 +0100

    a report to show SNMP failures

commit 3d136a54de
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue May 23 19:37:58 2017 +0100

    only resolve fqdn once

commit 4550b8a84c
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue May 23 17:27:43 2017 +0100

    skipover now implicit from deferrals/actionset; fix sql where logic with better correlation

commit b51edbccd2
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue May 23 16:11:29 2017 +0100

    only abort lock if action matches badactions

commit 415559b24f
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue May 23 13:56:42 2017 +0100

    set skipover true when adding to actionset

commit 1086f2c467
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue May 23 13:50:56 2017 +0100

    fix empty actionset

commit 31962580b8
Merge: 9b2e993e 6808133b
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue May 23 13:25:08 2017 +0100

    Merge branch 'og-device_skip' of github.com:netdisco/netdisco into og-device_skip

commit 6808133bdb
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue May 23 13:19:54 2017 +0100

    in-job checks for acls are required for netdisco-do foreground actions

commit 3944dd7813
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue May 23 13:18:30 2017 +0100

    avoid extra device lookup

commit 9b2e993e0f
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue May 23 12:31:36 2017 +0100

    also delete device_skip rows when deleting device

commit b55854e91d
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue May 23 11:34:27 2017 +0100

    actions in device_skip table are now an array/set

commit 5e126eef07
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue May 23 09:36:33 2017 +0100

    typo

commit 44266f2767
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue May 23 09:14:25 2017 +0100

    *able checks within jobs should not be necessary with skiplist

commit e7c22e7d11
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue May 23 08:58:57 2017 +0100

    increment deferrals field when job is deferred

commit 88ae9c00ba
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue May 23 08:40:27 2017 +0100

    turn connect fail into defer

commit eac1857043
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue May 23 08:26:59 2017 +0100

    rename failures column to be deferrals

commit 96ed444bbb
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon May 22 22:52:51 2017 +0100

    set up list of jobs the backend instance should skip

commit 3a0019296d
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon May 22 22:01:50 2017 +0100

    separate out is_*able last_* checks

commit cf8589aba2
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun May 21 22:35:38 2017 +0100

    change from ignore to skip name

commit ed193356f8
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun May 21 14:52:33 2017 +0100

    device_ignore table to track devices to skip in polling
2017-05-27 08:50:08 +01:00

325 lines
6.9 KiB
YAML

# This is the main configuration file for Netdisco web and backend apps
#
# DO NOT EDIT THIS FILE
#
# Overrides should go to ~/environments/deployment.yml
# See App::Netdisco::Manual::Configuration for explanations
# ----------------
# GENERAL SETTINGS
# ----------------
log: 'warning'
logger_format: '[%P] %U %L %m'
include_paths: []
template_paths: []
site_local_files: false
external_databases: []
# ------------
# WEB FRONTEND
# ------------
domain_suffix: ""
no_auth: false
suggest_guest: false
navbar_autocomplete: true
trust_remote_user: false
trust_x_remote_user: false
#ldap:
# servers: []
# user_string: 'MYDOMAIN\%USER%'
# base: ""
# proxy_user: ""
# proxy_pass: ""
# opts:
# debug: 3
# tls_opts: {}
path: '/'
web_plugins:
- Inventory
- Report::PortVLANMismatch
- Report::PortAdminDown
- Report::PortBlocking
- Report::PortMultiNodes
- Report::PortSsid
- Report::PortUtilization
- Report::ApChannelDist
- Report::ApClients
- Report::ApRadioChannelPower
- Report::HalfDuplex
- Report::DeviceAddrNoDNS
- Report::DeviceByLocation
- Report::InventoryByModelByOS
- Report::DeviceDnsMismatch
- Report::DevicePoeStatus
- Report::DuplexMismatch
- Report::IpInventory
- Report::ModuleInventory
- Report::Netbios
- Report::NodeMultiIPs
- Report::NodeVendor
- Report::NodesDiscovered
- Report::SsidInventory
- Report::VlanInventory
- Report::SubnetUtilization
- Report::PortLog
- AdminTask::JobQueue
- AdminTask::Topology
- AdminTask::PollerPerformance
- AdminTask::PseudoDevice
- AdminTask::SlowDevices
- AdminTask::UndiscoveredNeighbors
- AdminTask::OrphanedDevices
- AdminTask::TimedOutDevices
- AdminTask::UserLog
- AdminTask::Users
- Search::Device
- Search::Node
- Search::VLAN
- Search::Port
- Device::Details
- Device::Ports
- Device::Modules
- Device::Neighbors
- Device::Addresses
extra_web_plugins: []
jobqueue_refresh: 10
safe_password_store: true
reports: []
table_pagesize: 10
table_showrecordsmenu:
- [10, 25, 50, 100, '-1']
- [10, 25, 50, 100, 'All']
vlanctl: true
portctl_nameonly: false
portctl_nophones: false
portctl_vlans: false
portctl_uplinks: false
port_control_reasons:
address: 'Address Allocation Abuse'
copyright: 'Copyright Violation'
dos: 'Denial of Service'
bandwidth: 'Excessive Bandwidth'
polling: 'Excessive Polling of DNS/DHCP/SNMP'
noserv: 'Not In Service'
exploit: 'Remote Exploit Possible'
compromised: 'System Compromised'
other: 'Other'
resolved: 'Issue Resolved'
check_userlog: true
devport_vlan_limit: 150
login_logo: ""
# -------------
# NETDISCO CORE
# -------------
# mibhome is discovered from environment
# mibdirs defaults to contents of mibhome
host_groups: {}
device_identity: {}
community: ['public']
community_rw: ['private']
snmp_auth: []
get_community: ""
bulkwalk_off: false
bulkwalk_no: []
bulkwalk_repeaters: 20
nonincreasing: false
snmpver: 3
snmptimeout: 3000000
snmpretries: 2
discover_no: []
discover_only: []
discover_no_type: []
discover_min_age: 0
macsuck_no: []
macsuck_only: []
macsuck_all_vlans: false
macsuck_no_unnamed: false
macsuck_no_vlan:
- 'fddi-default'
- 'token-ring-default'
- 'fddinet-default'
- 'trnet-default'
macsuck_no_devicevlan: []
macsuck_unsupported: []
macsuck_unsupported_type: []
macsuck_bleed: false
macsuck_min_age: 0
snmpforce_v1: []
snmpforce_v2: []
snmpforce_v3: []
arpnip_no: []
arpnip_only: []
arpnip_min_age: 0
nbtstat_no: []
nbtstat_only: []
nbtstat_max_age: 7
nbtstat_interval: 0.02
nbtstat_timeout: 1
node_freshness: 0
expire_devices: 60
expire_nodes: 90
expire_nodes_archive: 60
expire_jobs: 14
store_wireless_clients: true
store_modules: true
ignore_interfaces:
- 'EOBC'
- 'unrouted VLAN(?: \d+)?'
- 'StackPort'
- 'Control Plane Interface'
- 'SPAN (S|R)P Interface'
- 'StackSub-.*'
- 'StackPort\d+'
- 'netflow'
- 'Vlan\d+-mpls layer'
- 'BRI\S+-Bearer Channel'
- 'BRI\S+-Physical'
- 'BRI\S+-Signalling'
- 'Embedded-Service-Engine\d+\/\d+'
- 'Virtual-Template\d+'
- 'Virtual-Access\d+'
- '(E|T)\d \d\/\d\/\d'
ignore_private_nets: false
reverse_sysname: false
phone_capabilities:
- '(?i:phone)'
phone_platforms:
- '(?i:mitel.5\d{3})'
phone_ouis: []
wap_capabilities:
- 'wlanAccessPoint'
wap_platforms:
- '(?i:\bw?ap\b)'
- 'cisco\s+AIR-[L|C]?AP'
- '-K9W8-'
wap_ouis: []
# --------------
# BACKEND DAEMON
# --------------
workers:
tasks: 'AUTO * 2'
sleep_time: 1
min_runtime: 0
max_deferrals: 10
queue: PostgreSQL
dns:
max_outstanding: 50
hosts_file: '/etc/hosts'
no: ['fe80::/64','169.254.0.0/16']
#schedule:
# discoverall:
# when: '5 7 * * *'
# macwalk:
# when:
# min: 20
# arpwalk:
# when:
# min: 50
# nbtwalk:
# when: '0 8,13,21 * * *'
# expire:
# when: '20 23 * * *'
job_prio:
high:
- location
- contact
- portcontrol
- portname
- vlan
- power
normal:
- discoverall
- discover
- arpwalk
- arpnip
- macwalk
- macsuck
- nbtwalk
- nbtstat
- expire
# ---------------
# GraphViz Export
# ---------------
graph:
# ---- Graph Settings ----
edge_color : wheat
graph : 'graph/netmap.gif'
graph_png : 'graph/netmap.png'
graph_bg : black
graph_clusters : false # try fdp layout
graph_color : white
graph_default : png
#graph_dir : net_dir.gif
graph_epsilon : 6
graph_layout : twopi # try neato or fdp too
graph_map : 'graph/netmap.map'
graph_overlap : scale
graph_nodesep : 2
graph_ranksep : .3
graph_raw : 'graph/graph_raw.dot'
graph_splines : false
graph_svg : 'graph/netmap.svg'
graph_timeout : 90
graph_x : 30
graph_y : 30
node_fillcolor : dimgrey
node_font : lucon
node_fontsize : 46.0
node_fontcolor : white
node_problem : red
node_shape : box
node_style : filled
#edge_style : setlinewidth(10)
# ---- Node Maps ----
# variable:matching pattern:node attribute:attribute value:key:key name
#node_map:
# - 'label:cat(?!-g):fillcolor:blue:cat:Blue Box - Catalyst Device'
# - 'label:-g:fillcolor:darkgreen:dev-g:Green Box - Gateway / Router'
# - 'ip:^192.168\.:color:yellow:dev:Yellow Border - ResNet'
# ---------------
# DANCER INTERNAL
# ---------------
charset: 'UTF-8'
warnings: false
show_errors: false
logger: 'console'
engines:
netdisco_template_toolkit:
encoding: 'utf8'
start_tag: '[%'
end_tag: '%]'
PRE_CHOMP: 1
INCLUDE_PATH: []
layout: 'main'
plugins:
Auth::Extensible:
no_api_change_warning: true
no_default_pages: true
no_login_handler: true
realms:
users:
provider: 'App::Netdisco::Web::Auth::Provider::DBIC'
schema_name: 'netdisco'
session: 'cookie'
session_cookie_key: 'this_will_be_overridden_on_webapp_startup'
template: 'netdisco_template_toolkit'
route_cache: true
appname: 'Netdisco'
behind_proxy: false