Commit Graph

163 Commits

Author SHA1 Message Date
Oliver Gorwits
dc1f76c1aa Feature to gather SNMP Walk, use as Pseudo Device, and Browse Objects
* fix anomalous name

* add gather worker

* fix encoding of binary storage

* store results back to job

* now parsing mbis report to translate

* fix the broken report parser

* rename gather to snapshot

* implement walk code copied from SNMP::Info

* can now bulkwalk and parse mibs report and store resolved walk in cache

* add func/glob aliasing broken

* better aliasing

* implement aliasing from globals and funcs

* fix regexp for matching netdisco-mibs report

* fake cache entry for all ND2 methods called, add comments

* also save to logs/snapshots/IP

* add doc for netdisco-do

* add is_pseudo column to device table

* support for loading cache for pseudo devices

* check for hrSystemUptime as well as sysUpTime for snmp connect

* display pseudo devices with yellow pill for name

* color all cells for layers for pseudo

* no need to b64 encode binary data in scalars as we b64 whole thing after

* tweaked uptime check

* store snapshot to database instead of Job

* expose snapshots in device details tab

* small ux improvements on snap download

* fixes for errors in subnet mask searching

* hide snapshot management for pseudo devices

* update to use new netdisco-mibs object cache

* update for new format oids file

* start of work on loading walk into db for browsing

* store values and meta

* add auto increment col and oid index to browser

* start web plugin for browser

* add virtual search for oid children

* have all oid in separte table (60 seconds load on my laptop)

* rename table and add relation

* store oid as int array

* fix sql for children

* make jstree start working

* working very slow tree expand

* fix to work when first displaying tree

* store both oid and oid_parts

* simplify SQL to speed up (more complicated perl)

* fix sql bug, add better index, prettify tree

* render the snmp node detail

* add node template, make scrollable, pretty print data values (insecure)

* store munge hint

* some dubious code to munge the data

* make sure to filter by IP on device_browser

* make safer the rendering of value data (but need to come back to key ordering)

* fix sorting on object values

* limit the opening of child nodes to keep response good and unclutter

* factor out the munge and make safer

* reject unknown mungers

* show the munger and option (not working) to change

* additional js for munge select

* complete custom munge

* change so that saving to database is only at CLI and on request

* hide snmp tab if no browser rows in the db

* add helpful message when no browser rows for the device

* stub handler for search and add recurse control

* working search

* minor ui fixes

* implement typeahead for leaf search

* limit rows in typeahead

* make sure device_browser is visited in delete and renumber

* add requirements for this branch

* update manifest

* make sure node search and typeahead are restricted to current device only
2021-11-06 07:47:29 +00:00
Oliver Gorwits
202d02a64a #827 add ignore_interface_types setting 2021-10-12 13:21:07 +01:00
Oliver Gorwits
741190e5f0 add helpful comment 2021-10-04 14:10:52 +01:00
Oliver Gorwits
854f19d49e #816 skip duplicate bssid on ports on wireless discovery 2021-09-26 08:27:46 +01:00
Oliver Gorwits
0cbce4ab3f Swap error and info return status from Jobs to avoid info() on error 2021-09-03 21:32:48 +01:00
Oliver Gorwits
2498d9766b #800 gather and report SNMP Chassis ID http://oid-info.com/cgi-bin/display?oid=1.3.6.1.4.1.9.3.6.3&a=display 2021-08-14 13:28:13 +01:00
nick n
83a946d36e replace unicode nonbreaking space with ascii space (#794) 2021-06-23 22:00:33 +01:00
Christian Ramseyer
3f3de34f51 Allow <"> in remote_port (#782)
Makes the remote_port info work better with Nokia TiMOS devices.
2021-04-21 13:09:39 +02:00
Oliver Gorwits
351927d384 add arpnip and macsuck hooks and exec hook implementation 2020-12-30 20:38:45 +00:00
Oliver Gorwits
28349eb489 add macsuck and arpnip hooks, add debug on hook queue 2020-12-29 18:15:58 +00:00
Oliver Gorwits
669eec46db Implement Hooks per #726 2020-12-23 11:18:38 +00:00
Oliver Gorwits
2b72233738 release 2.046004 2020-10-17 14:28:53 +01:00
Oliver Gorwits
4cd0c3eef9 #747 handle missing snmpEngineID 2020-10-17 12:54:41 +01:00
Oliver Gorwits
a146f53d67 #744 remove whitespace from serial numbers 2020-09-22 18:31:14 +01:00
Oliver Gorwits
650c419dc7 error out on discover failing to get good interfaces from SNMP 2020-07-03 09:32:58 +01:00
Oliver Gorwits
7e51fe2295 try to safeguard some snmp leaf gather failures 2020-07-01 21:21:09 +01:00
Oliver Gorwits
dc87a50076 #728 store SNMP Engine ID 2020-06-29 14:38:51 +01:00
Oliver Gorwits
83de59fcc0 #729 Discovery partially fails on invalid indexes 2020-06-29 14:05:44 +01:00
Ambroise
224bfbca52 - add a function to avoid to have two times the same device exported in rancid config (#725)
- avoid some strange warning message and add a default domain in makerancidconf plugin
2020-06-05 09:08:19 +01:00
Christian Ramseyer
dcb45e4cad Fix previous commit
* revert unwanted commit of bogus Web/Plugin/Device/Modules.pm
 * commit actual fix for #710
2020-04-25 17:32:01 +02:00
Christian Ramseyer
4ae9b32973 Fix #704, allow delete without device_auth
* This typically comes up when using a deployment.yml generated from
   some inventory source, the device disappears and can then no
   longer be deleted since netdisco-delete unnecessarily insists on
   needing a valid device_auth
2020-04-21 00:43:59 +02:00
Oliver Gorwits
4f57a4d30a rename ports column in device table to be num_ports 2020-04-18 16:50:49 +01:00
Oliver Gorwits
dff26abc5c API implementation (#712)
* initial v0 creator

* working json api for generic reports

* add require login

* move report swagger into plugin, and set new default layout of noop

* require proper role and also use new util func

* start to tidy authn

* some work on cleaning up web authn

* clean up the authN checks

* fix bug

* fix the auth for api

* fixes to json handling

* set swagger sort order

* enable most reports for api endpoints

* fix doc

* add paramters to reports

* add missed report

* allow api_parameters in reports config

* reorganise api

* add vlan search

* add port search

* make sure to enable layout processing

* add device search

* add v1 to api paths

* add Node Search

* support api_responses

* add device object search; fix spurious ports field in device result class

* handle some plugins just returning undef if search fails

* errors from api seamlessley

* fix error in date range default

* more sensible default for prefix

* change order of endpoints in swagger-ui

* all db row classes can now TO_JSON

* add device_port api endpoint

* add device ports endpoint

* do not expand docs

* add swagger ui json tree formatter

* add all relations from Device table

* add port relations

* add nodes retrieve on device or vlan

* rename to GetAPIKey

* update config for previous commit
2020-04-15 21:15:52 +01:00
Christian Ramseyer
f2f5d908bb Fix regression from PR #680
Hi @ollyg! Unfortunately I have found some issues with the code we
finally released in #680:

* get_port_macs expects an array ref but values() returns array,
  so the code was never called due to the return unless... check
* When fw_mac_list had exactly two entries, only the second value
  was bound as a scalar to the parameter. This is probably due
  to the shorthand bind formats described in
  https://metacpan.org/pod/DBIx::Class::ResultSet#DBIC-BIND-VALUES,
  but I'm not a 100% on this.
* return unless now checks for an entry in the list, with the old
  check the statement was also executed for empty lists

In cases where only the device(_port)?.mac lookup worked for uplink
detection, users of 02.044005 - 010 might get a lot of uplinks not
labeled as such.
2020-01-26 22:08:49 +01:00
Oliver Gorwits
e42c2135c4 fix #686 makerancidconf excluded setting should be (ACL) list not dict 2019-12-30 11:09:17 +00:00
Oliver Gorwits
f7f2d2088b #81 avoid DEPRECATED single warning from DBIC 2019-12-22 09:07:12 +00:00
Oliver Gorwits
9ab1b24a77 safety on snmp::info methods 2019-12-21 21:34:39 +00:00
Oliver Gorwits
9a1938ddc9 safety and tidy the port_macs patch 2019-12-21 20:28:20 +00:00
Christian Ramseyer
26d3fbdd40 Reduce macsuck bandwidth usage to database (#680)
* Reduce macsuck bandwidth usage to database

get_port_macs transfers a full list of all device_port.(mac,ip) in every macsuck.
With 8k devices and 40k interfaces it takes up around 15MB. Transferring them 8k times
during an 1h macsuck cycle requires bandwidth in the 300 to 400 mbit/s range.

This patch changes get_port_macs to be called inside walk_fwtable and only transfer
the macs found in the current target device/vlan.
2019-12-21 18:28:47 +00:00
Oliver Gorwits
884a75a866 #659 speed up Macsuck::Nodes while using $snmp->cisco_comm_indexing (#661)
* #659 speed up Macsuck::Nodes while using $snmp->cisco_comm_indexing

* Do not read qb_fw_vlan when using cisco_comm_indexing
2019-10-14 22:08:38 +02:00
Oliver Gorwits
7f65d6afdc fix file endling 2019-09-23 14:00:28 +01:00
Christian Ramseyer
8ffdff3bbc Add speed_admin to device ports tab (#650) (#653)
* Add speed_admin to device ports tab (#650)

* move from device_port_properties to device_port table, and tidy up
2019-09-23 13:58:26 +01:00
Christian Ramseyer
26960bdd34 Fix potential crash in Discover/Properties::_get_ipv6_aliases
* On e.g. Cisco N9K, ipv6 objects can be returned undef which crashes the ip alias discovery with `Can't use an undefined value as a HASH reference at .../Discover/Properties.pm line 346.`
 * Discovery continues but leaves device_ip empty, which in turn makes device->neighbor non-functional
 * Fixed by adding various `$snmp->x || {}` safeguards
2019-09-20 13:52:34 +02:00
Christian Ramseyer
8edbd515e7 Store i_speed_admin (#650) 2019-09-19 14:11:04 +02:00
pyro3d
73e9f0bf0d Add DOCSIS/Cable Modem Discovery (#646)
* Add DOCSIS/Cable Modem Discovery
2019-09-18 16:19:18 -07:00
Oliver Gorwits
542837d266 #591 domain_suffix can be list and supports (simple) regexp
Squashed commit of the following:

commit 975e4c6afc
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 3 13:35:26 2019 +0100

    also support regexp in domain_suffix

commit 0a2b5c8fa2
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 3 13:17:17 2019 +0100

    fix rancid, graph, and nodemonitor

commit 6d881de3ff
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 3 13:11:54 2019 +0100

    improve docs and set default domain_suffix to be list

commit 1dcafc08a8
Merge: 9a752e02 14ac69dc
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 3 13:00:39 2019 +0100

    Merge branch 'master' into og-multiple-domain-suffix

commit 9a752e0298
Merge: c836619f 82a99ea9
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 3 09:45:25 2019 +0100

    Merge branch 'master' into og-multiple-domain-suffix

commit c836619f8c
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Thu Jun 13 07:52:45 2019 +0100

    hokey fix for nodes with domains

commit fed14bd810
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Thu Jun 13 07:02:09 2019 +0100

    basic implementation, rancid graph and nodemonitor missing
2019-09-03 13:35:43 +01:00
Oliver Gorwits
14ac69dc2e might as well make other *walk not care about pseudo and use layers instead 2019-09-03 13:00:22 +01:00
Oliver Gorwits
2897eda684 #587 #561 update pseudo devices to better support ssh arpnip
this patch resets all pseudo devices to have no layer3 support but adds a
feature to the pseudo devices admin panel to enable layer3 support. it also
changes arpnip and arpwalk behaviour to always permit the action if layer3
is available (ignoring the vendor).

documentation will need updating to tell users to create pseudo devices
with layer3 support when they want to arpnip an unsupported platform.

arpnip with ssh/cli against a supported platform (one that can be discovered)
will continue to work normally.

Squashed commit of the following:

commit 9dad5be81d
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 3 09:03:53 2019 +0100

    allow pseudo with layer 3 to run arpnip

commit 7d97943fcd
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 3 08:59:10 2019 +0100

    allow pseudo devices with layer 2/3 capability

commit d1fdf574e3
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 3 08:55:41 2019 +0100

    move pseudo and layer checks to is_able from is_able_now

commit e0f72ef67d
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 3 08:51:42 2019 +0100

    ports defaults to one

commit 86ba01270c
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 3 08:50:45 2019 +0100

    add tooltip for arpnip toggle

commit cdd2470228
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 3 08:34:46 2019 +0100

    simplify template

commit 46236d68ea
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Sep 1 23:53:56 2019 +0100

    a fix up for pseudo devices which need layer 3

commit 016d249efc
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Sep 1 20:37:11 2019 +0100

    do not wrap buttons

commit 1ec1402e0c
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Sep 1 20:33:03 2019 +0100

    implement user settable layer-three service for pseudo devices

commit a267efa3d8
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Sep 1 18:39:22 2019 +0100

    only set layer if successful action

commit b108be5e23
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Sep 1 18:32:19 2019 +0100

    should defer SNMP against pseudo devices

commit 897ba3a629
Merge: e0ddbaab a7348900
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Sep 1 14:54:36 2019 +0100

    Merge branch 'master' into og-pseudo-vs-cli-arpnip

commit e0ddbaab08
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Aug 26 11:35:13 2019 +0100

    as last commit, for discover

commit 61f9c89040
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Aug 25 23:55:38 2019 +0100

    move pseudo and layer checks into is_*able functions

commit 8b010d4023
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Aug 25 18:38:11 2019 +0100

    any device completing macsuck/arpnip must have that layer

commit a11bce7863
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Aug 25 18:33:27 2019 +0100

    clean up device layers

commit d2661bff61
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Aug 25 18:18:02 2019 +0100

    first make arpnip behave like other jobs towards pseudo devices
2019-09-03 09:09:55 +01:00
nick n
22cb988a8f typo 2019-08-17 10:34:19 +02:00
nick n
205a3932f2 typo 2019-08-17 10:33:32 +02:00
nick n
d31cae99bc typo 2019-08-17 10:32:13 +02:00
nick n
860985673b typo 2019-08-17 10:30:25 +02:00
Oliver Gorwits
2c138a5690 #581 skip storing and displaying fabricated vlan 0 2019-05-30 06:58:34 +01:00
nick n
3ec136aea3 allow setting some speudo device info (#572)
* allow port name to be changed on pseudo devices, was once added in commit 1f020e07ee

* also allow this for location and contact
2019-05-02 13:19:46 +01:00
nick n
fd4752897d netdisco rancid updates (#558)
* typo

* make this a link

* netdisco-rancid-export has been replaced

* rancid worker config

* netdisco-rancid-export now a no-op

* note debugging option
2019-04-05 01:28:30 +02:00
Oliver Gorwits
731eb765b1 fix strange interaction between CORE::sort and List::MoreUtils::uniq 2019-04-03 14:34:09 +01:00
Oliver Gorwits
db71fffe54 refactor to try device port vlan update again 2019-04-02 15:31:20 +01:00
Oliver Gorwits
223c616cb0 better var names and change logic to skip port 2019-04-01 23:15:44 +01:00
Oliver Gorwits
b22c32a578 better sql update, fixed worker return, better logic on vlans 2019-04-01 22:36:35 +01:00
Oliver Gorwits
a6e7943c03 #528 record egress untagged vlans in device_port_vlan table 2019-04-01 14:38:27 +01:00