Commit Graph

84 Commits

Author SHA1 Message Date
Oliver Gorwits
dc1f76c1aa Feature to gather SNMP Walk, use as Pseudo Device, and Browse Objects
* fix anomalous name

* add gather worker

* fix encoding of binary storage

* store results back to job

* now parsing mbis report to translate

* fix the broken report parser

* rename gather to snapshot

* implement walk code copied from SNMP::Info

* can now bulkwalk and parse mibs report and store resolved walk in cache

* add func/glob aliasing broken

* better aliasing

* implement aliasing from globals and funcs

* fix regexp for matching netdisco-mibs report

* fake cache entry for all ND2 methods called, add comments

* also save to logs/snapshots/IP

* add doc for netdisco-do

* add is_pseudo column to device table

* support for loading cache for pseudo devices

* check for hrSystemUptime as well as sysUpTime for snmp connect

* display pseudo devices with yellow pill for name

* color all cells for layers for pseudo

* no need to b64 encode binary data in scalars as we b64 whole thing after

* tweaked uptime check

* store snapshot to database instead of Job

* expose snapshots in device details tab

* small ux improvements on snap download

* fixes for errors in subnet mask searching

* hide snapshot management for pseudo devices

* update to use new netdisco-mibs object cache

* update for new format oids file

* start of work on loading walk into db for browsing

* store values and meta

* add auto increment col and oid index to browser

* start web plugin for browser

* add virtual search for oid children

* have all oid in separte table (60 seconds load on my laptop)

* rename table and add relation

* store oid as int array

* fix sql for children

* make jstree start working

* working very slow tree expand

* fix to work when first displaying tree

* store both oid and oid_parts

* simplify SQL to speed up (more complicated perl)

* fix sql bug, add better index, prettify tree

* render the snmp node detail

* add node template, make scrollable, pretty print data values (insecure)

* store munge hint

* some dubious code to munge the data

* make sure to filter by IP on device_browser

* make safer the rendering of value data (but need to come back to key ordering)

* fix sorting on object values

* limit the opening of child nodes to keep response good and unclutter

* factor out the munge and make safer

* reject unknown mungers

* show the munger and option (not working) to change

* additional js for munge select

* complete custom munge

* change so that saving to database is only at CLI and on request

* hide snmp tab if no browser rows in the db

* add helpful message when no browser rows for the device

* stub handler for search and add recurse control

* working search

* minor ui fixes

* implement typeahead for leaf search

* limit rows in typeahead

* make sure device_browser is visited in delete and renumber

* add requirements for this branch

* update manifest

* make sure node search and typeahead are restricted to current device only
2021-11-06 07:47:29 +00:00
Oliver Gorwits
202d02a64a #827 add ignore_interface_types setting 2021-10-12 13:21:07 +01:00
Oliver Gorwits
3c3b9f286c better Content-Security-Policy 2021-10-06 17:56:46 +01:00
Oliver Gorwits
381f412df9 clickjacking prevention via X-Frame-Options and Content-Security-Policy headers 2021-10-06 16:44:36 +01:00
Oliver Gorwits
43ce26ef3d Allow port_control_reasons setting to actually work 2021-09-03 21:17:11 +01:00
Oliver Gorwits
5f3ae3bf10 Config setting portctl_nowaps to prevent port control on WAP ports 2021-09-03 06:22:16 +01:00
Oliver Gorwits
e953e3945f ACLs portctl_no and portctl_only for limiting port control targets 2021-09-03 06:15:20 +01:00
Oliver Gorwits
8509cd96cc #778 exec hooks were not configured to be high priority 2021-07-14 13:15:40 +01:00
nick n
83a946d36e replace unicode nonbreaking space with ascii space (#794) 2021-06-23 22:00:33 +01:00
Oliver Gorwits
351927d384 add arpnip and macsuck hooks and exec hook implementation 2020-12-30 20:38:45 +00:00
Oliver Gorwits
669eec46db Implement Hooks per #726 2020-12-23 11:18:38 +00:00
Oliver Gorwits
31fc6e1ee5 #734 Multiple TACACS+/RADIUS servers now supported 2020-07-10 22:18:29 +01:00
Oliver Gorwits
fc46384fc6 #227 Do not overwrite partially retrieved data with null or incorrect values
this and two related commits implements:
- setting snmp_field_protection which can be used to protect any
  database field from having a null value set
- extra device discover worker which prefetches SNMP interfaces and
  performs some sanity checks and comparison against the DB
the two checks will return job status ERROR with CANCELLED if they
see anything fishy.
2020-07-06 19:49:57 +01:00
Oliver Gorwits
7e51fe2295 try to safeguard some snmp leaf gather failures 2020-07-01 21:21:09 +01:00
Oliver Gorwits
da33478504 try to make swagger safer with send_file 2020-04-19 11:22:28 +01:00
Oliver Gorwits
4f0117e22c many fixes to swagger hosting to allow path relocation (part complete) 2020-04-18 17:26:04 +01:00
Oliver Gorwits
dff26abc5c API implementation (#712)
* initial v0 creator

* working json api for generic reports

* add require login

* move report swagger into plugin, and set new default layout of noop

* require proper role and also use new util func

* start to tidy authn

* some work on cleaning up web authn

* clean up the authN checks

* fix bug

* fix the auth for api

* fixes to json handling

* set swagger sort order

* enable most reports for api endpoints

* fix doc

* add paramters to reports

* add missed report

* allow api_parameters in reports config

* reorganise api

* add vlan search

* add port search

* make sure to enable layout processing

* add device search

* add v1 to api paths

* add Node Search

* support api_responses

* add device object search; fix spurious ports field in device result class

* handle some plugins just returning undef if search fails

* errors from api seamlessley

* fix error in date range default

* more sensible default for prefix

* change order of endpoints in swagger-ui

* all db row classes can now TO_JSON

* add device_port api endpoint

* add device ports endpoint

* do not expand docs

* add swagger ui json tree formatter

* add all relations from Device table

* add port relations

* add nodes retrieve on device or vlan

* rename to GetAPIKey

* update config for previous commit
2020-04-15 21:15:52 +01:00
Oliver Gorwits
ab6305b4cf default access list for device_auth should be ipv4 and ipv6 2020-02-12 16:55:16 +00:00
Oliver Gorwits
4030000b34 #687 Add an option to set main page after logging in 2019-12-21 22:24:19 +00:00
Oliver Gorwits
deb9b62c7f Enforce escaping on all template content 2019-09-23 14:22:00 +01:00
Oliver Gorwits
5f378a39ea better column titles 2019-09-23 14:05:04 +01:00
Christian Ramseyer
8ffdff3bbc Add speed_admin to device ports tab (#650) (#653)
* Add speed_admin to device ports tab (#650)

* move from device_port_properties to device_port table, and tidy up
2019-09-23 13:58:26 +01:00
pyro3d
73e9f0bf0d Add DOCSIS/Cable Modem Discovery (#646)
* Add DOCSIS/Cable Modem Discovery
2019-09-18 16:19:18 -07:00
Oliver Gorwits
542837d266 #591 domain_suffix can be list and supports (simple) regexp
Squashed commit of the following:

commit 975e4c6afc
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 3 13:35:26 2019 +0100

    also support regexp in domain_suffix

commit 0a2b5c8fa2
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 3 13:17:17 2019 +0100

    fix rancid, graph, and nodemonitor

commit 6d881de3ff
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 3 13:11:54 2019 +0100

    improve docs and set default domain_suffix to be list

commit 1dcafc08a8
Merge: 9a752e02 14ac69dc
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 3 13:00:39 2019 +0100

    Merge branch 'master' into og-multiple-domain-suffix

commit 9a752e0298
Merge: c836619f 82a99ea9
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 3 09:45:25 2019 +0100

    Merge branch 'master' into og-multiple-domain-suffix

commit c836619f8c
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Thu Jun 13 07:52:45 2019 +0100

    hokey fix for nodes with domains

commit fed14bd810
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Thu Jun 13 07:02:09 2019 +0100

    basic implementation, rancid graph and nodemonitor missing
2019-09-03 13:35:43 +01:00
Oliver Gorwits
262fa88009 #585 make toastr notifications less annoying 2019-05-27 12:53:56 +01:00
Oliver Gorwits
8f1f35abab fix inconsistency in titlebar global search when port is selected 2019-04-16 17:10:13 +01:00
nick n
fd4752897d netdisco rancid updates (#558)
* typo

* make this a link

* netdisco-rancid-export has been replaced

* rancid worker config

* netdisco-rancid-export now a no-op

* note debugging option
2019-04-05 01:28:30 +02:00
nick n
3f407685b1 unneeded question marks 2019-03-23 02:07:17 +01:00
Oliver Gorwits
46d8c52a08 Implement changes for API authentication and Swagger UI (#541)
* initial token-based-api login handler

* add token schema and validation

* initial import of pyro3d api code

* basic Swagger spec support

* Merge in working copy of API/Device.pm

* Fix some error handling for API/Device.pm

* Break out utility functions into separate file, to allow other api portions to use

* Add NodeIP support.

* Add nodeip plugin to config

* remove double define of "plugin:" (#448)

disclaimer: i did not test this is any way, came across it when looking for something else.

* only AuthZ header for api use, and alway regen key on login

* use RFC7235

* workaround for Swagger plugin weird response body

* do not autodiscover swagger routes

* code formatting only

* move api util to utils area

* initial full swagger spec for nodeip search

* add api user role and fix api auth failure response

* update version of swagger-ui to 3.20.3

* add more openapi defs

* fixes to SQL and api spec

* clean up subs

* improvements to login/logout for API

* make api logout work

* add openapi tags to group operations

* allow api params to be generated from DBIC schema spec

* remove API calls for nodes and devices

* remove some poor assumptions about api calls

* tidy up

* remove DDP

* make login and logout similar

* example of api call being handled by ajax call

* make the branch authonly
2019-03-17 20:27:19 +00:00
Oliver Gorwits
9b39d1e452 rename CLI Transport to SSH 2019-03-13 22:23:38 +00:00
Oliver Gorwits
d44a8f56ea Integrate netdisco-sshcollector into Worker::Plugin architecture (#489) (#535)
* update changes and SNMP::Info dep

* Integrate netdisco-sshcollector into Worker::Plugin architecture (#489)

* Initial integration of sshcollector into Worker::Plugin architecture

 * add NodesBySSH.pm
 * update Build.PL and config.yml to integrate the new module

* Further integration of sshcollector into Worker::Plugin architecture

 * added App::Netdisco::Transport::CLI loosely based on ::SNMP counterpart
 * switched to the more prevalent two-space tabs style
 * removed various TBD items, some new ones

* Further steps to integration of sshcollector into Worker::Plugin architecture

 * cleaned up code
 * added various error handling
 * warning for bin/netdisco-sshcollector deprecation
 * device_auth allows passing master_opts to Net::OpenSSH
 * netdisco-do -D also toggles Net::OpenSSH debug

* Merged NodesBySSH.pm into Nodes.pm

 * see https://github.com/netdisco/netdisco/pull/489#pullrequestreview-205603516

* Further integration of sshcollector into Worker::Plugin architecture

 * add snmp_arpnip_also option to sshcollector device_auth
 * cleanup code

* Remove big TBD: comment from CLI.pm as doc is updated now

* add transport/cli.pm to manifest

* revert some changes to allow simpler merging

* silent exit legacy script unless explicitly requested

* move ssh code into Transport, part one

* rewrite the CLI transport to provide an API

* merge in og-get_external_credentials

Squashed commit of the following:

commit 3fe8f383a7
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Mar 11 17:07:42 2019 +0000

    add debug lines and tested

commit 3249739e42
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Mar 11 16:54:11 2019 +0000

    change config name to get_credentials

commit e78558397a
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Mar 11 16:51:11 2019 +0000

    separate out generic device auth to DeviceAuth module

commit 249f05165f
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Mar 6 18:43:31 2019 +0000

    release 2.040007

commit e3af64df77
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Mar 6 18:42:47 2019 +0000

    #521-redux fix wifi date search

commit 48857ae300
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Mar 4 12:03:31 2019 +0000

    release 2.040006

commit e09dab5362
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Mar 4 11:39:12 2019 +0000

    #527 update List::MoreUtils version requirement

commit 6e7de3fff3
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Mar 4 09:59:41 2019 +0000

    release 2.040005

commit 0c98318a45
Author: Oliver Gorwits <oliver@spike.local>
Date:   Mon Mar 4 09:57:18 2019 +0000

    #526 fix discover syntax bug

commit e9efc45182
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Mar 3 14:56:48 2019 +0000

    release 2.040004

commit 6cdfd80d10
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Mar 3 14:34:00 2019 +0000

    allow undiscovered neighbors report to use discover_{waps,phones} setting

commit ac381e0802
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Mar 3 14:13:20 2019 +0000

    #506 was a red herring

commit b83e614c85
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Mar 3 13:00:36 2019 +0000

    make discover_{phones,waps} work with LLDP capabilities as well

commit 189d234b55
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Mar 3 12:47:38 2019 +0000

    check discover_no_type and friends earlier on in neighbors list build

commit 9c956466f3
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Mar 3 12:32:07 2019 +0000

    also update default config for new discover_phones and discover_waps settings

commit 09d29954d2
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Mar 3 12:26:50 2019 +0000

    #512 fix regression in phone/wap discovery exclusion

commit 2bae91f1b6
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Mar 3 12:01:34 2019 +0000

    rename match_devicetype() to match_to_setting()

commit 57cb6ddb70
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Mar 3 09:19:39 2019 +0000

    fix for over-eager fix to #506

commit ef560fb59a
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Mar 2 22:41:40 2019 +0000

    #506 relax device renumber so it works for an alias

commit 7a8bcb094e
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Mar 2 22:23:39 2019 +0000

    #521 Search Node Date Range not working

commit a643820a62
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Mar 2 21:54:27 2019 +0000

    #428 Port-Channels not showing in netmap

commit 5ba5bcd295
Merge: e7aacddb a1f95028
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Mar 2 20:04:11 2019 +0000

    Merge branch 'master' of github.com:netdisco/netdisco

commit e7aacddbc6
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Mar 2 20:01:05 2019 +0000

    #498 Map with VLAN filter omits unconnected devices

commit a1f95028ca
Author: nick n <39005454+inphobia@users.noreply.github.com>
Date:   Sat Mar 2 19:54:22 2019 +0100

    catch up with changes

    noticed that rc-sshcollector-core received updates to changes, add them here as well.

    didn't mention #499 & #522

commit ce1b847cea
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Mar 2 18:47:44 2019 +0000

    fix bug showing no nodes when only one matches in netmap

commit 78e30a7926
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Mar 2 16:28:15 2019 +0000

    #500 filtering in device/ports on native vlan duplicates entries

commit 9952f0c6c7
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Mar 2 15:02:12 2019 +0000

    #499 netdisco-do renumber reports wrong ip (inphobia)

commit ca3fd8f466
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Mar 2 15:00:18 2019 +0000

    #505 device renumber should update device port properties and device skips

commit 1265bc8470
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Mar 2 14:52:21 2019 +0000

    #520 catch slave ports defined without a master

commit d4c7579c10
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Mar 2 14:47:49 2019 +0000

    #522 TypeAhead.pm can reference empty data (inphobia)

commit 77decc23b7
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Mar 2 14:45:37 2019 +0000

    #514 inconsistent results in ip inventory (inphobia)

commit 3f211650b8
Author: nick n <39005454+inphobia@users.noreply.github.com>
Date:   Fri Mar 1 12:34:42 2019 +0100

    last pieces for db schema upgrade

    last piece of #510

* import legacy sshcollector config

* add default use_legacy_sshcollector config

* remove unneeded deps

* various fixes and now tested

* enable sshcollector platform tests
2019-03-12 18:50:24 +00:00
Oliver Gorwits
b9218d91c8 Support for external command to provide device_auth stanza (#532)
* separate out generic device auth to DeviceAuth module

* change config name to get_credentials

* add debug lines and tested
2019-03-12 13:13:06 +00:00
Oliver Gorwits
09d29954d2 #512 fix regression in phone/wap discovery exclusion 2019-03-03 12:26:50 +00:00
nick n
6a36baa961 add a new tab to device -> vlans (#483)
* first steps for vlan tab

* export sort_vlans, use strict/warnings

* update manifest, get worker closer to functioning

* remove ie 9 script & extra ./th

* make it work

* cleanup

* readd internet explorer code

* start for csv output

* move things to debug logging instead of info

* use message which was discussed

* fix for address table sorting

* link os

* final touches thx to ollyg

* fetch origin/master

* VLAN is the preferred style for user interface usage

* does a missing libexpat-dev make travis builds complain?

* Revert "does a missing libexpat-dev make travis builds complain?"

This reverts commit 0cebc66f42708ff0f946213aab4bcbcc1b1b1379.
2019-01-14 21:57:37 +00:00
nick n
485abbc9bd trailing whitespace 2018-12-30 07:52:27 +01:00
nick n
8aed736925 ignore notpresent interfaces, with type filtering config item (set to [] to not filter). work done by postilion/nic bernstein, filter by me. also includes trailing space fix by nic. (#464) 2018-12-28 16:08:40 +00:00
nick n
86c38c9466 make job queue list in webinterface configurable (#469)
* add an option so you can define the amount of jobs returned in the webinterface jobq

* and now the actually working version
2018-12-28 16:01:40 +00:00
nick n
9f8dd0039b docs are maintained in wiki (#470)
per @ollyg:
"the manual files were all converted into GitHub wiki pages"
2018-12-28 14:32:05 +00:00
nick n
3474d028ec 1 more isdn interface to ignore
not sure if this is a typo or not, but bri singaling interfaces have 1 'l' on cisco devices.

"BRI0-Signaling" was taken from a netdisco report, router is a "897VaK9" running ios "15.5(3)M7".

as such, i've added the 1 'l' variant to the ignore_interfaces list. if this was indeed a typo perhaps the double 'll' variant can be deleted.
2018-11-28 02:43:15 +01:00
Oliver Gorwits
ea2fde1e56 #439 add expire_userlog with default 365 days 2018-10-19 15:11:26 +01:00
Oliver Gorwits
75c26717f0 fix error in regexp as identified by Nick / inphobia 2018-10-19 14:42:02 +01:00
nick n
19ed3d9747 Update config.yml (#438)
* Update config.yml

these are used internally in cisco ucs blade chassis, there is no reason i can think of to collect these.

* update config.yml, ignore ii1/1/1 interfaces

nexus 9000's have controllers to control whatever they run between their asics. for some reason they get exposed over snmp, but they are only needed in the internal fabric. as such, ignore them.

(they are actually called ii*/*/*, no fancy long names for them as for most other interfaces, if you want to find out more you'll need to google "higig2")
2018-10-19 14:40:58 +01:00
Oliver Gorwits
74825aff37 ignore some Microsoft interfaces (L. Ferguson) 2018-04-30 08:21:46 +01:00
Oliver Gorwits
75a199690c #400 add defanged_admin config to allow disabling of risky actions 2018-04-22 18:49:15 +01:00
Oliver Gorwits
a2ce65c8b8 working folters and colors for host group and location 2018-03-19 17:47:22 +00:00
Oliver Gorwits
b1bfb04bb1 add menu option to run stats job 2018-03-19 11:48:13 +00:00
Oliver Gorwits
74e6b2f685 more work on netmap sidebar 2018-03-10 19:30:24 +00:00
Oliver Gorwits
fe14a30261 set dynamic size on by default 2018-03-10 11:54:49 +00:00
Oliver Gorwits
66e3d2d64e #31 Configurable Free Time in Port Utilization Report 2018-03-10 10:45:39 +00:00
Oliver Gorwits
dd44877059 #48 Node Monitor supports matching on OUI 2018-03-10 09:26:23 +00:00