Commit Graph

61 Commits

Author SHA1 Message Date
Oliver Gorwits
826e1db39d API Endpoints to submit arpnip and macsuck results (#942)
* Add macsuck worker to collect various PortAccessEntity (NAC) attributes

* Incorporate PAE feedback on #937

 * missing Result/Device.pm column added
 * pae_is... columns instead of pae_capabilities
 * moved most code to Util/PortAccessEntity.pm so the update can
   be done in discover and macsuck

* Refactor PAE attributes during discover as separate Plugin

* PortAccessEntity: don't use device->dns in log string

* Fix "Experimental keys on scalar is now forbidden" test failure

* Revamp pae_control and add missing attribute

 - device.pae_control (text) is now device.pae_is_enabled (bool)
 - also store pae_authconfig_port_control (port mode auto/force(un)Auth)

* Fix "Experimental keys on scalar is now forbidden" test failure

 - ... again because of botched merge
 - at least perlgolfed away a set of curly braces

* Update PortAccessEntity.pm

* Incorporate @ollyg PR feedback

* allow actions without transport to run when there are also no creds

* initial refactor for separate gather, process, store phases for macsuck

* factor out the vlan sanity check

* additional help with log of action workers

* cleanup logic in check macsuck

* refactor to make main phases only

* some fixes

* implement file slurp. amazingly the whole thing works

* remove outdated noop from test

* treat error as critical, use cancel to suppress further drivers

* big refactor to share mac sanity code to both paths

* fix inverted logic on vlan sanity filter

* some code tidy

* fix error in default value

* fix for vlan 0 nodes input from cli

* ensure imported MACs are IEEE format

* add api endpoint, no useful return status yet

* exit status if error from nodes PUT

* suppress other networked workers when direct workers are active

* better log showing worker

* fix status recording to get first error or last done message

* implement arpnip API PUT

* avoid package redeclaration error

* make sure write API methods require admin status

* add doc for passing JSON data to arpnip and macsuck

* update manifest

* remove option to do jobs in web handler; all by queue now

* use job entry timestamp for offline queued jobs

* fix store username and IP on api PUT

* never de-duplicate user-submitted jobs; never reset DeviceSkip for offline jobs

* myworker no longer needed

* make logic cleaner

Co-authored-by: Christian Ramseyer <ramseyer@netnea.com>
2022-11-25 15:24:23 +00:00
Oliver Gorwits
dd468efdd5 #834 VLAN Mismatch report - add sysname, port comment, and vlan diff 2022-08-16 18:12:35 +01:00
Oliver Gorwits
e85198731f add remote dns to undiscovered neighbors report 2022-08-15 09:17:11 -04:00
Oliver Gorwits
1e2d9f36bf #912 use new all_oids format with additional status,enum,descr 2022-08-07 23:03:45 +01:00
Oliver Gorwits
2ae86b9fcd make the SNMP Browser faster and colour branches with data 2022-08-04 22:56:23 +01:00
Oliver Gorwits
94c43fcc33 #906 snmp browser is very slow sometimes 2022-08-03 17:43:23 +01:00
Oliver Gorwits
b40c30606e #900 make Port VLAN mismatch respect p_hide1002 setting 2022-08-03 10:25:23 +01:00
Oliver Gorwits
db7c480be4 #877 fix Port VLAN Mismatches report incorrect with Cisco subinterfaces 2022-08-02 20:25:50 +01:00
Oliver Gorwits
ec4cf74cea #872 separate column for NetBIOS name in IP Inventory 2022-07-27 22:48:36 +01:00
Oliver Gorwits
56ce4956cc #877 ignore vlan 1002-1005 in port vlan mismatch report 2022-07-26 21:09:34 +01:00
Oliver Gorwits
1d2e41975c #895 fix for neighbor showing phantom connections when remote_ip is not unique 2022-07-26 20:54:31 +01:00
Oliver Gorwits
a9b0d58b79 #897 port free should only apply to physical interfaces 2022-07-25 17:42:38 +01:00
Oliver Gorwits
e1bfd917a9 update undiscovered neighbors and port properties discovery to use remote_is_discoverable 2022-05-15 23:16:49 +01:00
Oliver Gorwits
dc1f76c1aa Feature to gather SNMP Walk, use as Pseudo Device, and Browse Objects
* fix anomalous name

* add gather worker

* fix encoding of binary storage

* store results back to job

* now parsing mbis report to translate

* fix the broken report parser

* rename gather to snapshot

* implement walk code copied from SNMP::Info

* can now bulkwalk and parse mibs report and store resolved walk in cache

* add func/glob aliasing broken

* better aliasing

* implement aliasing from globals and funcs

* fix regexp for matching netdisco-mibs report

* fake cache entry for all ND2 methods called, add comments

* also save to logs/snapshots/IP

* add doc for netdisco-do

* add is_pseudo column to device table

* support for loading cache for pseudo devices

* check for hrSystemUptime as well as sysUpTime for snmp connect

* display pseudo devices with yellow pill for name

* color all cells for layers for pseudo

* no need to b64 encode binary data in scalars as we b64 whole thing after

* tweaked uptime check

* store snapshot to database instead of Job

* expose snapshots in device details tab

* small ux improvements on snap download

* fixes for errors in subnet mask searching

* hide snapshot management for pseudo devices

* update to use new netdisco-mibs object cache

* update for new format oids file

* start of work on loading walk into db for browsing

* store values and meta

* add auto increment col and oid index to browser

* start web plugin for browser

* add virtual search for oid children

* have all oid in separte table (60 seconds load on my laptop)

* rename table and add relation

* store oid as int array

* fix sql for children

* make jstree start working

* working very slow tree expand

* fix to work when first displaying tree

* store both oid and oid_parts

* simplify SQL to speed up (more complicated perl)

* fix sql bug, add better index, prettify tree

* render the snmp node detail

* add node template, make scrollable, pretty print data values (insecure)

* store munge hint

* some dubious code to munge the data

* make sure to filter by IP on device_browser

* make safer the rendering of value data (but need to come back to key ordering)

* fix sorting on object values

* limit the opening of child nodes to keep response good and unclutter

* factor out the munge and make safer

* reject unknown mungers

* show the munger and option (not working) to change

* additional js for munge select

* complete custom munge

* change so that saving to database is only at CLI and on request

* hide snmp tab if no browser rows in the db

* add helpful message when no browser rows for the device

* stub handler for search and add recurse control

* working search

* minor ui fixes

* implement typeahead for leaf search

* limit rows in typeahead

* make sure device_browser is visited in delete and renumber

* add requirements for this branch

* update manifest

* make sure node search and typeahead are restricted to current device only
2021-11-06 07:47:29 +00:00
nick n
83a946d36e replace unicode nonbreaking space with ascii space (#794) 2021-06-23 22:00:33 +01:00
Oliver Gorwits
dff26abc5c API implementation (#712)
* initial v0 creator

* working json api for generic reports

* add require login

* move report swagger into plugin, and set new default layout of noop

* require proper role and also use new util func

* start to tidy authn

* some work on cleaning up web authn

* clean up the authN checks

* fix bug

* fix the auth for api

* fixes to json handling

* set swagger sort order

* enable most reports for api endpoints

* fix doc

* add paramters to reports

* add missed report

* allow api_parameters in reports config

* reorganise api

* add vlan search

* add port search

* make sure to enable layout processing

* add device search

* add v1 to api paths

* add Node Search

* support api_responses

* add device object search; fix spurious ports field in device result class

* handle some plugins just returning undef if search fails

* errors from api seamlessley

* fix error in date range default

* more sensible default for prefix

* change order of endpoints in swagger-ui

* all db row classes can now TO_JSON

* add device_port api endpoint

* add device ports endpoint

* do not expand docs

* add swagger ui json tree formatter

* add all relations from Device table

* add port relations

* add nodes retrieve on device or vlan

* rename to GetAPIKey

* update config for previous commit
2020-04-15 21:15:52 +01:00
Christian Ramseyer
031c3e6d95 Optimize PortMAC query
* We meant well but it turns out that the array unnest and join is
  actually very slow, as the join arguments do not get pushed down
  into the CTE (in Postgres 9/10 at least, later versions remove some
  of the optimization barriers in that specifc type of query)
* This caused a seq scan on both device and device_port, and the query
  is executed many times during macsuck
* The query is now rewritten to use ANY (macaddr[]) and without CTE,
  which seems to be around 20x faster
2020-01-31 10:32:02 +01:00
Oliver Gorwits
3044d7ebe6 fixes to make the PortMacs query work 2020-01-19 15:08:38 +00:00
Oliver Gorwits
0eab9213c0 should be faster than using IN on large lists 2019-12-21 21:32:27 +00:00
Oliver Gorwits
4e3bfee214 Add TACACS+ support per #660 (#662) 2019-10-15 18:23:51 +01:00
Oliver Gorwits
542837d266 #591 domain_suffix can be list and supports (simple) regexp
Squashed commit of the following:

commit 975e4c6afc
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 3 13:35:26 2019 +0100

    also support regexp in domain_suffix

commit 0a2b5c8fa2
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 3 13:17:17 2019 +0100

    fix rancid, graph, and nodemonitor

commit 6d881de3ff
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 3 13:11:54 2019 +0100

    improve docs and set default domain_suffix to be list

commit 1dcafc08a8
Merge: 9a752e02 14ac69dc
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 3 13:00:39 2019 +0100

    Merge branch 'master' into og-multiple-domain-suffix

commit 9a752e0298
Merge: c836619f 82a99ea9
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 3 09:45:25 2019 +0100

    Merge branch 'master' into og-multiple-domain-suffix

commit c836619f8c
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Thu Jun 13 07:52:45 2019 +0100

    hokey fix for nodes with domains

commit fed14bd810
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Thu Jun 13 07:02:09 2019 +0100

    basic implementation, rancid graph and nodemonitor missing
2019-09-03 13:35:43 +01:00
Oliver Gorwits
86daf7174d add comments 2019-08-23 09:25:17 +01:00
Oliver Gorwits
d45745dd1d fixed only free ports in device ports view (showing free ports TODO) 2019-08-20 11:13:52 +01:00
Oliver Gorwits
22302114cb working free ports after reboot but bug: row expansion 2019-08-19 23:25:06 +01:00
Oliver Gorwits
c73c3e7569 improvements to radius patch 2019-08-18 09:16:57 +01:00
Oliver Gorwits
88788534ba #589 only show link with highest agg speed if map links between two devices are asymmetric 2019-05-27 15:36:17 +01:00
Oliver Gorwits
087ca8e33c #586 do not show poller performance rows for still running jobs 2019-05-27 14:23:09 +01:00
Oliver Gorwits
46d8c52a08 Implement changes for API authentication and Swagger UI (#541)
* initial token-based-api login handler

* add token schema and validation

* initial import of pyro3d api code

* basic Swagger spec support

* Merge in working copy of API/Device.pm

* Fix some error handling for API/Device.pm

* Break out utility functions into separate file, to allow other api portions to use

* Add NodeIP support.

* Add nodeip plugin to config

* remove double define of "plugin:" (#448)

disclaimer: i did not test this is any way, came across it when looking for something else.

* only AuthZ header for api use, and alway regen key on login

* use RFC7235

* workaround for Swagger plugin weird response body

* do not autodiscover swagger routes

* code formatting only

* move api util to utils area

* initial full swagger spec for nodeip search

* add api user role and fix api auth failure response

* update version of swagger-ui to 3.20.3

* add more openapi defs

* fixes to SQL and api spec

* clean up subs

* improvements to login/logout for API

* make api logout work

* add openapi tags to group operations

* allow api params to be generated from DBIC schema spec

* remove API calls for nodes and devices

* remove some poor assumptions about api calls

* tidy up

* remove DDP

* make login and logout similar

* example of api call being handled by ajax call

* make the branch authonly
2019-03-17 20:27:19 +00:00
Oliver Gorwits
6cdfd80d10 allow undiscovered neighbors report to use discover_{waps,phones} setting 2019-03-03 14:34:00 +00:00
Oliver Gorwits
a643820a62 #428 Port-Channels not showing in netmap 2019-03-02 21:54:27 +00:00
Oliver Gorwits
d614382fcd #436 new SQL query for neigbor map to allow mismatched/one-way links (Linwood-F) 2019-01-02 20:37:19 +00:00
Oliver Gorwits
e39cf197bf further tune job picker 2018-06-17 21:55:18 +01:00
Oliver Gorwits
0cce3b3834 jobs with username are only allowed one attempt to unskip 2018-06-07 23:17:13 +01:00
Oliver Gorwits
6098b893ad #410 improvements to Undiscovered Neighbors report 2018-05-07 17:15:11 +01:00
Oliver Gorwits
0193d453ec also fix DevicePortSpeed 2018-04-22 12:44:43 +01:00
Oliver Gorwits
cab129a579 #380 should check descr not name for "vlan" 2018-04-22 11:54:36 +01:00
Oliver Gorwits
ea592c6b28 #398 user submitted jobs are run regardless of max deferrals 2018-04-18 21:57:26 +01:00
Oliver Gorwits
642820d25b try to match remote port in netmap against port, name, and descr 2018-04-10 21:45:41 +01:00
Oliver Gorwits
60e22dda3c Merge branch 'master' into og-remoteport-fix 2018-04-08 13:13:04 +01:00
Oliver Gorwits
02c9da4e99 do not require remote_port to exist at remote_ip
this has a bug whereby if the list of remote_ports is not the same
size as the list of real matching ports, then the list of port names
will be short, and port descriptions may be assigned to the wrong
ports, or the query may crash. needs testing.
2018-04-08 00:55:36 +01:00
Oliver Gorwits
97d5501ba0 do not include logical aggregate masters in netmap/speed calc 2018-04-07 23:59:59 +01:00
Oliver Gorwits
927f2f0a74 #390 cannot take logarithm of zero (C. Stromsoe) 2018-04-07 23:27:11 +01:00
Oliver Gorwits
850610f6b0 network map working with pseudo devices 2018-03-22 21:22:58 +00:00
Oliver Gorwits
a2ce65c8b8 working folters and colors for host group and location 2018-03-19 17:47:22 +00:00
Oliver Gorwits
43edd06b56 use port type filter for links instead of ethernetCsmacd
per is_vlan_interface from App::Netdisco::Util::Port
see also https://www.iana.org/assignments/ianaiftype-mib/ianaiftype-mib
2018-03-19 14:30:11 +00:00
Oliver Gorwits
5d8475822b speed should be 0 on links with no data, to hide label 2018-03-10 11:42:01 +00:00
Oliver Gorwits
147ef33d5e dynamic size in neighbor map should use device_port_properties 2018-03-10 11:38:41 +00:00
Oliver Gorwits
9fa8372685 #274 errors in IP Inventory report 2018-03-10 11:32:02 +00:00
Oliver Gorwits
66e3d2d64e #31 Configurable Free Time in Port Utilization Report 2018-03-10 10:45:39 +00:00
Oliver Gorwits
dd44877059 #48 Node Monitor supports matching on OUI 2018-03-10 09:26:23 +00:00