Commit Graph

154 Commits

Author SHA1 Message Date
Christian Ramseyer
8cc8bdcd19 Add NAC columns to UI (#941)
* Add NAC columns to UI

* Compact variant with a "NAC Summary" column

* display PAE stuff as very short summary in web and full columbs in csv

* add tooltips for actual port status and state detail

* minor naming tweaks

Co-authored-by: Oliver Gorwits <oliver@cpan.org>
2022-12-13 09:16:32 +01:00
Oliver Gorwits
1c7c749f0e custom fields on devices and ports in the web from config (#945)
* custom device field web display and edit

* make display work; relies on T::T calling dict slot or method with same syntax

* add storing port custom fields

* use resultset method instead, use cf_ prefix

* update Pg min ver for jsonb

* allow override of position and default for port custom fields

* support hidden for custom fields

* update description of Objects API class

* allow left and mid position for custom fields

* add custom fields in csv

* change port control sidebar label

* fix default missing bug on backend jobs
2022-12-09 10:20:26 +00:00
Oliver Gorwits
d03eab02db #944 DevicePort.pm should use vlan_entry instead of vlan 2022-12-06 19:11:51 +00:00
Oliver Gorwits
826e1db39d API Endpoints to submit arpnip and macsuck results (#942)
* Add macsuck worker to collect various PortAccessEntity (NAC) attributes

* Incorporate PAE feedback on #937

 * missing Result/Device.pm column added
 * pae_is... columns instead of pae_capabilities
 * moved most code to Util/PortAccessEntity.pm so the update can
   be done in discover and macsuck

* Refactor PAE attributes during discover as separate Plugin

* PortAccessEntity: don't use device->dns in log string

* Fix "Experimental keys on scalar is now forbidden" test failure

* Revamp pae_control and add missing attribute

 - device.pae_control (text) is now device.pae_is_enabled (bool)
 - also store pae_authconfig_port_control (port mode auto/force(un)Auth)

* Fix "Experimental keys on scalar is now forbidden" test failure

 - ... again because of botched merge
 - at least perlgolfed away a set of curly braces

* Update PortAccessEntity.pm

* Incorporate @ollyg PR feedback

* allow actions without transport to run when there are also no creds

* initial refactor for separate gather, process, store phases for macsuck

* factor out the vlan sanity check

* additional help with log of action workers

* cleanup logic in check macsuck

* refactor to make main phases only

* some fixes

* implement file slurp. amazingly the whole thing works

* remove outdated noop from test

* treat error as critical, use cancel to suppress further drivers

* big refactor to share mac sanity code to both paths

* fix inverted logic on vlan sanity filter

* some code tidy

* fix error in default value

* fix for vlan 0 nodes input from cli

* ensure imported MACs are IEEE format

* add api endpoint, no useful return status yet

* exit status if error from nodes PUT

* suppress other networked workers when direct workers are active

* better log showing worker

* fix status recording to get first error or last done message

* implement arpnip API PUT

* avoid package redeclaration error

* make sure write API methods require admin status

* add doc for passing JSON data to arpnip and macsuck

* update manifest

* remove option to do jobs in web handler; all by queue now

* use job entry timestamp for offline queued jobs

* fix store username and IP on api PUT

* never de-duplicate user-submitted jobs; never reset DeviceSkip for offline jobs

* myworker no longer needed

* make logic cleaner

Co-authored-by: Christian Ramseyer <ramseyer@netnea.com>
2022-11-25 15:24:23 +00:00
Oliver Gorwits
0e9ff81cf5 Add worker to collect various PortAccessEntity (NAC) attributes (PR #937, partially implements #887)
* Add macsuck worker to collect various PortAccessEntity (NAC) attributes

* Incorporate PAE feedback on #937

 * missing Result/Device.pm column added
 * pae_is... columns instead of pae_capabilities
 * moved most code to Util/PortAccessEntity.pm so the update can
   be done in discover and macsuck

* Refactor PAE attributes during discover as separate Plugin

* PortAccessEntity: don't use device->dns in log string

* Fix "Experimental keys on scalar is now forbidden" test failure

* Revamp pae_control and add missing attribute

 - device.pae_control (text) is now device.pae_is_enabled (bool)
 - also store pae_authconfig_port_control (port mode auto/force(un)Auth)

* Fix "Experimental keys on scalar is now forbidden" test failure

 - ... again because of botched merge
 - at least perlgolfed away a set of curly braces

* Update PortAccessEntity.pm

* Incorporate @ollyg PR feedback

Co-authored-by: Christian Ramseyer <ramseyer@netnea.com>
2022-11-04 10:03:26 +01:00
Oliver Gorwits
790c51b257 #925 implement ignore_deviceports and hide_deviceports 2022-09-20 20:32:35 +01:00
Oliver Gorwits
73862d5ac8 fix device sysname showing in node search 2022-08-17 06:09:43 -04:00
Oliver Gorwits
4d49f04d2a #875 Node Search - swap IP and DNS, add system name, highlight most recent, in the Switch Port rows 2022-08-16 22:25:27 +01:00
Oliver Gorwits
dd468efdd5 #834 VLAN Mismatch report - add sysname, port comment, and vlan diff 2022-08-16 18:12:35 +01:00
Oliver Gorwits
e85198731f add remote dns to undiscovered neighbors report 2022-08-15 09:17:11 -04:00
Oliver Gorwits
b01d0f08fd add a remote_dns port property 2022-08-14 21:36:50 +01:00
Oliver Gorwits
3684451f2a add ifindex column for show 2022-08-14 21:13:27 +01:00
Oliver Gorwits
1e2d9f36bf #912 use new all_oids format with additional status,enum,descr 2022-08-07 23:03:45 +01:00
Oliver Gorwits
2ae86b9fcd make the SNMP Browser faster and colour branches with data 2022-08-04 22:56:23 +01:00
Oliver Gorwits
94c43fcc33 #906 snmp browser is very slow sometimes 2022-08-03 17:43:23 +01:00
Oliver Gorwits
b40c30606e #900 make Port VLAN mismatch respect p_hide1002 setting 2022-08-03 10:25:23 +01:00
Oliver Gorwits
db7c480be4 #877 fix Port VLAN Mismatches report incorrect with Cisco subinterfaces 2022-08-02 20:25:50 +01:00
Oliver Gorwits
a1aac2777d use i_subinterfaces to group vlan subinterfaces like link aggregates in device ports view 2022-08-02 18:22:36 +01:00
Oliver Gorwits
2c5c344ece fix bug in vlan names feature missing ports without vlan table entry 2022-08-01 17:48:50 +01:00
Oliver Gorwits
ec4cf74cea #872 separate column for NetBIOS name in IP Inventory 2022-07-27 22:48:36 +01:00
Oliver Gorwits
5ecb09dcb7 option to renumber to existing interface address 2022-07-26 22:32:58 +01:00
Oliver Gorwits
56ce4956cc #877 ignore vlan 1002-1005 in port vlan mismatch report 2022-07-26 21:09:34 +01:00
Oliver Gorwits
1d2e41975c #895 fix for neighbor showing phantom connections when remote_ip is not unique 2022-07-26 20:54:31 +01:00
Oliver Gorwits
7566e64886 #882 option to display vlan names instead of numbers in device ports table 2022-07-26 09:49:07 +01:00
Oliver Gorwits
a9b0d58b79 #897 port free should only apply to physical interfaces 2022-07-25 17:42:38 +01:00
Oliver Gorwits
e1bfd917a9 update undiscovered neighbors and port properties discovery to use remote_is_discoverable 2022-05-15 23:16:49 +01:00
Oliver Gorwits
1a810d89f3 support for storing remote_is_discoverable on a port 2022-05-15 22:17:36 +01:00
Oliver Gorwits
dc1f76c1aa Feature to gather SNMP Walk, use as Pseudo Device, and Browse Objects
* fix anomalous name

* add gather worker

* fix encoding of binary storage

* store results back to job

* now parsing mbis report to translate

* fix the broken report parser

* rename gather to snapshot

* implement walk code copied from SNMP::Info

* can now bulkwalk and parse mibs report and store resolved walk in cache

* add func/glob aliasing broken

* better aliasing

* implement aliasing from globals and funcs

* fix regexp for matching netdisco-mibs report

* fake cache entry for all ND2 methods called, add comments

* also save to logs/snapshots/IP

* add doc for netdisco-do

* add is_pseudo column to device table

* support for loading cache for pseudo devices

* check for hrSystemUptime as well as sysUpTime for snmp connect

* display pseudo devices with yellow pill for name

* color all cells for layers for pseudo

* no need to b64 encode binary data in scalars as we b64 whole thing after

* tweaked uptime check

* store snapshot to database instead of Job

* expose snapshots in device details tab

* small ux improvements on snap download

* fixes for errors in subnet mask searching

* hide snapshot management for pseudo devices

* update to use new netdisco-mibs object cache

* update for new format oids file

* start of work on loading walk into db for browsing

* store values and meta

* add auto increment col and oid index to browser

* start web plugin for browser

* add virtual search for oid children

* have all oid in separte table (60 seconds load on my laptop)

* rename table and add relation

* store oid as int array

* fix sql for children

* make jstree start working

* working very slow tree expand

* fix to work when first displaying tree

* store both oid and oid_parts

* simplify SQL to speed up (more complicated perl)

* fix sql bug, add better index, prettify tree

* render the snmp node detail

* add node template, make scrollable, pretty print data values (insecure)

* store munge hint

* some dubious code to munge the data

* make sure to filter by IP on device_browser

* make safer the rendering of value data (but need to come back to key ordering)

* fix sorting on object values

* limit the opening of child nodes to keep response good and unclutter

* factor out the munge and make safer

* reject unknown mungers

* show the munger and option (not working) to change

* additional js for munge select

* complete custom munge

* change so that saving to database is only at CLI and on request

* hide snmp tab if no browser rows in the db

* add helpful message when no browser rows for the device

* stub handler for search and add recurse control

* working search

* minor ui fixes

* implement typeahead for leaf search

* limit rows in typeahead

* make sure device_browser is visited in delete and renumber

* add requirements for this branch

* update manifest

* make sure node search and typeahead are restricted to current device only
2021-11-06 07:47:29 +00:00
Oliver Gorwits
d8a9ad9007 #808 avoid dns for fuzzy search on IPish thing 2021-10-06 08:28:56 +01:00
Oliver Gorwits
a040e8ae38 #814 when deleting device, avoid deleting jobs running on the device 2021-10-04 19:35:42 +01:00
Oliver Gorwits
3e6ec6883f #809 fix crash in Node search due to regexp translation perl to SQL 2021-09-08 20:39:50 +01:00
Oliver Gorwits
2498d9766b #800 gather and report SNMP Chassis ID http://oid-info.com/cgi-bin/display?oid=1.3.6.1.4.1.9.3.6.3&a=display 2021-08-14 13:28:13 +01:00
Oliver Gorwits
d206b9ccbf a bunch of logic changes to allow tests to run standalone 2021-08-12 10:58:55 +01:00
nick n
83a946d36e replace unicode nonbreaking space with ascii space (#794) 2021-06-23 22:00:33 +01:00
Oliver Gorwits
2e6bfa42b4 fix error when using API /api/v1/object/device/ip/port_vlans 2021-02-14 14:04:08 +00:00
Oliver Gorwits
facc41e310 release 2.046005 2020-10-17 14:40:07 +01:00
Oliver Gorwits
2b72233738 release 2.046004 2020-10-17 14:28:53 +01:00
Oliver Gorwits
c7b8c3bcbd #748 always call NetAddr::MAC->new() with named mac param 2020-10-17 13:05:46 +01:00
Oliver Gorwits
3aeed20b78 #735 Slow web search for devices 2020-08-06 21:25:01 +01:00
Oliver Gorwits
f882f98319 #721 redux - better MAC address support in searching 2020-06-30 09:46:19 +01:00
Oliver Gorwits
dc87a50076 #728 store SNMP Engine ID 2020-06-29 14:38:51 +01:00
Oliver Gorwits
4f57a4d30a rename ports column in device table to be num_ports 2020-04-18 16:50:49 +01:00
Oliver Gorwits
524f4792c3 restore port column needed for discovery 2020-04-18 09:47:05 +01:00
Oliver Gorwits
dff26abc5c API implementation (#712)
* initial v0 creator

* working json api for generic reports

* add require login

* move report swagger into plugin, and set new default layout of noop

* require proper role and also use new util func

* start to tidy authn

* some work on cleaning up web authn

* clean up the authN checks

* fix bug

* fix the auth for api

* fixes to json handling

* set swagger sort order

* enable most reports for api endpoints

* fix doc

* add paramters to reports

* add missed report

* allow api_parameters in reports config

* reorganise api

* add vlan search

* add port search

* make sure to enable layout processing

* add device search

* add v1 to api paths

* add Node Search

* support api_responses

* add device object search; fix spurious ports field in device result class

* handle some plugins just returning undef if search fails

* errors from api seamlessley

* fix error in date range default

* more sensible default for prefix

* change order of endpoints in swagger-ui

* all db row classes can now TO_JSON

* add device_port api endpoint

* add device ports endpoint

* do not expand docs

* add swagger ui json tree formatter

* add all relations from Device table

* add port relations

* add nodes retrieve on device or vlan

* rename to GetAPIKey

* update config for previous commit
2020-04-15 21:15:52 +01:00
Christian Ramseyer
031c3e6d95 Optimize PortMAC query
* We meant well but it turns out that the array unnest and join is
  actually very slow, as the join arguments do not get pushed down
  into the CTE (in Postgres 9/10 at least, later versions remove some
  of the optimization barriers in that specifc type of query)
* This caused a seq scan on both device and device_port, and the query
  is executed many times during macsuck
* The query is now rewritten to use ANY (macaddr[]) and without CTE,
  which seems to be around 20x faster
2020-01-31 10:32:02 +01:00
Oliver Gorwits
db6e89607d #696 complete implementation of #688 2020-01-25 18:07:43 +00:00
Oliver Gorwits
3044d7ebe6 fixes to make the PortMacs query work 2020-01-19 15:08:38 +00:00
Oliver Gorwits
c282a5a565 #688 Amend vlan search to show number of ports actually using the vlan 2019-12-22 08:33:11 +00:00
Oliver Gorwits
d133eb97c2 #667 Network Map: Filter by VLAN doesnt work as expected 2019-12-22 00:16:04 +00:00
Oliver Gorwits
0eab9213c0 should be faster than using IN on large lists 2019-12-21 21:32:27 +00:00