Commit Graph

48 Commits

Author SHA1 Message Date
Oliver Gorwits
dc1f76c1aa Feature to gather SNMP Walk, use as Pseudo Device, and Browse Objects
* fix anomalous name

* add gather worker

* fix encoding of binary storage

* store results back to job

* now parsing mbis report to translate

* fix the broken report parser

* rename gather to snapshot

* implement walk code copied from SNMP::Info

* can now bulkwalk and parse mibs report and store resolved walk in cache

* add func/glob aliasing broken

* better aliasing

* implement aliasing from globals and funcs

* fix regexp for matching netdisco-mibs report

* fake cache entry for all ND2 methods called, add comments

* also save to logs/snapshots/IP

* add doc for netdisco-do

* add is_pseudo column to device table

* support for loading cache for pseudo devices

* check for hrSystemUptime as well as sysUpTime for snmp connect

* display pseudo devices with yellow pill for name

* color all cells for layers for pseudo

* no need to b64 encode binary data in scalars as we b64 whole thing after

* tweaked uptime check

* store snapshot to database instead of Job

* expose snapshots in device details tab

* small ux improvements on snap download

* fixes for errors in subnet mask searching

* hide snapshot management for pseudo devices

* update to use new netdisco-mibs object cache

* update for new format oids file

* start of work on loading walk into db for browsing

* store values and meta

* add auto increment col and oid index to browser

* start web plugin for browser

* add virtual search for oid children

* have all oid in separte table (60 seconds load on my laptop)

* rename table and add relation

* store oid as int array

* fix sql for children

* make jstree start working

* working very slow tree expand

* fix to work when first displaying tree

* store both oid and oid_parts

* simplify SQL to speed up (more complicated perl)

* fix sql bug, add better index, prettify tree

* render the snmp node detail

* add node template, make scrollable, pretty print data values (insecure)

* store munge hint

* some dubious code to munge the data

* make sure to filter by IP on device_browser

* make safer the rendering of value data (but need to come back to key ordering)

* fix sorting on object values

* limit the opening of child nodes to keep response good and unclutter

* factor out the munge and make safer

* reject unknown mungers

* show the munger and option (not working) to change

* additional js for munge select

* complete custom munge

* change so that saving to database is only at CLI and on request

* hide snmp tab if no browser rows in the db

* add helpful message when no browser rows for the device

* stub handler for search and add recurse control

* working search

* minor ui fixes

* implement typeahead for leaf search

* limit rows in typeahead

* make sure device_browser is visited in delete and renumber

* add requirements for this branch

* update manifest

* make sure node search and typeahead are restricted to current device only
2021-11-06 07:47:29 +00:00
nick n
83a946d36e replace unicode nonbreaking space with ascii space (#794) 2021-06-23 22:00:33 +01:00
Oliver Gorwits
dff26abc5c API implementation (#712)
* initial v0 creator

* working json api for generic reports

* add require login

* move report swagger into plugin, and set new default layout of noop

* require proper role and also use new util func

* start to tidy authn

* some work on cleaning up web authn

* clean up the authN checks

* fix bug

* fix the auth for api

* fixes to json handling

* set swagger sort order

* enable most reports for api endpoints

* fix doc

* add paramters to reports

* add missed report

* allow api_parameters in reports config

* reorganise api

* add vlan search

* add port search

* make sure to enable layout processing

* add device search

* add v1 to api paths

* add Node Search

* support api_responses

* add device object search; fix spurious ports field in device result class

* handle some plugins just returning undef if search fails

* errors from api seamlessley

* fix error in date range default

* more sensible default for prefix

* change order of endpoints in swagger-ui

* all db row classes can now TO_JSON

* add device_port api endpoint

* add device ports endpoint

* do not expand docs

* add swagger ui json tree formatter

* add all relations from Device table

* add port relations

* add nodes retrieve on device or vlan

* rename to GetAPIKey

* update config for previous commit
2020-04-15 21:15:52 +01:00
Christian Ramseyer
031c3e6d95 Optimize PortMAC query
* We meant well but it turns out that the array unnest and join is
  actually very slow, as the join arguments do not get pushed down
  into the CTE (in Postgres 9/10 at least, later versions remove some
  of the optimization barriers in that specifc type of query)
* This caused a seq scan on both device and device_port, and the query
  is executed many times during macsuck
* The query is now rewritten to use ANY (macaddr[]) and without CTE,
  which seems to be around 20x faster
2020-01-31 10:32:02 +01:00
Oliver Gorwits
3044d7ebe6 fixes to make the PortMacs query work 2020-01-19 15:08:38 +00:00
Oliver Gorwits
0eab9213c0 should be faster than using IN on large lists 2019-12-21 21:32:27 +00:00
Oliver Gorwits
4e3bfee214 Add TACACS+ support per #660 (#662) 2019-10-15 18:23:51 +01:00
Oliver Gorwits
542837d266 #591 domain_suffix can be list and supports (simple) regexp
Squashed commit of the following:

commit 975e4c6afc
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 3 13:35:26 2019 +0100

    also support regexp in domain_suffix

commit 0a2b5c8fa2
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 3 13:17:17 2019 +0100

    fix rancid, graph, and nodemonitor

commit 6d881de3ff
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 3 13:11:54 2019 +0100

    improve docs and set default domain_suffix to be list

commit 1dcafc08a8
Merge: 9a752e02 14ac69dc
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 3 13:00:39 2019 +0100

    Merge branch 'master' into og-multiple-domain-suffix

commit 9a752e0298
Merge: c836619f 82a99ea9
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 3 09:45:25 2019 +0100

    Merge branch 'master' into og-multiple-domain-suffix

commit c836619f8c
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Thu Jun 13 07:52:45 2019 +0100

    hokey fix for nodes with domains

commit fed14bd810
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Thu Jun 13 07:02:09 2019 +0100

    basic implementation, rancid graph and nodemonitor missing
2019-09-03 13:35:43 +01:00
Oliver Gorwits
86daf7174d add comments 2019-08-23 09:25:17 +01:00
Oliver Gorwits
d45745dd1d fixed only free ports in device ports view (showing free ports TODO) 2019-08-20 11:13:52 +01:00
Oliver Gorwits
22302114cb working free ports after reboot but bug: row expansion 2019-08-19 23:25:06 +01:00
Oliver Gorwits
c73c3e7569 improvements to radius patch 2019-08-18 09:16:57 +01:00
Oliver Gorwits
88788534ba #589 only show link with highest agg speed if map links between two devices are asymmetric 2019-05-27 15:36:17 +01:00
Oliver Gorwits
087ca8e33c #586 do not show poller performance rows for still running jobs 2019-05-27 14:23:09 +01:00
Oliver Gorwits
46d8c52a08 Implement changes for API authentication and Swagger UI (#541)
* initial token-based-api login handler

* add token schema and validation

* initial import of pyro3d api code

* basic Swagger spec support

* Merge in working copy of API/Device.pm

* Fix some error handling for API/Device.pm

* Break out utility functions into separate file, to allow other api portions to use

* Add NodeIP support.

* Add nodeip plugin to config

* remove double define of "plugin:" (#448)

disclaimer: i did not test this is any way, came across it when looking for something else.

* only AuthZ header for api use, and alway regen key on login

* use RFC7235

* workaround for Swagger plugin weird response body

* do not autodiscover swagger routes

* code formatting only

* move api util to utils area

* initial full swagger spec for nodeip search

* add api user role and fix api auth failure response

* update version of swagger-ui to 3.20.3

* add more openapi defs

* fixes to SQL and api spec

* clean up subs

* improvements to login/logout for API

* make api logout work

* add openapi tags to group operations

* allow api params to be generated from DBIC schema spec

* remove API calls for nodes and devices

* remove some poor assumptions about api calls

* tidy up

* remove DDP

* make login and logout similar

* example of api call being handled by ajax call

* make the branch authonly
2019-03-17 20:27:19 +00:00
Oliver Gorwits
6cdfd80d10 allow undiscovered neighbors report to use discover_{waps,phones} setting 2019-03-03 14:34:00 +00:00
Oliver Gorwits
a643820a62 #428 Port-Channels not showing in netmap 2019-03-02 21:54:27 +00:00
Oliver Gorwits
d614382fcd #436 new SQL query for neigbor map to allow mismatched/one-way links (Linwood-F) 2019-01-02 20:37:19 +00:00
Oliver Gorwits
e39cf197bf further tune job picker 2018-06-17 21:55:18 +01:00
Oliver Gorwits
0cce3b3834 jobs with username are only allowed one attempt to unskip 2018-06-07 23:17:13 +01:00
Oliver Gorwits
6098b893ad #410 improvements to Undiscovered Neighbors report 2018-05-07 17:15:11 +01:00
Oliver Gorwits
0193d453ec also fix DevicePortSpeed 2018-04-22 12:44:43 +01:00
Oliver Gorwits
cab129a579 #380 should check descr not name for "vlan" 2018-04-22 11:54:36 +01:00
Oliver Gorwits
ea592c6b28 #398 user submitted jobs are run regardless of max deferrals 2018-04-18 21:57:26 +01:00
Oliver Gorwits
642820d25b try to match remote port in netmap against port, name, and descr 2018-04-10 21:45:41 +01:00
Oliver Gorwits
60e22dda3c Merge branch 'master' into og-remoteport-fix 2018-04-08 13:13:04 +01:00
Oliver Gorwits
02c9da4e99 do not require remote_port to exist at remote_ip
this has a bug whereby if the list of remote_ports is not the same
size as the list of real matching ports, then the list of port names
will be short, and port descriptions may be assigned to the wrong
ports, or the query may crash. needs testing.
2018-04-08 00:55:36 +01:00
Oliver Gorwits
97d5501ba0 do not include logical aggregate masters in netmap/speed calc 2018-04-07 23:59:59 +01:00
Oliver Gorwits
927f2f0a74 #390 cannot take logarithm of zero (C. Stromsoe) 2018-04-07 23:27:11 +01:00
Oliver Gorwits
850610f6b0 network map working with pseudo devices 2018-03-22 21:22:58 +00:00
Oliver Gorwits
a2ce65c8b8 working folters and colors for host group and location 2018-03-19 17:47:22 +00:00
Oliver Gorwits
43edd06b56 use port type filter for links instead of ethernetCsmacd
per is_vlan_interface from App::Netdisco::Util::Port
see also https://www.iana.org/assignments/ianaiftype-mib/ianaiftype-mib
2018-03-19 14:30:11 +00:00
Oliver Gorwits
5d8475822b speed should be 0 on links with no data, to hide label 2018-03-10 11:42:01 +00:00
Oliver Gorwits
147ef33d5e dynamic size in neighbor map should use device_port_properties 2018-03-10 11:38:41 +00:00
Oliver Gorwits
9fa8372685 #274 errors in IP Inventory report 2018-03-10 11:32:02 +00:00
Oliver Gorwits
66e3d2d64e #31 Configurable Free Time in Port Utilization Report 2018-03-10 10:45:39 +00:00
Oliver Gorwits
dd44877059 #48 Node Monitor supports matching on OUI 2018-03-10 09:26:23 +00:00
Oliver Gorwits
8a90236984 add missing col to schema 2018-03-05 22:46:33 +00:00
Oliver Gorwits
66ecb203a7 #370 Missing Map Links due to inability to parse port speeds 2018-03-05 21:42:02 +00:00
Oliver Gorwits
93155d3f26 move random() in TastyJobs to where it is more useful 2018-02-23 23:54:08 +00:00
Oliver Gorwits
15a38edbc1 tune the job picking sql to avoid user jobs ignoring skips 2018-02-16 08:22:07 +00:00
Oliver Gorwits
bfc34049db handle jobs with device key first 2018-02-11 21:53:29 +00:00
Oliver Gorwits
a024716495 fix error in jobs pick 2018-02-10 19:48:44 +00:00
Oliver Gorwits
f71aae75fd web jobs are highest prio; make sql cleaner 2018-02-10 16:33:46 +00:00
Oliver Gorwits
160d3f2a05 pick jobs from queue with lowest retries first 2018-02-10 10:49:27 +00:00
Oliver Gorwits
d332eaf293 new network map (neighbors) implementation
also link to manual topology from device ports
fix tooltip reinit on datatables paging

Squashed commit of the following:

commit 85d52ed174
Merge: 66e35366 07446ed7
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Dec 31 08:59:19 2017 +0000

    Merge branch 'master' into og-newnetmap

commit 66e353669d
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Dec 31 00:16:49 2017 +0000

    add docs link

commit 809f1ede76
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Dec 31 00:11:29 2017 +0000

    node links to other devices

commit 187e1ea803
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Dec 30 23:58:56 2017 +0000

    add node infostring

commit 6453f99b28
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Dec 30 23:41:19 2017 +0000

    link infostrings!

commit f3ba260142
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Dec 30 22:42:09 2017 +0000

    maxnoderadius is now based on dynamic size calc

commit 0af415e391
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Dec 30 22:33:22 2017 +0000

    fix to get aggregate ports in matching order on left and right side

commit 3f36121a4c
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Dec 30 22:07:28 2017 +0000

    initial aggregate link summing

commit e5b96b0781
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Dec 30 14:55:00 2017 +0000

    tidy and refactor

commit 79f22a47b3
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Dec 30 13:42:58 2017 +0000

    get rid of d3 v3/v4 and only have v3

commit 18c3f16c8f
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Dec 30 12:54:45 2017 +0000

    fullscreen exit now works

commit 93f8181fc8
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Dec 30 08:58:04 2017 +0000

    implement show link speeds on links

commit 72961f172d
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Thu Dec 28 23:51:26 2017 +0000

    add speed to retrieved devicelinks

commit ae5496310e
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Dec 26 11:21:12 2017 +0000

    remove annealing on label change

commit 885cb92bd3
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Dec 26 11:15:21 2017 +0000

    reorder options

commit 015a1debe0
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Dec 26 11:11:01 2017 +0000

    fix d3-force debug link messing up svg text elements

commit 10ccaa78b1
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Dec 26 10:52:47 2017 +0000

    show ips functionality; improvement to maximise icon; bugfix d3-force label wrap

commit 71a1a20909
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Dec 25 13:21:23 2017 +0000

    move labels on neighbor view

commit 9578215349
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Dec 25 13:13:13 2017 +0000

    force running spinner

commit 6618fba548
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Dec 24 23:13:33 2017 +0000

    fix neighbors pinning

commit 938848551e
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Dec 24 18:12:04 2017 +0000

    implement dynamic sizing option

commit e4b19be5d7
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Dec 24 13:25:00 2017 +0000

    zoom to device on neighbor show; do not save positions on neighbor show

commit 58b576b464
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Dec 24 13:10:48 2017 +0000

    better neighbors/save interaction

commit 2c77660f2f
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Dec 24 13:03:46 2017 +0000

    fix labels on checkboxes to be clickable

commit 2c2dc62cda
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Dec 24 12:50:26 2017 +0000

    prevent save positions when in neighbor view mode

commit 78c558b5be
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Dec 24 12:37:51 2017 +0000

    add tooltip to show why host groups radio is disabled

commit 7000d4b0eb
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Dec 24 12:12:29 2017 +0000

    allow port control role to edit manual topology

commit 547d2934cb
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Dec 24 12:09:40 2017 +0000

    working save/restore node positions with groups

commit bf7e231ffe
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Dec 24 10:09:26 2017 +0000

    node colors!

commit e23594b041
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Dec 23 23:42:17 2017 +0000

    wooo device group selection is working

commit b1b22c9811
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Dec 23 22:31:23 2017 +0000

    send full query str from netmap for json

commit 9a13bb7f9b
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Dec 23 22:06:24 2017 +0000

    implement host_group_displaynames and fix vlan save bug

commit 35ed34ed2f
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Dec 23 17:52:14 2017 +0000

    when neighbors only can restrict the links search

commit f9e5ec02ab
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Dec 23 17:44:41 2017 +0000

    tweak mapshow

commit 9f9971ff4e
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Dec 23 00:47:17 2017 +0000

    better sidebar look and layout

commit 1def41da67
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Thu Dec 21 23:54:44 2017 +0000

    sidebar with many options!!

commit 4203cbb225
Merge: 7c12df6b 7db65932
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Thu Dec 21 20:14:55 2017 +0000

    Merge branch 'master' into og-newnetmap

commit 7c12df6b83
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Thu Dec 21 00:13:03 2017 +0000

    refresh tooltips when the datatables table is updated

commit 081b89d5dc
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Dec 20 23:59:15 2017 +0000

    shortcut from device port to manual topology

commit 9520438960
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Dec 20 23:34:44 2017 +0000

    tighten the sql for neighbor links

commit d6b0275931
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Dec 20 23:26:22 2017 +0000

    tweak layout again

commit 9e5e2f0ed5
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Dec 20 23:18:57 2017 +0000

    fullscreen mode for netmap

commit aba359d2a9
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Dec 20 22:05:51 2017 +0000

    save and load netmap positions for same device groups

commit b7cd0251ff
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Dec 19 23:24:47 2017 +0000

    basic positions upload

commit 786115ebda
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Dec 19 22:13:16 2017 +0000

    working group drag

commit 91f98d4f33
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Dec 18 23:00:16 2017 +0000

    drag all selected nodes

commit ba48eee5f7
Merge: 60412cd6 7f8e3edf
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Dec 18 20:14:20 2017 +0000

    Merge branch 'master' into og-newnetmap

commit 60412cd6e6
Merge: 39ca1491 7dc79f72
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Dec 17 20:23:41 2017 +0000

    Merge branch 'master' into og-newnetmap

commit 39ca1491b0
Merge: 69622582 c8806bea
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Dec 17 19:52:49 2017 +0000

    Merge branch 'master' into og-newnetmap

commit 6962258223
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Dec 17 19:52:26 2017 +0000

    bleed versions

commit 444e82b8a3
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Dec 16 15:33:41 2017 +0000

    use d3 v4 only

commit 01cd7a908d
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Dec 16 15:22:08 2017 +0000

    two d3 connected

commit 854f37fc94
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Thu Dec 14 21:34:49 2017 +0000

    more fiddling with d3

commit 61a3a9e06d
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Thu Dec 14 19:15:21 2017 +0000

    working worker

commit ebadebe69a
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Thu Dec 14 07:57:55 2017 +0000

    borken worker

commit c327b28513
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Dec 13 13:45:03 2017 +0000

    fix node center

commit 33bd9d39da
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Dec 13 01:14:50 2017 +0000

    improvements to graphing

commit 6178bf2298
Merge: 1c627eaf 925940c9
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Dec 12 23:44:48 2017 +0000

    Merge branch 'master' into og-newnetmap

commit 1c627eaf3c
Merge: 77ca8f96 000cbaa1
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Dec 12 23:38:00 2017 +0000

    Merge branch 'master' into og-newnetmap

commit 77ca8f96e1
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Dec 11 00:09:08 2017 +0000

    working rendering using d3-force plugin

commit 187265fc03
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Dec 10 23:19:23 2017 +0000

    initial work on moving to d3-force-apex-plugin - json data
2017-12-31 09:13:01 +00:00
Oliver Gorwits
3ec84161ae New report for Port VLAN Mismatches (M. Bernstein) 2017-04-25 12:26:09 +01:00
Oliver Gorwits
d23b32500f relocate repo files so ND2 is the only code 2017-04-14 23:08:55 +01:00