Commit Graph

545 Commits

Author SHA1 Message Date
Oliver Gorwits
4f57a4d30a rename ports column in device table to be num_ports 2020-04-18 16:50:49 +01:00
Oliver Gorwits
524f4792c3 restore port column needed for discovery 2020-04-18 09:47:05 +01:00
Oliver Gorwits
dff26abc5c API implementation (#712)
* initial v0 creator

* working json api for generic reports

* add require login

* move report swagger into plugin, and set new default layout of noop

* require proper role and also use new util func

* start to tidy authn

* some work on cleaning up web authn

* clean up the authN checks

* fix bug

* fix the auth for api

* fixes to json handling

* set swagger sort order

* enable most reports for api endpoints

* fix doc

* add paramters to reports

* add missed report

* allow api_parameters in reports config

* reorganise api

* add vlan search

* add port search

* make sure to enable layout processing

* add device search

* add v1 to api paths

* add Node Search

* support api_responses

* add device object search; fix spurious ports field in device result class

* handle some plugins just returning undef if search fails

* errors from api seamlessley

* fix error in date range default

* more sensible default for prefix

* change order of endpoints in swagger-ui

* all db row classes can now TO_JSON

* add device_port api endpoint

* add device ports endpoint

* do not expand docs

* add swagger ui json tree formatter

* add all relations from Device table

* add port relations

* add nodes retrieve on device or vlan

* rename to GetAPIKey

* update config for previous commit
2020-04-15 21:15:52 +01:00
Christian Ramseyer
a8a77a2df1 Change SSHCollector::NXOS to use expect (#707)
* $ssh->capture() aka. directly executing a command via
   ssh <device> <command> does not seem to work on NXOS7, maybe
   also others

 * changed this module to use Expect, successfully tested on
   NXOS6 - NXOS9
2020-03-19 13:01:34 +01:00
Oliver Gorwits
8b42fb3774 fix pod 2020-02-13 14:23:27 +00:00
Oliver Gorwits
912a971a75 tidy the Clavister module 2020-02-13 10:24:10 +00:00
Henningen
2d849e60ce Create Clavister.pm (#674) 2020-02-13 10:20:24 +00:00
Oliver Gorwits
ab6305b4cf default access list for device_auth should be ipv4 and ipv6 2020-02-12 16:55:16 +00:00
Christian Ramseyer
031c3e6d95 Optimize PortMAC query
* We meant well but it turns out that the array unnest and join is
  actually very slow, as the join arguments do not get pushed down
  into the CTE (in Postgres 9/10 at least, later versions remove some
  of the optimization barriers in that specifc type of query)
* This caused a seq scan on both device and device_port, and the query
  is executed many times during macsuck
* The query is now rewritten to use ANY (macaddr[]) and without CTE,
  which seems to be around 20x faster
2020-01-31 10:32:02 +01:00
Oliver Gorwits
b032ebc18e prevent Template::AutoFilter taking action on CSV output 2020-01-30 19:21:02 +00:00
Christian Ramseyer
16e451c77c Change IOSXR Expect close method
* we noticed some hanging SSH connections with soft_close (probably due to fds not being closed properly), so switched to hard_close instead
2020-01-28 17:43:06 +01:00
Christian Ramseyer
0938cd1651 New version of the IOSXR module using Expect
* The previous module only worked for 32-bit IOSXR, and already
  there some fiddling with the STDIN of the process was required
  when being run from netdisco-backend
* In 64-bit IOSXR, the STDIN workaround stopped working and
  created a stuck ssh process on every arpnip
* This new version uses Expect instead of plain SSH, so a proper
  pty is provided under any circumstances
* Successfully tested on recent 32- and 64-bit variants
2020-01-27 21:03:23 +01:00
Christian Ramseyer
f2f5d908bb Fix regression from PR #680
Hi @ollyg! Unfortunately I have found some issues with the code we
finally released in #680:

* get_port_macs expects an array ref but values() returns array,
  so the code was never called due to the return unless... check
* When fw_mac_list had exactly two entries, only the second value
  was bound as a scalar to the parameter. This is probably due
  to the shorthand bind formats described in
  https://metacpan.org/pod/DBIx::Class::ResultSet#DBIC-BIND-VALUES,
  but I'm not a 100% on this.
* return unless now checks for an entry in the list, with the old
  check the statement was also executed for empty lists

In cases where only the device(_port)?.mac lookup worked for uplink
detection, users of 02.044005 - 010 might get a lot of uplinks not
labeled as such.
2020-01-26 22:08:49 +01:00
Oliver Gorwits
db6e89607d #696 complete implementation of #688 2020-01-25 18:07:43 +00:00
antonc42
3d231ec135 new SSHCollector platform for Extreme VSP (#695)
This module works for Extreme (formerly Avaya) VSP switches running the
VOSS operating system. When using multiple VRFs, the default Arpnip does
not work.

The way to query the ARP tables of different VRFs is to append "::X" to
the end of the SNMP community string, where "X" is the VRF ID number.
Since Netdisco doesn't support querying with multiple community strings
at this time, this module uses SSH to query the ARP tables of all the
VRFs by default or select VRFs using the "vrfs" key in "device_auth"
(see documentation in the module for an example).
2020-01-23 16:06:06 +01:00
Oliver Gorwits
bb46797201 smarter fix for vlan filter in netmap 2020-01-22 21:23:57 +00:00
Oliver Gorwits
ee86e62ac7 fix bug in vlan filtered netmap, duplicate nodes 2020-01-22 21:19:04 +00:00
Oliver Gorwits
3044d7ebe6 fixes to make the PortMacs query work 2020-01-19 15:08:38 +00:00
Oliver Gorwits
e982aded09 add some protections to ACLs and an error log, following #686 2019-12-30 11:23:17 +00:00
Oliver Gorwits
e42c2135c4 fix #686 makerancidconf excluded setting should be (ACL) list not dict 2019-12-30 11:09:17 +00:00
Oliver Gorwits
241ecf3353 #524 ipinventory misses mac address for device ips 2019-12-22 23:31:40 +00:00
Oliver Gorwits
f7f2d2088b #81 avoid DEPRECATED single warning from DBIC 2019-12-22 09:07:12 +00:00
Oliver Gorwits
c282a5a565 #688 Amend vlan search to show number of ports actually using the vlan 2019-12-22 08:33:11 +00:00
Oliver Gorwits
d133eb97c2 #667 Network Map: Filter by VLAN doesnt work as expected 2019-12-22 00:16:04 +00:00
Oliver Gorwits
4268fff785 #681 Add href link to a device in a report 2019-12-21 23:08:02 +00:00
Oliver Gorwits
4030000b34 #687 Add an option to set main page after logging in 2019-12-21 22:24:19 +00:00
Oliver Gorwits
b04ffffc06 fix #679 unint error 2019-12-21 21:53:50 +00:00
Oliver Gorwits
9ab1b24a77 safety on snmp::info methods 2019-12-21 21:34:39 +00:00
Oliver Gorwits
0eab9213c0 should be faster than using IN on large lists 2019-12-21 21:32:27 +00:00
Oliver Gorwits
9a1938ddc9 safety and tidy the port_macs patch 2019-12-21 20:28:20 +00:00
Christian Ramseyer
26d3fbdd40 Reduce macsuck bandwidth usage to database (#680)
* Reduce macsuck bandwidth usage to database

get_port_macs transfers a full list of all device_port.(mac,ip) in every macsuck.
With 8k devices and 40k interfaces it takes up around 15MB. Transferring them 8k times
during an 1h macsuck cycle requires bandwidth in the 300 to 400 mbit/s range.

This patch changes get_port_macs to be called inside walk_fwtable and only transfer
the macs found in the current target device/vlan.
2019-12-21 18:28:47 +00:00
nick n
b211c7c7e2 remove vim settings from files (#676)
vim is great, but just use .vimrc instead of settings in the source files.

or if anyone cares that much, write up a sample one and add it to git.
2019-10-31 03:36:19 +01:00
Oliver Gorwits
08969583cb change double negative logic 2019-10-30 19:50:04 +00:00
Ryan Gasik
441ac59661 Add check for is_arpnipable and is_macsuckable in Util::Device
Checks to make sure the device is in storage when checking if L2/L3
is supported on the device. Fixes #669.
2019-10-30 09:10:02 -07:00
Oliver Gorwits
4e3bfee214 Add TACACS+ support per #660 (#662) 2019-10-15 18:23:51 +01:00
Oliver Gorwits
ecc9c6f209 sort the os_ver in the web interface (#663) 2019-10-15 18:23:34 +01:00
Oliver Gorwits
884a75a866 #659 speed up Macsuck::Nodes while using $snmp->cisco_comm_indexing (#661)
* #659 speed up Macsuck::Nodes while using $snmp->cisco_comm_indexing

* Do not read qb_fw_vlan when using cisco_comm_indexing
2019-10-14 22:08:38 +02:00
Oliver Gorwits
7f65d6afdc fix file endling 2019-09-23 14:00:28 +01:00
Christian Ramseyer
8ffdff3bbc Add speed_admin to device ports tab (#650) (#653)
* Add speed_admin to device ports tab (#650)

* move from device_port_properties to device_port table, and tidy up
2019-09-23 13:58:26 +01:00
Christian Ramseyer
26960bdd34 Fix potential crash in Discover/Properties::_get_ipv6_aliases
* On e.g. Cisco N9K, ipv6 objects can be returned undef which crashes the ip alias discovery with `Can't use an undefined value as a HASH reference at .../Discover/Properties.pm line 346.`
 * Discovery continues but leaves device_ip empty, which in turn makes device->neighbor non-functional
 * Fixed by adding various `$snmp->x || {}` safeguards
2019-09-20 13:52:34 +02:00
Christian Ramseyer
8edbd515e7 Store i_speed_admin (#650) 2019-09-19 14:11:04 +02:00
pyro3d
73e9f0bf0d Add DOCSIS/Cable Modem Discovery (#646)
* Add DOCSIS/Cable Modem Discovery
2019-09-18 16:19:18 -07:00
Oliver Gorwits
d8d2471c09 #644 Device DNS Mismatch report broken by domain_suffix change 2019-09-04 13:33:12 +01:00
Oliver Gorwits
409b7c1409 fix tests 2019-09-03 19:59:03 +01:00
Oliver Gorwits
542837d266 #591 domain_suffix can be list and supports (simple) regexp
Squashed commit of the following:

commit 975e4c6afc
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 3 13:35:26 2019 +0100

    also support regexp in domain_suffix

commit 0a2b5c8fa2
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 3 13:17:17 2019 +0100

    fix rancid, graph, and nodemonitor

commit 6d881de3ff
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 3 13:11:54 2019 +0100

    improve docs and set default domain_suffix to be list

commit 1dcafc08a8
Merge: 9a752e02 14ac69dc
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 3 13:00:39 2019 +0100

    Merge branch 'master' into og-multiple-domain-suffix

commit 9a752e0298
Merge: c836619f 82a99ea9
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 3 09:45:25 2019 +0100

    Merge branch 'master' into og-multiple-domain-suffix

commit c836619f8c
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Thu Jun 13 07:52:45 2019 +0100

    hokey fix for nodes with domains

commit fed14bd810
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Thu Jun 13 07:02:09 2019 +0100

    basic implementation, rancid graph and nodemonitor missing
2019-09-03 13:35:43 +01:00
Oliver Gorwits
14ac69dc2e might as well make other *walk not care about pseudo and use layers instead 2019-09-03 13:00:22 +01:00
Oliver Gorwits
82a99ea94e #642 requests to /admin/unknown-thing do not get 404 2019-09-03 09:36:50 +01:00
Oliver Gorwits
2897eda684 #587 #561 update pseudo devices to better support ssh arpnip
this patch resets all pseudo devices to have no layer3 support but adds a
feature to the pseudo devices admin panel to enable layer3 support. it also
changes arpnip and arpwalk behaviour to always permit the action if layer3
is available (ignoring the vendor).

documentation will need updating to tell users to create pseudo devices
with layer3 support when they want to arpnip an unsupported platform.

arpnip with ssh/cli against a supported platform (one that can be discovered)
will continue to work normally.

Squashed commit of the following:

commit 9dad5be81d
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 3 09:03:53 2019 +0100

    allow pseudo with layer 3 to run arpnip

commit 7d97943fcd
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 3 08:59:10 2019 +0100

    allow pseudo devices with layer 2/3 capability

commit d1fdf574e3
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 3 08:55:41 2019 +0100

    move pseudo and layer checks to is_able from is_able_now

commit e0f72ef67d
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 3 08:51:42 2019 +0100

    ports defaults to one

commit 86ba01270c
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 3 08:50:45 2019 +0100

    add tooltip for arpnip toggle

commit cdd2470228
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Tue Sep 3 08:34:46 2019 +0100

    simplify template

commit 46236d68ea
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Sep 1 23:53:56 2019 +0100

    a fix up for pseudo devices which need layer 3

commit 016d249efc
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Sep 1 20:37:11 2019 +0100

    do not wrap buttons

commit 1ec1402e0c
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Sep 1 20:33:03 2019 +0100

    implement user settable layer-three service for pseudo devices

commit a267efa3d8
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Sep 1 18:39:22 2019 +0100

    only set layer if successful action

commit b108be5e23
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Sep 1 18:32:19 2019 +0100

    should defer SNMP against pseudo devices

commit 897ba3a629
Merge: e0ddbaab a7348900
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Sep 1 14:54:36 2019 +0100

    Merge branch 'master' into og-pseudo-vs-cli-arpnip

commit e0ddbaab08
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Aug 26 11:35:13 2019 +0100

    as last commit, for discover

commit 61f9c89040
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Aug 25 23:55:38 2019 +0100

    move pseudo and layer checks into is_*able functions

commit 8b010d4023
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Aug 25 18:38:11 2019 +0100

    any device completing macsuck/arpnip must have that layer

commit a11bce7863
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Aug 25 18:33:27 2019 +0100

    clean up device layers

commit d2661bff61
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Aug 25 18:18:02 2019 +0100

    first make arpnip behave like other jobs towards pseudo devices
2019-09-03 09:09:55 +01:00
Oliver Gorwits
86daf7174d add comments 2019-08-23 09:25:17 +01:00
Oliver Gorwits
8a66cf91af typo fix in free ports 2019-08-23 09:22:21 +01:00