135/netdisco
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

merge in og-get_external_credentials

Browse Source 
Squashed commit of the following:

commit 3fe8f383a7
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Mar 11 17:07:42 2019 +0000

    add debug lines and tested

commit 3249739e42
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Mar 11 16:54:11 2019 +0000

    change config name to get_credentials

commit e78558397a
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Mar 11 16:51:11 2019 +0000

    separate out generic device auth to DeviceAuth module

commit 249f05165f
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Mar 6 18:43:31 2019 +0000

    release 2.040007

commit e3af64df77
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Wed Mar 6 18:42:47 2019 +0000

    #521-redux fix wifi date search

commit 48857ae300
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Mar 4 12:03:31 2019 +0000

    release 2.040006

commit e09dab5362
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Mar 4 11:39:12 2019 +0000

    #527 update List::MoreUtils version requirement

commit 6e7de3fff3
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Mon Mar 4 09:59:41 2019 +0000

    release 2.040005

commit 0c98318a45
Author: Oliver Gorwits <oliver@spike.local>
Date:   Mon Mar 4 09:57:18 2019 +0000

    #526 fix discover syntax bug

commit e9efc45182
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Mar 3 14:56:48 2019 +0000

    release 2.040004

commit 6cdfd80d10
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Mar 3 14:34:00 2019 +0000

    allow undiscovered neighbors report to use discover_{waps,phones} setting

commit ac381e0802
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Mar 3 14:13:20 2019 +0000

    #506 was a red herring

commit b83e614c85
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Mar 3 13:00:36 2019 +0000

    make discover_{phones,waps} work with LLDP capabilities as well

commit 189d234b55
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Mar 3 12:47:38 2019 +0000

    check discover_no_type and friends earlier on in neighbors list build

commit 9c956466f3
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Mar 3 12:32:07 2019 +0000

    also update default config for new discover_phones and discover_waps settings

commit 09d29954d2
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Mar 3 12:26:50 2019 +0000

    #512 fix regression in phone/wap discovery exclusion

commit 2bae91f1b6
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Mar 3 12:01:34 2019 +0000

    rename match_devicetype() to match_to_setting()

commit 57cb6ddb70
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sun Mar 3 09:19:39 2019 +0000

    fix for over-eager fix to #506

commit ef560fb59a
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Mar 2 22:41:40 2019 +0000

    #506 relax device renumber so it works for an alias

commit 7a8bcb094e
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Mar 2 22:23:39 2019 +0000

    #521 Search Node Date Range not working

commit a643820a62
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Mar 2 21:54:27 2019 +0000

    #428 Port-Channels not showing in netmap

commit 5ba5bcd295
Merge: e7aacddb a1f95028
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Mar 2 20:04:11 2019 +0000

    Merge branch 'master' of github.com:netdisco/netdisco

commit e7aacddbc6
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Mar 2 20:01:05 2019 +0000

    #498 Map with VLAN filter omits unconnected devices

commit a1f95028ca
Author: nick n <39005454+inphobia@users.noreply.github.com>
Date:   Sat Mar 2 19:54:22 2019 +0100

    catch up with changes

    noticed that rc-sshcollector-core received updates to changes, add them here as well.

    didn't mention #499 & #522

commit ce1b847cea
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Mar 2 18:47:44 2019 +0000

    fix bug showing no nodes when only one matches in netmap

commit 78e30a7926
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Mar 2 16:28:15 2019 +0000

    #500 filtering in device/ports on native vlan duplicates entries

commit 9952f0c6c7
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Mar 2 15:02:12 2019 +0000

    #499 netdisco-do renumber reports wrong ip (inphobia)

commit ca3fd8f466
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Mar 2 15:00:18 2019 +0000

    #505 device renumber should update device port properties and device skips

commit 1265bc8470
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Mar 2 14:52:21 2019 +0000

    #520 catch slave ports defined without a master

commit d4c7579c10
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Mar 2 14:47:49 2019 +0000

    #522 TypeAhead.pm can reference empty data (inphobia)

commit 77decc23b7
Author: Oliver Gorwits <oliver@cpan.org>
Date:   Sat Mar 2 14:45:37 2019 +0000

    #514 inconsistent results in ip inventory (inphobia)

commit 3f211650b8
Author: nick n <39005454+inphobia@users.noreply.github.com>
Date:   Fri Mar 1 12:34:42 2019 +0100

    last pieces for db schema upgrade

    last piece of #510
This commit is contained in:
Oliver Gorwits
2019-03-11 20:34:07 +00:00
parent 0347ba7bbd
commit 98d7a71024
25 changed files with 353 additions and 257 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
Build.PL
View File

@@ -47,7 +47,7 @@ Module::Build->new(
'JSON' => '2.90',
'JSON::XS' => '3.01',
'List::Util' => '1.49',
'List::MoreUtils' => '0.33',
'List::MoreUtils' => '0.428',
'MIME::Base64' => '3.13',
'Module::Load' => '0.32',
'Moo' => '1.001000',

10
META.json
View File

@@ -60,7 +60,7 @@
"IO::Socket::SSL" : "2.048",
"JSON" : "2.90",
"JSON::XS" : "3.01",
"List::MoreUtils" : "0.33",
"List::MoreUtils" : "0.428",
"List::Util" : "1.49",
"MCE" : "1.703",
"MIME::Base64" : "3.13",
@@ -81,7 +81,7 @@
"Plack::Middleware::ReverseProxy" : "0.15",
"Pod::Usage" : "0",
"Role::Tiny" : "1.002005",
"SNMP::Info" : "3.64",
"SNMP::Info" : "3.65",
"SQL::Abstract" : "1.85",
"SQL::Translator" : "0.11024",
"Scope::Guard" : "0",
@@ -118,7 +118,7 @@
"provides" : {
"App::Netdisco" : {
"file" : "lib/App/Netdisco.pm",
"version" : "2.040003"
"version" : "2.040007"
},
"App::Netdisco::AnyEvent::Nbtstat" : {
"file" : "lib/App/Netdisco/AnyEvent/Nbtstat.pm"
@@ -140,7 +140,7 @@
},
"App::Netdisco::DB" : {
"file" : "lib/App/Netdisco/DB.pm",
"version" : "53"
"version" : "54"
},
"App::Netdisco::DB::ExplicitLocking" : {
"file" : "lib/App/Netdisco/DB/ExplicitLocking.pm"
@@ -800,6 +800,6 @@
"x_IRC" : "irc://irc.freenode.org/#netdisco",
"x_MailingList" : "https://lists.sourceforge.net/lists/listinfo/netdisco-users"
},
"version" : "2.040003",
"version" : "2.040007",
"x_serialization_backend" : "JSON::PP version 2.97001"
}

10
META.yml
View File

@@ -22,7 +22,7 @@ name: App-Netdisco
provides:
App::Netdisco:
file: lib/App/Netdisco.pm
version: '2.040003'
version: '2.040007'
App::Netdisco::AnyEvent::Nbtstat:
file: lib/App/Netdisco/AnyEvent/Nbtstat.pm
App::Netdisco::Backend::Job:
@@ -37,7 +37,7 @@ provides:
file: lib/App/Netdisco/Configuration.pm
App::Netdisco::DB:
file: lib/App/Netdisco/DB.pm
version: '53'
version: '54'
App::Netdisco::DB::ExplicitLocking:
file: lib/App/Netdisco/DB/ExplicitLocking.pm
App::Netdisco::DB::Result::Admin:
@@ -497,7 +497,7 @@ requires:
IO::Socket::SSL: '2.048'
JSON: '2.90'
JSON::XS: '3.01'
List::MoreUtils: '0.33'
List::MoreUtils: '0.428'
List::Util: '1.49'
MCE: '1.703'
MIME::Base64: '3.13'
@@ -518,7 +518,7 @@ requires:
Plack::Middleware::ReverseProxy: '0.15'
Pod::Usage: '0'
Role::Tiny: '1.002005'
SNMP::Info: '3.64'
SNMP::Info: '3.65'
SQL::Abstract: '1.85'
SQL::Translator: '0.11024'
Scope::Guard: '0'
@@ -549,5 +549,5 @@ resources:
homepage: http://netdisco.org/
license: http://opensource.org/licenses/bsd-license.php
repository: https://github.com/netdisco/netdisco
version: '2.040003'
version: '2.040007'
x_serialization_backend: 'CPAN::Meta::YAML version 0.018'

2
lib/App/Netdisco.pm
View File

@@ -4,7 +4,7 @@ use strict;
use warnings;
use 5.010_000;
our $VERSION = '2.040003';
our $VERSION = '2.040007';
use App::Netdisco::Configuration;
=head1 NAME

5
lib/App/Netdisco/Configuration.pm
View File

@@ -1,7 +1,7 @@
package App::Netdisco::Configuration;
use App::Netdisco::Environment;
use App::Netdisco::Util::SNMP ();
use App::Netdisco::Util::DeviceAuth ();
use Dancer ':script';
use Path::Class 'dir';
@@ -84,7 +84,8 @@ if ((setting('snmp_auth') and 0 == scalar @{ setting('snmp_auth') })
config->{'community_rw'} = [ @{setting('community_rw')}, 'private' ];
}
# fix up device_auth (or create it from old snmp_auth and community settings)
config->{'device_auth'} = [ App::Netdisco::Util::SNMP::fixup_device_auth() ];
config->{'device_auth'}
= [ App::Netdisco::Util::DeviceAuth::fixup_device_auth() ];
# defaults for workers
setting('workers')->{queue} ||= 'PostgreSQL';

11
lib/App/Netdisco/DB/Result/Device.pm
View File

@@ -262,20 +262,27 @@ sub renumber {
foreach my $set (qw/
DeviceIp
DeviceModule
DevicePower
DeviceVlan
DevicePort
DevicePortLog
DevicePortPower
DevicePortProperties
DevicePortSsid
DevicePortVlan
DevicePortWireless
DevicePower
DeviceVlan
/) {
$schema->resultset($set)
->search({ip => $old_ip})
->update({ip => $new_ip});
}
$schema->resultset('DeviceSkip')
->search({device => $new_ip})->delete;
$schema->resultset('DeviceSkip')
->search({device => $old_ip})
->update({device => $new_ip});
$schema->resultset('DevicePort')
->search({remote_ip => $old_ip})
->update({remote_ip => $new_ip});

20
lib/App/Netdisco/DB/Result/Virtual/DeviceLinks.pm
View File

@@ -22,8 +22,8 @@ __PACKAGE__->result_source_instance->view_definition(<<ENDSQL
array_agg(dp.port) AS left_port,
array_agg(dp.name) AS left_descr,
count(dpp.*) AS aggports,
sum(COALESCE(dpp.raw_speed, 0)) AS aggspeed,
count(*) AS aggports,
di.ip AS right_ip,
rd.dns AS right_dns,
@@ -32,8 +32,14 @@ __PACKAGE__->result_source_instance->view_definition(<<ENDSQL
array_agg(dp2.name) AS right_descr
FROM device_port dp
LEFT OUTER JOIN device_port_properties dpp USING (ip,
port)
LEFT OUTER JOIN device_port_properties dpp ON (
(dp.ip = dpp.ip) AND (dp.port = dpp.port)
AND (dp.type IS NULL
OR dp.type !~* '^(53|ieee8023adLag|propVirtual|l2vlan|l3ipvlan|135|136|137)\$')
AND (dp.is_master = 'false'
OR dp.slave_of IS NOT NULL) )
INNER JOIN device ld ON dp.ip = ld.ip
INNER JOIN device_ip di ON dp.remote_ip = di.alias
INNER JOIN device rd ON di.ip = rd.ip
@@ -45,12 +51,7 @@ __PACKAGE__->result_source_instance->view_definition(<<ENDSQL
WHERE dp.remote_port IS NOT NULL
AND dp.port !~* 'vlan'
AND (dp.descr IS NULL
OR dp.descr !~* 'vlan')
AND (dp.type IS NULL
OR dp.type !~* '^(53|ieee8023adLag|propVirtual|l2vlan|l3ipvlan|135|136|137)\$')
AND (dp.is_master = 'false'
OR dp.slave_of IS NOT NULL)
AND (dp.descr IS NULL OR dp.descr !~* 'vlan')
GROUP BY left_ip,
left_dns,
@@ -58,6 +59,7 @@ __PACKAGE__->result_source_instance->view_definition(<<ENDSQL
right_ip,
right_dns,
right_name )
SELECT *
FROM BothWays b
WHERE NOT EXISTS

6
lib/App/Netdisco/DB/Result/Virtual/UndiscoveredNeighbors.pm
View File

@@ -15,6 +15,7 @@ __PACKAGE__->result_source_instance->view_definition(<<'ENDSQL');
d.ip, d.name, d.dns,
p.port, p.name AS port_description,
p.remote_ip, p.remote_id, p.remote_type, p.remote_port,
dpp.remote_is_wap, dpp.remote_is_phone,
l.log AS comment,
a.log, a.finished
@@ -23,6 +24,7 @@ __PACKAGE__->result_source_instance->view_definition(<<'ENDSQL');
INNER JOIN device d USING (ip)
LEFT OUTER JOIN device_skip ds
ON ('discover' = ANY(ds.actionset) AND p.remote_ip = ds.device)
LEFT OUTER JOIN device_port_properties dpp USING (ip, port)
LEFT OUTER JOIN device_port_log l USING (ip, port)
LEFT OUTER JOIN admin a
ON (p.remote_ip = a.device AND a.action = 'discover')
@@ -58,6 +60,10 @@ __PACKAGE__->add_columns(
{ data_type => "text", is_nullable => 1 },
"remote_id",
{ data_type => "text", is_nullable => 1 },
"remote_is_wap",
{ data_type => "boolean", is_nullable => 1 },
"remote_is_phone",
{ data_type => "boolean", is_nullable => 1 },
"comment",
{ data_type => "text", is_nullable => 1 },
"log",

14
lib/App/Netdisco/DB/ResultSet/Device.pm
View File

@@ -47,16 +47,16 @@ sub with_times {
->search({},
{
'+columns' => {
uptime_age => \("replace(age(timestamp 'epoch' + uptime / 100 * interval '1 second', "
uptime_age => \("replace(age(timestamp 'epoch' + me.uptime / 100 * interval '1 second', "
."timestamp '1970-01-01 00:00:00-00')::text, 'mon', 'month')"),
first_seen_stamp => \"to_char(me.creation, 'YYYY-MM-DD HH24:MI')",
last_discover_stamp => \"to_char(last_discover, 'YYYY-MM-DD HH24:MI')",
last_macsuck_stamp => \"to_char(last_macsuck, 'YYYY-MM-DD HH24:MI')",
last_arpnip_stamp => \"to_char(last_arpnip, 'YYYY-MM-DD HH24:MI')",
last_discover_stamp => \"to_char(me.last_discover, 'YYYY-MM-DD HH24:MI')",
last_macsuck_stamp => \"to_char(me.last_macsuck, 'YYYY-MM-DD HH24:MI')",
last_arpnip_stamp => \"to_char(me.last_arpnip, 'YYYY-MM-DD HH24:MI')",
since_first_seen => \"extract(epoch from (age(now(), me.creation)))",
since_last_discover => \"extract(epoch from (age(now(), last_discover)))",
since_last_macsuck => \"extract(epoch from (age(now(), last_macsuck)))",
since_last_arpnip => \"extract(epoch from (age(now(), last_arpnip)))",
since_last_discover => \"extract(epoch from (age(now(), me.last_discover)))",
since_last_macsuck => \"extract(epoch from (age(now(), me.last_macsuck)))",
since_last_arpnip => \"extract(epoch from (age(now(), me.last_arpnip)))",
},
});
}

34
lib/App/Netdisco/Util/Device.pm
View File

@@ -10,7 +10,7 @@ our @EXPORT_OK = qw/
get_device
delete_device
renumber_device
match_devicetype
match_to_setting
is_discoverable is_discoverable_now
is_arpnipable is_arpnipable_now
is_macsuckable is_macsuckable_now
@@ -120,7 +120,7 @@ sub renumber_device {
schema('netdisco')->resultset('UserLog')->create({
username => session('logged_in_user'),
userip => scalar eval {request->remote_address},
event => (sprintf "Renumber device %s to %s", $device->ip, $new_ip),
event => (sprintf "Renumber device %s to %s", $ip, $new_ip),
});
$happy = 1;
@@ -129,7 +129,7 @@ sub renumber_device {
return $happy;
}
=head2 match_devicetype( $type, $setting_name )
=head2 match_to_setting( $type, $setting_name )
Given a C<$type> (which may be any text value), returns true if any of the
list of regular expressions in C<$setting_name> is matched, otherwise returns
@@ -137,7 +137,7 @@ false.
=cut
sub match_devicetype {
sub match_to_setting {
my ($type, $setting_name) = @_;
return 0 unless $type and $setting_name;
return (scalar grep {$type =~ m/$_/}
@@ -146,7 +146,7 @@ sub match_devicetype {
sub _bail_msg { debug $_[0]; return 0; }
=head2 is_discoverable( $ip, $device_type? )
=head2 is_discoverable( $ip, [$device_type, \@device_capabilities]? )
Given an IP address, returns C<true> if Netdisco on this host is permitted by
the local configuration to discover the device.
@@ -154,20 +154,32 @@ the local configuration to discover the device.
The configuration items C<discover_no> and C<discover_only> are checked
against the given IP.
If C<$device_type> is also given, then C<discover_no_type> will also be
checked.
If C<$device_type> is also given, then C<discover_no_type> will be checked.
Also respects C<discover_phones> and C<discover_waps> if either are set to
false.
Returns false if the host is not permitted to discover the target device.
=cut
sub is_discoverable {
my ($ip, $remote_type) = @_;
my ($ip, $remote_type, $remote_cap) = @_;
my $device = get_device($ip) or return 0;
$remote_type ||= '';
$remote_cap ||= [];
if (match_devicetype($remote_type, 'discover_no_type')) {
return _bail_msg("is_discoverable: $device matched discover_no_type");
}
return _bail_msg("is_discoverable: $device matches wap_platforms but discover_waps is not enabled")
if ((not setting('discover_waps')) and
(match_to_setting($remote_type, 'wap_platforms') or
scalar grep {match_to_setting($_, 'wap_capabilities')} @$remote_cap));
return _bail_msg("is_discoverable: $device matches phone_platforms but discover_phones is not enabled")
if ((not setting('discover_phones')) and
(match_to_setting($remote_type, 'phone_platforms') or
scalar grep {match_to_setting($_, 'phone_capabilities')} @$remote_cap));
return _bail_msg("is_discoverable: $device matched discover_no_type")
if (match_to_setting($remote_type, 'discover_no_type'));
return _bail_msg("is_discoverable: $device matched discover_no")
if check_acl_no($device, 'discover_no');

163
lib/App/Netdisco/Util/DeviceAuth.pm Normal file
View File

@@ -0,0 +1,163 @@
package App::Netdisco::Util::DeviceAuth;
use Dancer qw/:syntax :script/;
use App::Netdisco::Util::DNS 'hostname_from_ip';
use Try::Tiny;
use base 'Exporter';
our @EXPORT = ();
our @EXPORT_OK = qw/
fixup_device_auth get_external_credentials
/;
our %EXPORT_TAGS = (all => \@EXPORT_OK);
=head1 NAME
App::Netdisco::Util::DeviceAuth
=head1 DESCRIPTION
Helper functions for device authentication.
There are no default exports, however the C<:all> tag will export all
subroutines.
=head1 EXPORT_OK
=head2 fixup_device_auth
Rebuilds the C<device_auth> config with missing defaults and other fixups for
config changes over time. Returns a list which can replace C<device_auth>.
=cut
sub fixup_device_auth {
my $config = (setting('snmp_auth') || setting('device_auth'));
my @new_stanzas = ();
# new style snmp config
foreach my $stanza (@$config) {
# user tagged
my $tag = '';
if (1 == scalar keys %$stanza) {
$tag = (keys %$stanza)[0];
$stanza = $stanza->{$tag};
# corner case: untagged lone community
if ($tag eq 'community') {
$tag = $stanza;
$stanza = {community => $tag};
}
}
# defaults
$stanza->{tag} ||= $tag;
$stanza->{read} = 1 if !exists $stanza->{read};
$stanza->{no} ||= [];
$stanza->{only} ||= ['any'];
die "error: config: snmpv2 community in device_auth must be single item, not list\n"
if ref $stanza->{community};
die "error: config: stanza in device_auth must have a tag\n"
if not $stanza->{tag} and exists $stanza->{user};
push @new_stanzas, $stanza
}
# legacy config
# note: read strings tried before write
# note: read-write is no longer used for read operations
push @new_stanzas, map {{
read => 1, write => 0,
no => [], only => ['any'],
community => $_,
}} @{setting('community') || []};
push @new_stanzas, map {{
write => 1, read => 0,
no => [], only => ['any'],
community => $_,
}} @{setting('community_rw') || []};
foreach my $stanza (@new_stanzas) {
$stanza->{driver} ||= 'snmp'
if exists $stanza->{community}
or exists $stanza->{user};
}
return @new_stanzas;
}
=head2 get_external_credentials( $device, $mode )
Runs a command to gather SNMP credentials or a C<device_auth> stanza.
Mode can be C<read> or C<write> and defaults to 'read'.
=cut
sub get_external_credentials {
my ($device, $mode) = @_;
my $cmd = (setting('get_credentials') || setting('get_community'));
my $ip = $device->ip;
my $host = ($device->dns || hostname_from_ip($ip) || $ip);
$mode ||= 'read';
if (defined $cmd and length $cmd) {
# replace variables
$cmd =~ s/\%MODE\%/$mode/egi;
$cmd =~ s/\%HOST\%/$host/egi;
$cmd =~ s/\%IP\%/$ip/egi;
my $result = `$cmd`; # BACKTICKS
return () unless defined $result and length $result;
my @lines = split (m/\n/, $result);
foreach my $line (@lines) {
if ($line =~ m/^community\s*=\s*(.*)\s*$/i) {
if (length $1 and $mode eq 'read') {
debug sprintf '[%s] external read credentials added',
$device->ip;
return map {{
read => 1,
only => [$device->ip],
community => $_,
}} split(m/\s*,\s*/,$1);
}
}
elsif ($line =~ m/^setCommunity\s*=\s*(.*)\s*$/i) {
if (length $1 and $mode eq 'write') {
debug sprintf '[%s] external write credentials added',
$device->ip;
return map {{
write => 1,
only => [$device->ip],
community => $_,
}} split(m/\s*,\s*/,$1);
}
}
else {
my $stanza = undef;
try {
$stanza = from_json( $line );
debug sprintf '[%s] external credentials stanza added',
$device->ip;
}
catch {
info sprintf '[%s] error! failed to parse external credentials stanza',
$device->ip;
};
return $stanza if ref $stanza;
}
}
}
return ();
}
true;

116
lib/App/Netdisco/Util/SNMP.pm
View File

@@ -1,14 +1,11 @@
package App::Netdisco::Util::SNMP;
use Dancer qw/:syntax :script/;
use App::Netdisco::Util::DNS 'hostname_from_ip';
use App::Netdisco::Util::Permission ':all';
use App::Netdisco::Util::DeviceAuth 'get_external_credentials';
use base 'Exporter';
our @EXPORT = ();
our @EXPORT_OK = qw/
fixup_device_auth get_communities snmp_comm_reindex
/;
our @EXPORT_OK = qw/ get_communities snmp_comm_reindex /;
our %EXPORT_TAGS = (all => \@EXPORT_OK);
=head1 NAME
@@ -24,72 +21,6 @@ subroutines.
=head1 EXPORT_OK
=head2 fixup_device_auth
Rebuilds the C<device_auth> config with missing defaults and other fixups for
config changes over time. Returns a list which can replace C<device_auth>.
=cut
sub fixup_device_auth {
my $config = (setting('snmp_auth') || setting('device_auth'));
my @new_stanzas = ();
# new style snmp config
foreach my $stanza (@$config) {
# user tagged
my $tag = '';
if (1 == scalar keys %$stanza) {
$tag = (keys %$stanza)[0];
$stanza = $stanza->{$tag};
# corner case: untagged lone community
if ($tag eq 'community') {
$tag = $stanza;
$stanza = {community => $tag};
}
}
# defaults
$stanza->{tag} ||= $tag;
$stanza->{read} = 1 if !exists $stanza->{read};
$stanza->{no} ||= [];
$stanza->{only} ||= ['any'];
die "error: config: snmpv2 community in device_auth must be single item, not list\n"
if ref $stanza->{community};
die "error: config: stanza in device_auth must have a tag\n"
if not $stanza->{tag} and exists $stanza->{user};
push @new_stanzas, $stanza
}
# legacy config
# note: read strings tried before write
# note: read-write is no longer used for read operations
push @new_stanzas, map {{
read => 1, write => 0,
no => [], only => ['any'],
community => $_,
}} @{setting('community') || []};
push @new_stanzas, map {{
write => 1, read => 0,
no => [], only => ['any'],
community => $_,
}} @{setting('community_rw') || []};
foreach my $stanza (@new_stanzas) {
$stanza->{driver} ||= 'snmp'
if exists $stanza->{community}
or exists $stanza->{user};
}
return @new_stanzas;
}
=head2 get_communities( $device, $mode )
Takes the current C<device_auth> setting and pushes onto the front of the list
@@ -106,8 +37,7 @@ sub get_communities {
my @communities = ();
# first of all, use external command if configured
push @communities, _get_external_community($device, $mode)
if setting('get_community') and length setting('get_community');
push @communities, get_external_credentials($device, $mode);
# last known-good by tag
my $tag_name = 'snmp_auth_tag_'. $mode;
@@ -145,46 +75,6 @@ sub get_communities {
return ( @communities, @$config );
}
sub _get_external_community {
my ($device, $mode) = @_;
my $cmd = setting('get_community');
my $ip = $device->ip;
my $host = ($device->dns || hostname_from_ip($ip) || $ip);
if (defined $cmd and length $cmd) {
# replace variables
$cmd =~ s/\%HOST\%/$host/egi;
$cmd =~ s/\%IP\%/$ip/egi;
my $result = `$cmd`; # BACKTICKS
return () unless defined $result and length $result;
my @lines = split (m/\n/, $result);
foreach my $line (@lines) {
if ($line =~ m/^community\s*=\s*(.*)\s*$/i) {
if (length $1 and $mode eq 'read') {
return map {{
read => 1,
only => [$device->ip],
community => $_,
}} split(m/\s*,\s*/,$1);
}
}
elsif ($line =~ m/^setCommunity\s*=\s*(.*)\s*$/i) {
if (length $1 and $mode eq 'write') {
return map {{
write => 1,
only => [$device->ip],
community => $_,
}} split(m/\s*,\s*/,$1);
}
}
}
}
return ();
}
=head2 snmp_comm_reindex( $snmp, $device, $vlan )
Takes an established L<SNMP::Info> instance and makes a fresh connection using

4
lib/App/Netdisco/Web/Device.pm
View File

@@ -7,7 +7,7 @@ use Dancer::Plugin::Auth::Extensible;
use URI ();
use URL::Encode 'url_params_mixed';
use App::Netdisco::Util::Device 'match_devicetype';
use App::Netdisco::Util::Device 'match_to_setting';
# build view settings for port connected nodes and devices
set('connected_properties' => [
@@ -20,7 +20,7 @@ hook 'before_template' => sub {
my $tokens = shift;
# allow checking of discoverability of remote connected device
$tokens->{has_snmp} = sub { not match_devicetype(shift, 'discover_no_type') };
$tokens->{has_snmp} = sub { not match_to_setting(shift, 'discover_no_type') };
my $defaults = var('sidebar_defaults')->{'device_ports'}
or return;

2
lib/App/Netdisco/Web/Plugin/AdminTask/UndiscoveredNeighbors.pm
View File

@@ -27,6 +27,8 @@ get '/ajax/content/admin/undiscoveredneighbors' => require_role admin => sub {
# create a new row object to avoid hitting the DB in get_device()
my $dev = schema('netdisco')->resultset('Device')->new({ip => $r->{remote_ip}});
next unless is_discoverable( $dev, $r->{remote_type} );
next if (not setting('discover_waps')) and $r->{remote_is_wap};
next if (not setting('discover_phones')) and $r->{remote_is_phone};
push @discoverable_results, $r;
}
return unless scalar @discoverable_results;

45
lib/App/Netdisco/Web/Plugin/Device/Neighbors.pm
View File

@@ -125,18 +125,13 @@ sub make_link_infostring {
(my $left_name = lc($link->{left_dns} || $link->{left_name} || $link->{left_ip})) =~ s/$domain$//;
(my $right_name = lc($link->{right_dns} || $link->{right_name} || $link->{right_ip})) =~ s/$domain$//;
if ($link->{aggports} == 1) {
return sprintf '<b>%s:%s</b> (%s)<br><b>%s:%s</b> (%s)',
$left_name, $link->{left_port}->[0],
($link->{left_descr}->[0] || 'no description'),
$right_name, $link->{right_port}->[0],
($link->{right_descr}->[0] || 'no description');
}
else {
return sprintf '<b>%s:(%s)</b><br><b>%s:(%s)</b>',
$left_name, join(',', @{$link->{left_port}}),
$right_name, join(',', @{$link->{right_port}});
}
my @zipped = List::MoreUtils::zip6
@{$link->{left_port}}, @{$link->{left_descr}},
@{$link->{right_port}}, @{$link->{right_descr}};
return join '<br><br>', map { sprintf '<b>%s:%s</b> (%s)<br><b>%s:%s</b> (%s)',
$left_name, $_->[0], ($_->[1] || 'no description'),
$right_name, $_->[2], ($_->[3] || 'no description') } @zipped;
}
ajax '/ajax/data/device/netmap' => require_login sub {
@@ -179,17 +174,6 @@ ajax '/ajax/data/device/netmap' => require_login sub {
]) : ())
}, { result_class => 'DBIx::Class::ResultClass::HashRefInflator' });
if ($vlan) {
$links = $links->search({
-or => [
{ 'left_vlans.vlan' => $vlan },
{ 'right_vlans.vlan' => $vlan },
],
}, {
join => [qw/left_vlans right_vlans/],
});
}
while (my $link = $links->next) {
push @{$data{'links'}}, {
FROMID => $link->{left_ip},
@@ -217,10 +201,19 @@ ajax '/ajax/data/device/netmap' => require_login sub {
join => 'throughput',
})->with_times;
# filter by vlan for all or neighbors only
if ($vlan) {
$devices = $devices->search(
{ 'vlans.vlan' => $vlan },
{ join => 'vlans' }
);
}
DEVICE: while (my $device = $devices->next) {
# if in neighbors or vlan mode then use %ok_dev to filter
next DEVICE if (($mapshow eq 'neighbors') or $vlan)
and (not $ok_dev{$device->ip});
# if in neighbors mode then use %ok_dev to filter
next DEVICE if ($device->ip ne $qdev->ip)
and ($mapshow eq 'neighbors')
and (not $ok_dev{$device->ip}); # showing only neighbors but no link
# if location picked then filter
next DEVICE if ((scalar @lgrplist) and ((!defined $device->location)

57
lib/App/Netdisco/Web/Plugin/Device/Ports.pm
View File

@@ -25,25 +25,6 @@ get '/ajax/content/device/ports' => require_login sub {
if ($f) {
if (($prefer eq 'vlan') or (not $prefer and $f =~ m/^\d+$/)) {
return unless $f =~ m/^\d+$/;
if (param('invert')) {
$set = $set->search({
'me.vlan' => { '!=' => $f },
'port_vlans.vlan' => [
'-or' => { '!=' => $f }, { '=' => undef }
],
}, { join => 'port_vlans' });
}
else {
$set = $set->search({
-or => {
'me.vlan' => $f,
'port_vlans.vlan' => $f,
},
}, { join => 'port_vlans' });
}
return unless $set->count;
}
else {
if (param('partial')) {
@@ -120,7 +101,7 @@ get '/ajax/content/device/ports' => require_login sub {
# now begin to join tables depending on the selected columns/options
# get vlans on the port
# leave this query dormant (lazy) unless c_vmember is set
# leave this query dormant (lazy) unless c_vmember is set or vlan filtering
my $vlans = $set->search({}, {
select => [
'port',
@@ -131,7 +112,7 @@ get '/ajax/content/device/ports' => require_login sub {
group_by => 'me.port',
});
if (param('c_vmember')) {
if (param('c_vmember') or ($prefer eq 'vlan') or (not $prefer and $f =~ m/^\d+$/)) {
$vlans = { map {(
$_->port => {
# DBIC smart enough to work out this should be an arrayref :)
@@ -194,13 +175,37 @@ get '/ajax/content/device/ports' => require_login sub {
# also get remote LLDP inventory if asked for
$set = $set->with_remote_inventory if param('n_inventory');
# sort ports (empty set would be a 'no records' msg)
my $results = [ sort { &App::Netdisco::Util::Web::sort_port($a->port, $b->port) } $set->all ];
return unless scalar @$results;
# run query
my @results = $set->all;
# filter for tagged vlan using existing agg query,
# which is better than join inflation
if (($prefer eq 'vlan') or (not $prefer and $f =~ m/^\d+$/)) {
if (param('invert')) {
@results = grep {
(!defined $_->vlan or $_->vlan ne $f)
and
(0 == scalar grep {defined and $_ ne $f} @{ $vlans->{$_->port}->{vlan_set} })
} @results;
}
else {
@results = grep {
($_->vlan eq $f)
or
(scalar grep {defined and $_ eq $f} @{ $vlans->{$_->port}->{vlan_set} })
} @results;
}
}
# sort ports
@results = sort { &App::Netdisco::Util::Web::sort_port($a->port, $b->port) } @results;
# empty set would be a 'no records' msg
return unless scalar @results;
if (request->is_ajax) {
template 'ajax/device/ports.tt', {
results => $results,
results => \@results,
nodes => $nodes_name,
ips => $ips_name,
device => $device,
@@ -210,7 +215,7 @@ get '/ajax/content/device/ports' => require_login sub {
else {
header( 'Content-Type' => 'text/comma-separated-values' );
template 'ajax/device/ports_csv.tt', {
results => $results,
results => \@results,
nodes => $nodes_name,
ips => $ips_name,
device => $device,

4
lib/App/Netdisco/Web/Plugin/Report/IpInventory.pm
View File

@@ -110,12 +110,12 @@ get '/ajax/content/report/ipinventory' => require_login sub {
'ip', 'mac', 'dns', 'time_last', 'time_first',
'active', 'node', 'age'
],
order_by => [{-asc => 'ip'}, {-desc => 'active'}],
order_by => [{-asc => 'ip'}, {-desc => 'active'}, {-asc => 'node'}],
}
)->as_query;
my $rs;
if ( $start && $end ) {
if ( $start and $end ) {
$start = $start . ' 00:00:00';
$end = $end . ' 23:59:59';

37
lib/App/Netdisco/Web/Plugin/Search/Node.pm
View File

@@ -30,32 +30,45 @@ ajax '/ajax/content/search/node' => require_login sub {
if ( $start and $end ) {
$start = $start . ' 00:00:00';
$end = $end . ' 23:59:59';
if ($agenot) {
@times = (-or => [
time_first => [ { '<', $start }, undef ],
time_last => { '>', $end },
time_first => [ undef ],
time_last => [ { '<', $start }, { '>', $end } ]
]);
@wifitimes = (-or => [
time_last => { '<', $start },
time_last => { '>', $end },
time_last => [ undef ],
time_last => [ { '<', $start }, { '>', $end } ],
]);
@porttimes = (-or => [
creation => { '<', $start },
creation => { '>', $end },
creation => [ undef ],
creation => [ { '<', $start }, { '>', $end } ]
]);
}
else {
@times = (-and => [
time_first => { '>=', $start },
time_last => { '<=', $end },
]);
@wifitimes = (-and => [
@times = (-or => [
-and => [
time_first => undef,
time_last => undef,
],
-and => [
time_last => { '>=', $start },
time_last => { '<=', $end },
],
]);
@porttimes = (-and => [
@wifitimes = (-or => [
time_last => undef,
-and => [
time_last => { '>=', $start },
time_last => { '<=', $end },
],
]);
@porttimes = (-or => [
creation => undef,
-and => [
creation => { '>=', $start },
creation => { '<=', $end },
],
]);
}
}

2
lib/App/Netdisco/Web/TypeAhead.pm
View File

@@ -49,7 +49,7 @@ ajax '/ajax/data/port/typeahead' => require_login sub {
if $port;
my $results = [
map {{ label => (sprintf "%s (%s)", $_->port, $_->name), value => $_->port }}
map {{ label => (sprintf "%s (%s)", $_->port, ($_->name || '')), value => $_->port }}
sort { &App::Netdisco::Util::Web::sort_port($a->port, $b->port) } $set->all
];

21
lib/App/Netdisco/Worker/Plugin/Discover/Neighbors.pm
View File

@@ -45,7 +45,7 @@ register_worker({ phase => 'main', driver => 'snmp' }, sub {
# only enqueue if device is not already discovered,
# discover_* config permits the discovery
foreach my $neighbor (@to_discover) {
my ($ip, $remote_type, $remote_id) = @$neighbor;
my ($ip, $remote_id) = @$neighbor;
if ($seen_ip{ $ip }++) {
debug sprintf
' queue - skip: IP %s is already queued from %s',
@@ -63,13 +63,6 @@ register_worker({ phase => 'main', driver => 'snmp' }, sub {
my $newdev = get_device($ip);
next if $newdev->in_storage;
if (not is_discoverable($newdev, $remote_type)) {
debug sprintf
' queue - skip: %s of type [%s] excluded by discover_* config',
$ip, ($remote_type || '');
next;
}
# risk of things going wrong...?
# https://quickview.cloudapps.cisco.com/quickview/bug/CSCur12254
@@ -127,6 +120,7 @@ sub store_neighbors {
my $c_port = $snmp->c_port;
my $c_id = $snmp->c_id;
my $c_platform = $snmp->c_platform;
my $c_cap = $snmp->c_cap;
# cache the device ports to save hitting the database for many single rows
vars->{'device_ports'} =
@@ -176,6 +170,7 @@ sub store_neighbors {
my $remote_port = undef;
my $remote_type = Encode::decode('UTF-8', $c_platform->{$entry} || '');
my $remote_id = Encode::decode('UTF-8', $c_id->{$entry});
my $remote_cap = $c_cap->{$entry} || [];
next unless $remote_ip;
my $r_netaddr = NetAddr::IP::Lite->new($remote_ip);
@@ -246,7 +241,15 @@ sub store_neighbors {
# what we came here to do.... discover the neighbor
debug sprintf ' [%s] neigh - %s with ID [%s] on %s',
$device->ip, $remote_ip, ($remote_id || ''), $port;
push @to_discover, [$remote_ip, $remote_type, $remote_id];
if (is_discoverable($remote_ip, $remote_type, $remote_cap)) {
push @to_discover, [$remote_ip, $remote_id];
}
else {
debug sprintf
' [%s] neigh - skip: %s of type [%s] excluded by discover_* config',
$device->ip, $remote_ip, ($remote_type || '');
}
$remote_port = $c_port->{$entry};
if (defined $remote_port) {

10
lib/App/Netdisco/Worker/Plugin/Discover/PortProperties.pm
View File

@@ -8,7 +8,7 @@ use App::Netdisco::Transport::SNMP ();
use Dancer::Plugin::DBIC 'schema';
use Encode;
use App::Netdisco::Util::Device 'match_devicetype';
use App::Netdisco::Util::Device 'match_to_setting';
register_worker({ phase => 'main', driver => 'snmp' }, sub {
my ($job, $workerconf) = @_;
@@ -86,12 +86,12 @@ register_worker({ phase => 'main', driver => 'snmp' }, sub {
my $remote_type = Encode::decode('UTF-8', $c_platform->{$idx} || '');
$properties{ $port }->{remote_is_wap} = 'true'
if scalar grep {match_devicetype($_, 'wap_capabilities')} @$remote_cap
or match_devicetype($remote_type, 'wap_platforms');
if scalar grep {match_to_setting($_, 'wap_capabilities')} @$remote_cap
or match_to_setting($remote_type, 'wap_platforms');
$properties{ $port }->{remote_is_phone} = 'true'
if scalar grep {match_devicetype($_, 'phone_capabilities')} @$remote_cap
or match_devicetype($remote_type, 'phone_platforms');
if scalar grep {match_to_setting($_, 'phone_capabilities')} @$remote_cap
or match_to_setting($remote_type, 'phone_platforms');
next unless scalar grep {defined && m/^inventory$/} @{ $rem_media_cap->{$idx} };

1
lib/App/Netdisco/Worker/Plugin/Discover/Properties.pm
View File

@@ -238,6 +238,7 @@ register_worker({ phase => 'early', driver => 'snmp' }, sub {
# must do this after building %interfaces so that we can set is_master
foreach my $sidx (keys %$agg_ports) {
my $slave = $interfaces->{$sidx} or next;
next unless defined $agg_ports->{$sidx}; # slave without a master?!
my $master = $interfaces->{ $agg_ports->{$sidx} } or next;
next unless exists $interfaces{$slave} and exists $interfaces{$master};

4
lib/App/Netdisco/Worker/Plugin/Macsuck/Nodes.pm
View File

@@ -7,7 +7,7 @@ use aliased 'App::Netdisco::Worker::Status';
use App::Netdisco::Transport::SNMP ();
use App::Netdisco::Util::Permission 'check_acl_no';
use App::Netdisco::Util::PortMAC 'get_port_macs';
use App::Netdisco::Util::Device 'match_devicetype';
use App::Netdisco::Util::Device 'match_to_setting';
use App::Netdisco::Util::Node 'check_mac';
use App::Netdisco::Util::SNMP 'snmp_comm_reindex';
use Dancer::Plugin::DBIC 'schema';
@@ -342,7 +342,7 @@ sub walk_fwtable {
# neighbors otherwise it would kill the DB with device lookups.
my $neigh_cannot_macsuck = eval { # can fail
check_acl_no(($device_port->neighbor || "0 but true"), 'macsuck_unsupported') ||
match_devicetype($device_port->remote_type, 'macsuck_unsupported_type') };
match_to_setting($device_port->remote_type, 'macsuck_unsupported_type') };
if ($device_port->is_uplink) {
if ($neigh_cannot_macsuck) {

8
share/config.yml
View File

@@ -216,7 +216,7 @@ device_identity: []
community: []
community_rw: []
device_auth: []
get_community: ""
get_credentials: ""
bulkwalk_off: false
bulkwalk_no: []
bulkwalk_repeaters: 20
@@ -229,9 +229,9 @@ devices_no: []
devices_only: []
discover_no: []
discover_only: []
discover_no_type:
- '(?i)phone'
- '(?i)(?:wap|wireless)'
discover_no_type: []
discover_waps: true
discover_phones: false
discover_min_age: 0
macsuck_no: []
macsuck_only: []

14
share/environments/deployment.yml
View File

@@ -39,20 +39,18 @@ device_auth:
# ¯`·.¸¸.·´¯`·.¸¸.·´¯`·.¸¸.·´¯`·.¸¸.·´¯`·.¸¸.·´¯`·.¸¸.·´¯`·.¸¸.·´¯`·.¸¸
#
# SOME MORE INTERESTING SETTINGS WHERE THE DEFAULTS ARE PROBABLY OKAY
#
# ¯`·.¸¸.·´¯`·.¸¸.·´¯`·.¸¸.·´¯`·.¸¸.·´¯`·.¸¸.·´¯`·.¸¸.·´¯`·.¸¸.·´¯`·.¸¸
# ---------------------------------------------------------------
# OTHER INTERESTING SETTINGS WHERE THE DEFAULTS ARE PROBABLY OKAY
# ---------------------------------------------------------------
# do not discover IP Phones or Wireless Access Points.
# discover Wireless Access Points, but not IP Phones
# usually these are visible as device neighbors but don't support
# SNMP, which just clogs up the job queue.
# ```````````````````````````````````````````````````````````````
#discover_no_type:
# - '(?i)phone'
# - '(?i)(?:wap|wireless)'
#discover_waps: true
#disover_phones: false
# this is the schedule for automatically keeping netdisco up-to-date;
# these are good defaults, so only uncomment if needing to change.