From 03f41f1177902eb8c432c2318db49def6a2700e7 Mon Sep 17 00:00:00 2001 From: Oliver Gorwits Date: Sat, 6 May 2017 15:00:17 +0100 Subject: [PATCH] Enhance the ACL options to include AND and negation Squashed commit of the following: commit 7673f3ee1e5048d59b4942fc39b278849e31499a Author: Oliver Gorwits Date: Sat May 6 14:19:19 2017 +0100 allow check_acl to accept Device or NetAddr::IP instance commit c31059bc01e4e2b4dcfccd67ac6b5b88fed3af94 Author: Oliver Gorwits Date: Sat May 6 14:19:00 2017 +0100 update docs commit deaeab2670b430fe7a170cacc1b9ad93a5849fa6 Author: Oliver Gorwits Date: Sat May 6 14:18:27 2017 +0100 SNMP only stanza has access to full check_acl features commit 4a44fa5863d8a56a96d00656b95a6c28dc474de1 Author: Oliver Gorwits Date: Mon May 1 18:49:38 2017 +0100 add AND operator and negation support to ACLs --- lib/App/Netdisco/Manual/Configuration.pod | 139 ++++++++++++---------- lib/App/Netdisco/Util/DNS.pm | 2 +- lib/App/Netdisco/Util/Node.pm | 50 +------- lib/App/Netdisco/Util/Permission.pm | 112 +++++++++++------ lib/App/Netdisco/Util/SNMP.pm | 2 +- t/20-checkacl.t | 94 +++++++++++++++ 6 files changed, 252 insertions(+), 147 deletions(-) create mode 100644 t/20-checkacl.t diff --git a/lib/App/Netdisco/Manual/Configuration.pod b/lib/App/Netdisco/Manual/Configuration.pod index 77b5d69b..ad8cef3b 100644 --- a/lib/App/Netdisco/Manual/Configuration.pod +++ b/lib/App/Netdisco/Manual/Configuration.pod @@ -7,11 +7,7 @@ App::Netdisco::Manual::Configuration - How to Configure Netdisco The configuration files for Netdisco come with all options set to sensible default values, and just a few that you must initially set yourself. -However as you use the system over time, there are many situations where you -might want to tune the behaviour of Netdisco, and for that we have a lot of -configuration settings available. - -=head2 GUIDANCE +=head2 YAML GUIDANCE There are two configuration files: C (which lives inside Netdisco) and C (which usually lives in C<${HOME}/environments>). @@ -22,7 +18,7 @@ file. The two are merged when Netdisco starts, with your settings in C overriding the defaults from C. The configuration file format for Netdisco is YAML. This is easy for humans to -edit, but you should take care over whitespace and avoid TAB characters. YAML +edit, but you should take care with whitespace and avoid TAB characters. YAML supports several data types: =over 4 @@ -30,24 +26,80 @@ supports several data types: =item * Boolean - True/False value, using C<1> and C<0> or C and C -respectively +respectively, e.g.: + + check_userlog: true =item * -List - Set of things using C<[a, b, c]> on one line or C<-> on separate lines +List - Set of things using C<[a, b, c]> on one line or C<-> on separate lines, +e.g.: + + community: ['public', 'another'] + + discover_no: + - '192.0.2.0/24' + - '2001:db8::/32' =item * -Dictionary - Key/Value pairs (like Perl Hash) using C<{key1: val1, key2: -val2}> on one line or C on separate lines +Dictionary - Key/Value pairs (like Perl Hash) using C<< {key1: val1, key2: +val2} >> on one line or C on separate lines, e.g.: + + port_control_reasons: + address: 'Address Allocation Abuse' + copyright: 'Copyright Violation' =item * String - Quoted, just like in Perl (and essential if the item contains the -colon character) +colon character). =back +=head2 ACCESS CONTROL LISTS + +Access Control Lists (ACLs) appear in many places in the configuration file, +used to select or exclude devices or hosts for certain settings. ACLs are a +YAML list of items, which can contain: + +=over 4 + +=item * + +Hostname, IP address, IP prefix (i.e. subnet) + +=item * + +IP address range, using a hyphen on the last octet/hextet, and no whitespace + +=item * + +Regular expression in YAML format (no enforced anchors) which will match the +device DNS name (using a fresh DNS lookup, so works on new discovery), e.g.: + + - !!perl/regexp ^sep0.*$ + +=item * + +"C" - matched against a device property, such as C or +C (with enforced begin/end regexp anchors). + +=item * + +"C" to require all items to match (or not match) the provided IP or +device. Note that this includes IP address version mismatches (v4-v6). + +=back + +To negate any item in an ACL, prefix it with "C", for example +"C". In that case, the item must I match the device. This +does not apply to regular expressions (which you can achieve with nonmatching +lookahead). + +To match any device, use "C". To match no devices we suggest using +"C" in the list (or "C" may work). + =head1 SUPPORTED SETTINGS =head2 Essential Settings @@ -647,13 +699,8 @@ devices. For more fine-grained control see the C setting. Value: List of Network Identifiers or Device Properties. Default: Empty List. -IP addresses in the list will use C (and not C). You can -include hostnames, IP addresses, subnets (IPv4 or IPv6), YAML Regexp to match -the DNS name, and address ranges (using a hyphen and no whitespace) in the -list. - -Alternatively include a "C" entry to match the named property -of the device. The regex must match the complete value. +IP addresses in the list will use C (and not C). See +L for what you can use here. =head3 C @@ -714,25 +761,16 @@ Number of times to retry connecting to a device before giving up. Value: List of Network Identifiers or Device Properties. Default: Empty List. -IP addresses in the list will not be visited during device discovery. You can -include hostnames, IP addresses, subnets (IPv4 or IPv6), YAML Regexp to match -the DNS name, and address ranges (using a hyphen and no whitespace) in the -list. +IP addresses in the list will not be visited during device discovery. See +L for what you can use here. -Alternatively include a "C" entry to match the named property -of the device. The regex must match the complete value. =head3 C Value: List of Network Identifiers or Device Properties. Default: Empty List. If present, device discovery will be limited to IP addresses matching entries -in this list. You can include hostnames, IP addresses, subnets (IPv4 and -IPv6), YAML Regexp to match the DNS name, and address ranges (using a hyphen -and no whitespace). - -Alternatively include a "C" entry to match the named property -of the device. The regex must match the complete value. +in this list. See L for what you can use here. =head3 C @@ -758,24 +796,15 @@ discover jobs for a device. Value: List of Network Identifiers or Device Properties. Default: Empty List. -IP addresses in the list will not be visited for macsuck. You can include -hostnames, IP addresses, subnets (IPv4 or IPv6), YAML Regexp to match the DNS -name, and address ranges (using a hyphen and no whitespace) in the list. - -Alternatively include a "C" entry to match the named property -of the device. The regex must match the complete value. +IP addresses in the list will not be visited for macsuck. See L for what you can use here. =head3 C Value: List of Network Identifiers or Device Properties. Default: Empty List. If present, macsuck will be limited to IP addresses matching entries in this -list. You can include hostnames, IP addresses, subnets (IPv4 and IPv6), YAML -Regexp to match the DNS name, and address ranges (using a hyphen and no -whitespace). - -Alternatively include a "C" entry to match the named property -of the device. The regex must match the complete value. +list. See L for what you can use here. =head3 C @@ -815,6 +844,7 @@ Value: List of Network Identifiers or Device Properties. Default: Empty List. Similar to C, but instead of skipping nodes on this device, they are allowed to gather on the upstream device port. Useful for devices which can be discovered by Netdisco but do not provide a MAC address table via SNMP. +See L for what you can use here. =head3 C @@ -848,24 +878,15 @@ macsuck jobs for a device. Value: List of Network Identifiers or Device Properties. Default: Empty List. -IP addresses in the list will not be visited for arpnip. You can include -hostnames, IP addresses, subnets (IPv4 or IPv6), YAML Regexp to match the DNS -name, and address ranges (using a hyphen and no whitespace) in the list. - -Alternatively include a "C" entry to match the named property -of the device. The regex must match the complete value. +IP addresses in the list will not be visited for arpnip. See L for what you can use here. =head3 C Value: List of Network Identifiers or Device Properties. Default: Empty List. If present, arpnip will be limited to IP addresses matching entries in this -list. You can include hostnames, IP addresses, subnets (IPv4 and IPv6), YAML -Regexp to match the DNS name, and address ranges (using a hyphen and no -whitespace). - -Alternatively include a "C" entry to match the named property -of the device. The regex must match the complete value. +list. See L for what you can use here. =head3 C @@ -878,19 +899,15 @@ arpnip jobs for a device. Value: List of Network Identifiers. Default: Empty List. -IP addresses in the list will not be visited for nbtstat. You can include -hostnames, IP addresses, subnets (nbtstat only supports IPv4), YAML Regexp -to match the DNS name, and address ranges (using a hyphen and no whitespace) -in the list. +IP addresses in the list will not be visited for nbtstat. See L for what you can use here. =head3 C Value: List of Network Identifiers. Default: Empty List. If present, nbtstat will be limited to IP addresses matching entries in this -list. You can include hostnames, IP addresses, subnets -(nbtstat only supports IPv4), YAML Regexp to match the DNS name, and address -ranges (using a hyphen and no whitespace). +list. See L for what you can use here. =head3 C diff --git a/lib/App/Netdisco/Util/DNS.pm b/lib/App/Netdisco/Util/DNS.pm index 914024eb..b6439168 100644 --- a/lib/App/Netdisco/Util/DNS.pm +++ b/lib/App/Netdisco/Util/DNS.pm @@ -2,8 +2,8 @@ package App::Netdisco::Util::DNS; use strict; use warnings; - use Dancer ':script'; + use Net::DNS; use AnyEvent::DNS; use NetAddr::IP::Lite ':lower'; diff --git a/lib/App/Netdisco/Util/Node.pm b/lib/App/Netdisco/Util/Node.pm index f603ca5e..1f8dbe24 100644 --- a/lib/App/Netdisco/Util/Node.pm +++ b/lib/App/Netdisco/Util/Node.pm @@ -129,29 +129,8 @@ is undefined or empty, then C also returns false. print "rejected!" if check_node_no($ip, 'nbtstat_no'); -There are several options for what C<$setting_name> can contain: - -=over 4 - -=item * - -Hostname, IP address, IP prefix - -=item * - -IP address range, using a hyphen and no whitespace - -=item * - -Regular Expression in YAML format which will match the node DNS name, e.g.: - - - !!perl/regexp ^sep0.*$ - -=back - -To simply match all nodes, use "C" or IP Prefix "C<0.0.0.0/0>". All -regular expressions are anchored (that is, they must match the whole string). -To match no nodes we recommend an entry of "C" in the setting. +There are several options for what C<$setting_name> can contain. See +L for the details. =cut @@ -172,29 +151,8 @@ is undefined or empty, then C also returns true. print "rejected!" unless check_node_only($ip, 'nbtstat_only'); -There are several options for what C<$setting_name> can contain: - -=over 4 - -=item * - -Hostname, IP address, IP prefix - -=item * - -IP address range, using a hyphen and no whitespace - -=item * - -Regular Expression in YAML format which will match the node DNS name, e.g.: - - - !!perl/regexp ^sep0.*$ - -=back - -To simply match all nodes, use "C" or IP Prefix "C<0.0.0.0/0>". All -regular expressions are anchored (that is, they must match the whole string). -To match no nodes we recommend an entry of "C" in the setting. +There are several options for what C<$setting_name> can contain. See +L for the details. =cut diff --git a/lib/App/Netdisco/Util/Permission.pm b/lib/App/Netdisco/Util/Permission.pm index 9f26e973..6f926467 100644 --- a/lib/App/Netdisco/Util/Permission.pm +++ b/lib/App/Netdisco/Util/Permission.pm @@ -1,7 +1,8 @@ package App::Netdisco::Util::Permission; +use strict; +use warnings; use Dancer qw/:syntax :script/; -use Dancer::Plugin::DBIC 'schema'; use Scalar::Util 'blessed'; use NetAddr::IP::Lite ':lower'; @@ -27,12 +28,9 @@ subroutines. =head2 check_acl( $ip, \@config ) -Given an IP address, returns true if any of the items in C<< \@config >> -matches that address, otherwise returns false. - -Normally you use C and C, passing the name of the -configuration setting to load. This helper instead requires not the name of -the setting, but its value. +Given a Device or IP address, compares it to the items in C<< \@config >> +then returns true or false. You can control whether any item must match or +all must match, and items can be negated to invert the match logic. There are several options for what C<< \@config >> can contain: @@ -40,59 +38,82 @@ There are several options for what C<< \@config >> can contain: =item * -Hostname, IP address, IP prefix +Hostname, IP address, IP prefix (subnet) =item * -IP address range, using a hyphen and no whitespace +IP address range, using a hyphen on the last octet/hextet, and no whitespace =item * -Regular Expression in YAML format (no enforced anchors) which will match the +Regular expression in YAML format (no enforced anchors) which will match the device DNS name (using a fresh DNS lookup, so works on new discovery), e.g.: - !!perl/regexp ^sep0.*$ =item * -C<"property:regex"> - matched against a device property, such as C or -C (with enforced begin/end regex anchors) +"C" - matched against a device property, such as C or +C (with enforced begin/end regexp anchors). + +=item * + +"C" to require all items to match (or not match) the provided IP or +device. Note that this includes IP address version mismatches (v4-v6). =back -To simply match all devices, use "C" or IP Prefix "C<0.0.0.0/0>". -Property regular expressions are anchored (that is, they must match the whole -string). To match no devices we recommend an entry of "C" in the -setting. +To negate any entry, prefix it with "C", for example "C". In +that case, the item must I match the device. This does not apply to +regular expressions (which you can achieve with nonmatching lookahead). + +To match any device, use "C". To match no devices we suggest using +"C" in the list. =cut sub check_acl { my ($thing, $config) = @_; - my $real_ip = (blessed $thing ? $thing->ip : $thing); - my $addr = NetAddr::IP::Lite->new($real_ip); + my $real_ip = ( + (blessed $thing and $thing->can('ip')) ? $thing->ip : ( + (blessed $thing and $thing->can('addr')) ? $thing->addr : $thing )); + return 0 if blessed $real_ip; # class we do not understand + + my $addr = NetAddr::IP::Lite->new($real_ip); + my $name = hostname_from_ip($addr->addr) || '!!NO_HOSTNAME!!'; + my $all = (scalar grep {m/^op:and$/} @$config); + + INLIST: foreach my $item (@$config) { + next INLIST if $item eq 'op:and'; - foreach my $item (@$config) { if (ref qr// eq ref $item) { - my $name = hostname_from_ip($addr->addr) or next; - return 1 if $name =~ $item; - next; + if ($name =~ $item) { + return 1 if not $all; + } + else { + return 0 if $all; + } + next INLIST; } + my $neg = ($item =~ s/^!//); + if ($item =~ m/^([^:]+)\s*:\s*([^:]+)$/) { my $prop = $1; my $match = $2; # if not in storage, we can't do much with device properties - next unless blessed $thing and $thing->in_storage; + next INLIST unless blessed $thing and $thing->in_storage; # lazy version of vendor: and model: - if ($thing->can($prop) and defined $thing->$prop - and $thing->$prop =~ m/^$match$/) { - return 1; + if ($neg xor ($thing->can($prop) and defined $thing->$prop + and $thing->$prop =~ m/^$match$/)) { + return 1 if not $all; } - - next; + else { + return 0 if $all; + } + next INLIST; } if ($item =~ m/([a-f0-9]+)-([a-f0-9]+)$/i) { @@ -100,7 +121,7 @@ sub check_acl { my $last = $2; if ($item =~ m/:/) { - next unless $addr->bits == 128; + next INLIST if $addr->bits != 128 and not $all; $first = hex $first; $last = hex $last; @@ -109,31 +130,46 @@ sub check_acl { foreach my $part ($first .. $last) { my $ip = NetAddr::IP::Lite->new($header . sprintf('%x',$part) . '/128') or next; - return 1 if $ip == $addr; + if ($neg xor ($ip == $addr)) { + return 1 if not $all; + next INLIST; + } } + return 0 if (not $neg and $all); + return 1 if ($neg and not $all); } else { - next unless $addr->bits == 32; + next INLIST if $addr->bits != 32 and not $all; (my $header = $item) =~ s/\.[^.]+$/./; foreach my $part ($first .. $last) { my $ip = NetAddr::IP::Lite->new($header . $part . '/32') or next; - return 1 if $ip == $addr; + if ($neg xor ($ip == $addr)) { + return 1 if not $all; + next INLIST; + } } + return 0 if (not $neg and $all); + return 1 if ($neg and not $all); } - - next; + next INLIST; } my $ip = NetAddr::IP::Lite->new($item) - or next; - next unless $ip->bits == $addr->bits; + or next INLIST; + next INLIST if $ip->bits != $addr->bits and not $all; - return 1 if $ip->contains($addr); + if ($neg xor ($ip->contains($addr))) { + return 1 if not $all; + } + else { + return 0 if $all; + } + next INLIST; } - return 0; + return ($all ? 1 : 0); } 1; diff --git a/lib/App/Netdisco/Util/SNMP.pm b/lib/App/Netdisco/Util/SNMP.pm index 15339c84..57b504ee 100644 --- a/lib/App/Netdisco/Util/SNMP.pm +++ b/lib/App/Netdisco/Util/SNMP.pm @@ -300,7 +300,7 @@ sub _build_communities { if not $stanza->{tag} and !exists $stanza->{community}; - if ($stanza->{$mode} and check_acl($device->ip, $stanza->{only})) { + if ($stanza->{$mode} and check_acl($device, $stanza->{only})) { if ($device->in_storage and $stored_tag and $stored_tag eq $stanza->{tag}) { # last known-good by tag diff --git a/t/20-checkacl.t b/t/20-checkacl.t new file mode 100644 index 00000000..a9b7cb3d --- /dev/null +++ b/t/20-checkacl.t @@ -0,0 +1,94 @@ +#!/usr/bin/env perl + +use strict; use warnings FATAL => 'all'; +use Test::More 1.302083; + +BEGIN { + use_ok( 'App::Netdisco::Util::Permission', 'check_acl' ); +} + +my @conf = ( + # +ve match -ve match + 'localhost', '!www.example.com', # 0, 1 + '127.0.0.1', '!192.0.2.1', # 2, 3 + '::1', '!2001:db8::1', # 4, 5 + '127.0.0.0/29', '!192.0.2.0/24', # 6, 7 + '::1/128', '!2001:db8::/32', # 8, 9 + + '127.0.0.1-10', '!192.0.2.1-10', # 10,11 + '::1-10', '!2001:db8::1-10', # 12,13 + + qr/^localhost$/, qr/^www.example.com$/, # 14,15 + qr/(?!:www.example.com)/, '!127.0.0.0/29', # 16,17 + '!127.0.0.1-10', qr/(?!:localhost)/, # 18,19 + + 'op:and', # 20 +); + +# name, ipv4, ipv6, v4 prefix, v6 prefix +ok(check_acl('localhost',[$conf[0]]), 'same name'); +ok(check_acl('127.0.0.1',[$conf[2]]), 'same ipv4'); +ok(check_acl('::1',[$conf[4]]), 'same ipv6'); +ok(check_acl('127.0.0.0/29',[$conf[6]]), 'same v4 prefix'); +ok(check_acl('::1/128',[$conf[8]]), 'same v6 prefix'); + +# failed name, ipv4, ipv6, v4 prefix, v6 prefix +is(check_acl('www.microsoft.com',[$conf[0]]), 0, 'failed name'); +is(check_acl('172.20.0.1',[$conf[2]]), 0, 'failed ipv4'); +is(check_acl('2001:db8::5',[$conf[4]]), 0, 'failed ipv6'); +is(check_acl('172.16.1.3/29',[$conf[6]]), 0, 'failed v4 prefix'); +is(check_acl('2001:db8:f00d::/64',[$conf[8]]), 0, 'failed v6 prefix'); + +# negated name, ipv4, ipv6, v4 prefix, v6 prefix +ok(check_acl('localhost',[$conf[1]]), 'not same name'); +ok(check_acl('127.0.0.1',[$conf[3]]), 'not same ipv4'); +ok(check_acl('::1',[$conf[5]]), 'not same ipv6'); +ok(check_acl('127.0.0.0/29',[$conf[7]]), 'not same v4 prefix'); +ok(check_acl('::1/128',[$conf[9]]), 'not same v6 prefix'); + +# v4 range, v6 range +ok(check_acl('127.0.0.1',[$conf[10]]), 'in v4 range'); +ok(check_acl('::1',[$conf[12]]), 'in v6 range'); + +# failed v4 range, v6 range +is(check_acl('172.20.0.1',[$conf[10]]), 0, 'failed v4 range'); +is(check_acl('2001:db8::5',[$conf[12]]), 0, 'failed v6 range'); + +# negated v4 range, v6 range +ok(check_acl('127.0.0.1',[$conf[11]]), 'not in v4 range'); +ok(check_acl('::1',[$conf[13]]), 'not in v6 range'); + +# hostname regexp +ok(check_acl('localhost',[$conf[14]]), 'name regexp'); +ok(check_acl('127.0.0.1',[$conf[14]]), 'IP regexp'); +is(check_acl('www.google.com',[$conf[14]]), 0, 'failed regexp'); + +# OR of prefix, range, regexp, property (2 of, 3 of, 4 of) +ok(check_acl('127.0.0.1',[@conf[8,0]]), 'OR: prefix, name'); +ok(check_acl('127.0.0.1',[@conf[8,12,0]]), 'OR: prefix, range, name'); +ok(check_acl('127.0.0.1',[@conf[8,12,15,0]]), 'OR: prefix, range, regexp, name'); + +# OR of negated prefix, range, regexp, property (2 of, 3 of, 4 of) +ok(check_acl('127.0.0.1',[@conf[17,0]]), 'OR: !prefix, name'); +ok(check_acl('127.0.0.1',[@conf[17,18,0]]), 'OR: !prefix, !range, name'); +ok(check_acl('127.0.0.1',[@conf[17,18,19,0]]), 'OR: !prefix, !range, !regexp, name'); + +# AND of prefix, range, regexp, property (2 of, 3 of, 4 of) +ok(check_acl('127.0.0.1',[@conf[6,0,20]]), 'AND: prefix, name'); +ok(check_acl('127.0.0.1',[@conf[6,10,0,20]]), 'AND: prefix, range, name'); +ok(check_acl('127.0.0.1',[@conf[6,10,14,0,20]]), 'AND: prefix, range, regexp, name'); + +# failed AND on prefix, range, regexp +is(check_acl('127.0.0.1',[@conf[8,10,14,0,20]]), 0, 'failed AND: prefix!, range, regexp, name'); +is(check_acl('127.0.0.1',[@conf[6,12,14,0,20]]), 0, 'failed AND: prefix, range!, regexp, name'); +is(check_acl('127.0.0.1',[@conf[6,10,15,0,20]]), 0, 'failed AND: prefix, range, regexp!, name'); + +# AND of negated prefix, range, regexp, property (2 of, 3 of, 4 of) +ok(check_acl('127.0.0.1',[@conf[9,0,20]]), 'AND: !prefix, name'); +ok(check_acl('127.0.0.1',[@conf[7,11,0,20]]), 'AND: !prefix, !range, name'); +ok(check_acl('127.0.0.1',[@conf[9,13,16,0,20]]), 'AND: !prefix, !range, !regexp, name'); + +# device property +# negated device property + +done_testing;