Feature to gather SNMP Walk, use as Pseudo Device, and Browse Objects

* fix anomalous name * add gather worker * fix encoding of binary storage * store results back to job * now parsing mbis report to translate * fix the broken report parser * rename gather to snapshot * implement walk code copied from SNMP::Info * can now bulkwalk and parse mibs report and store resolved walk in cache * add func/glob aliasing broken * better aliasing * implement aliasing from globals and funcs * fix regexp for matching netdisco-mibs report * fake cache entry for all ND2 methods called, add comments * also save to logs/snapshots/IP * add doc for netdisco-do * add is_pseudo column to device table * support for loading cache for pseudo devices * check for hrSystemUptime as well as sysUpTime for snmp connect * display pseudo devices with yellow pill for name * color all cells for layers for pseudo * no need to b64 encode binary data in scalars as we b64 whole thing after * tweaked uptime check * store snapshot to database instead of Job * expose snapshots in device details tab * small ux improvements on snap download * fixes for errors in subnet mask searching * hide snapshot management for pseudo devices * update to use new netdisco-mibs object cache * update for new format oids file * start of work on loading walk into db for browsing * store values and meta * add auto increment col and oid index to browser * start web plugin for browser * add virtual search for oid children * have all oid in separte table (60 seconds load on my laptop) * rename table and add relation * store oid as int array * fix sql for children * make jstree start working * working very slow tree expand * fix to work when first displaying tree * store both oid and oid_parts * simplify SQL to speed up (more complicated perl) * fix sql bug, add better index, prettify tree * render the snmp node detail * add node template, make scrollable, pretty print data values (insecure) * store munge hint * some dubious code to munge the data * make sure to filter by IP on device_browser * make safer the rendering of value data (but need to come back to key ordering) * fix sorting on object values * limit the opening of child nodes to keep response good and unclutter * factor out the munge and make safer * reject unknown mungers * show the munger and option (not working) to change * additional js for munge select * complete custom munge * change so that saving to database is only at CLI and on request * hide snmp tab if no browser rows in the db * add helpful message when no browser rows for the device * stub handler for search and add recurse control * working search * minor ui fixes * implement typeahead for leaf search * limit rows in typeahead * make sure device_browser is visited in delete and renumber * add requirements for this branch * update manifest * make sure node search and typeahead are restricted to current device only
2021-11-06 07:47:29 +00:00
parent 74210dd78f
commit dc1f76c1aa
59 changed files with 10099 additions and 44 deletions
--- a/lib/App/Netdisco/Worker/Plugin/LoadMIBs.pm
+++ b/lib/App/Netdisco/Worker/Plugin/LoadMIBs.pm
@@ -0,0 +1,50 @@
+package App::Netdisco::Worker::Plugin::LoadMIBs;
+
+use Dancer ':syntax';
+use App::Netdisco::Worker::Plugin;
+use aliased 'App::Netdisco::Worker::Status';
+
+use Dancer::Plugin::DBIC 'schema';
+
+use File::Spec::Functions qw(catdir catfile);
+use File::Slurper qw(read_lines write_text);
+# use DDP;
+
+register_worker({ phase => 'main' }, sub {
+  my ($job, $workerconf) = @_;
+
+  debug "loadmibs - loading netdisco-mibs object cache";
+
+  my $home = (setting('mibhome') || catdir(($ENV{NETDISCO_HOME} || $ENV{HOME}), 'netdisco-mibs'));
+  my @report = read_lines(catfile($home, qw(EXTRAS reports all_oids)), 'latin-1');
+
+  my @browser = ();
+  foreach my $line (@report) {
+    my ($oid, $qual_leaf, $type, $access, $index) = split m/,/, $line;
+    next unless defined $oid and defined $qual_leaf;
+    my ($mib, $leaf) = split m/::/, $qual_leaf;
+    push @browser, {
+      oid    => $oid,
+      oid_parts => [ grep {length} (split m/\./, $oid) ],
+      mib    => $mib,
+      leaf   => $leaf,
+      type   => $type,
+      access => $access,
+      index  => [($index ? (split m/:/, $index) : ())],
+    };
+  }
+
+  debug sprintf "loadmibs - loaded %d objects from netdisco-mibs",
+    scalar @browser;
+
+  schema('netdisco')->txn_do(sub {
+    my $gone = schema('netdisco')->resultset('SNMPObject')->delete;
+    debug sprintf ' loadmibs - removed %d oids', $gone;
+    schema('netdisco')->resultset('SNMPObject')->populate(\@browser);
+    debug sprintf ' loadmibs - added %d new oids', scalar @browser;
+  });
+
+  return Status->done('Loaded MIBs');
+});
+
+true;