Feature to gather SNMP Walk, use as Pseudo Device, and Browse Objects

* fix anomalous name

* add gather worker

* fix encoding of binary storage

* store results back to job

* now parsing mbis report to translate

* fix the broken report parser

* rename gather to snapshot

* implement walk code copied from SNMP::Info

* can now bulkwalk and parse mibs report and store resolved walk in cache

* add func/glob aliasing broken

* better aliasing

* implement aliasing from globals and funcs

* fix regexp for matching netdisco-mibs report

* fake cache entry for all ND2 methods called, add comments

* also save to logs/snapshots/IP

* add doc for netdisco-do

* add is_pseudo column to device table

* support for loading cache for pseudo devices

* check for hrSystemUptime as well as sysUpTime for snmp connect

* display pseudo devices with yellow pill for name

* color all cells for layers for pseudo

* no need to b64 encode binary data in scalars as we b64 whole thing after

* tweaked uptime check

* store snapshot to database instead of Job

* expose snapshots in device details tab

* small ux improvements on snap download

* fixes for errors in subnet mask searching

* hide snapshot management for pseudo devices

* update to use new netdisco-mibs object cache

* update for new format oids file

* start of work on loading walk into db for browsing

* store values and meta

* add auto increment col and oid index to browser

* start web plugin for browser

* add virtual search for oid children

* have all oid in separte table (60 seconds load on my laptop)

* rename table and add relation

* store oid as int array

* fix sql for children

* make jstree start working

* working very slow tree expand

* fix to work when first displaying tree

* store both oid and oid_parts

* simplify SQL to speed up (more complicated perl)

* fix sql bug, add better index, prettify tree

* render the snmp node detail

* add node template, make scrollable, pretty print data values (insecure)

* store munge hint

* some dubious code to munge the data

* make sure to filter by IP on device_browser

* make safer the rendering of value data (but need to come back to key ordering)

* fix sorting on object values

* limit the opening of child nodes to keep response good and unclutter

* factor out the munge and make safer

* reject unknown mungers

* show the munger and option (not working) to change

* additional js for munge select

* complete custom munge

* change so that saving to database is only at CLI and on request

* hide snmp tab if no browser rows in the db

* add helpful message when no browser rows for the device

* stub handler for search and add recurse control

* working search

* minor ui fixes

* implement typeahead for leaf search

* limit rows in typeahead

* make sure device_browser is visited in delete and renumber

* add requirements for this branch

* update manifest

* make sure node search and typeahead are restricted to current device only

lib/App/Netdisco/Transport/SNMP.pm

@@ -10,7 +10,12 @@ use App::Netdisco::Util::Permission ':all';
 use SNMP::Info;
 use Try::Tiny;
 use Module::Load ();
+use Storable 'thaw';
+use File::Slurper 'read_text';
+use MIME::Base64 'decode_base64';
 use Path::Class 'dir';
+use File::Path 'make_path';
+use File::Spec::Functions qw(catdir catfile);
 use NetAddr::IP::Lite ':lower';
 use List::Util qw/pairkeys pairfirst/;
 
@@ -58,7 +63,9 @@ Returns C<undef> if the connection fails.
 sub reader_for {
   my ($class, $ip, $useclass) = @_;
   my $device = get_device($ip) or return undef;
-  return undef if $device->in_storage and $device->is_pseudo;
+
+  my $pseudo_cache = catfile( catdir(($ENV{NETDISCO_HOME} || $ENV{HOME}), 'logs', 'snapshots'), $device->ip );
+  return undef if $device->in_storage and $device->is_pseudo and ! -f $pseudo_cache;
 
   my $readers = $class->instance->readers or return undef;
   return $readers->{$device->ip} if exists $readers->{$device->ip};
@@ -107,6 +114,7 @@ Returns C<undef> if the connection fails.
 sub writer_for {
   my ($class, $ip, $useclass) = @_;
   my $device = get_device($ip) or return undef;
+
   return undef if $device->in_storage and $device->is_pseudo;
 
   my $writers = $class->instance->writers or return undef;
@@ -155,6 +163,13 @@ sub _snmp_connect_generic {
       $snmp_args{BulkWalk} = 0;
   }
 
+  # support for offline cache
+  if ($device->is_pseudo) {
+    my $pseudo_cache = catfile( catdir(($ENV{NETDISCO_HOME} || $ENV{HOME}), 'logs', 'snapshots'), $device->ip );
+    $snmp_args{Cache} = thaw( decode_base64( read_text($pseudo_cache) ) );
+    $snmp_args{Offline} = 1;
+  }
+
   # get the community string(s)
   my @communities = get_communities($device, $mode);
 
@@ -242,7 +257,7 @@ sub _try_read {
 
   return undef unless (
     (not defined $info->error)
-    and defined $info->uptime
+    and (defined $info->uptime or defined $info->hrSystemUptime or defined $info->sysUpTime)
     and ($info->layers or $info->description)
     and $info->class
   );