API Endpoints to submit arpnip and macsuck results (#942)

* Add macsuck worker to collect various PortAccessEntity (NAC) attributes

* Incorporate PAE feedback on #937

 * missing Result/Device.pm column added
 * pae_is... columns instead of pae_capabilities
 * moved most code to Util/PortAccessEntity.pm so the update can
   be done in discover and macsuck

* Refactor PAE attributes during discover as separate Plugin

* PortAccessEntity: don't use device->dns in log string

* Fix "Experimental keys on scalar is now forbidden" test failure

* Revamp pae_control and add missing attribute

 - device.pae_control (text) is now device.pae_is_enabled (bool)
 - also store pae_authconfig_port_control (port mode auto/force(un)Auth)

* Fix "Experimental keys on scalar is now forbidden" test failure

 - ... again because of botched merge
 - at least perlgolfed away a set of curly braces

* Update PortAccessEntity.pm

* Incorporate @ollyg PR feedback

* allow actions without transport to run when there are also no creds

* initial refactor for separate gather, process, store phases for macsuck

* factor out the vlan sanity check

* additional help with log of action workers

* cleanup logic in check macsuck

* refactor to make main phases only

* some fixes

* implement file slurp. amazingly the whole thing works

* remove outdated noop from test

* treat error as critical, use cancel to suppress further drivers

* big refactor to share mac sanity code to both paths

* fix inverted logic on vlan sanity filter

* some code tidy

* fix error in default value

* fix for vlan 0 nodes input from cli

* ensure imported MACs are IEEE format

* add api endpoint, no useful return status yet

* exit status if error from nodes PUT

* suppress other networked workers when direct workers are active

* better log showing worker

* fix status recording to get first error or last done message

* implement arpnip API PUT

* avoid package redeclaration error

* make sure write API methods require admin status

* add doc for passing JSON data to arpnip and macsuck

* update manifest

* remove option to do jobs in web handler; all by queue now

* use job entry timestamp for offline queued jobs

* fix store username and IP on api PUT

* never de-duplicate user-submitted jobs; never reset DeviceSkip for offline jobs

* myworker no longer needed

* make logic cleaner

Co-authored-by: Christian Ramseyer <ramseyer@netnea.com>

lib/App/Netdisco/Backend/Job.pm

@@ -23,6 +23,7 @@ foreach my $slot (qw/
       device_key
       job_priority
       is_cancelled
+      is_offline
 
       _current_phase
       _last_namespace
@@ -93,8 +94,6 @@ sub best_status {
   my $cur_level = 0;
   my $cur_status = '';
 
-  return Status->error()->status if $job->is_cancelled;
-
   foreach my $status (reverse @{ $job->_statuslist }) {
     next if $status->phase
       and $status->phase !~ m/^(?:early|main|store|late)$/;
@@ -116,19 +115,13 @@ Find the best status and log it into the job's C<status> and C<log> slots.
 
 sub finalise_status {
   my $job = shift;
-  # use DDP; p $job->_statuslist;
+  # use DDP; p $job->_statuslist;
 
   # fallback
   $job->status('error');
   $job->log('failed to report from any worker!');
 
-  my $max_level = Status->info()->level;
-
-  if ($job->is_cancelled and scalar @{ $job->_statuslist }) {
-    $job->status( $job->_statuslist->[-1]->status );
-    $job->log( $job->_statuslist->[-1]->log );
-    return;
-  }
+  my $max_level = 0;
 
   foreach my $status (reverse @{ $job->_statuslist }) {
     next if $status->phase
@@ -137,7 +130,12 @@ sub finalise_status {
     # done() from check phase should not be the action's done()
     next if $status->phase eq 'check' and $status->is_ok;
 
-    if ($status->level >= $max_level) {
+    # for done() we want the latest log message
+    # for error() (and others) we want the earliest log message
+
+    if (($max_level != Status->done()->level and $status->level >= $max_level)
+        or ($status->level > $max_level)) {
+
       $job->status( $status->status );
       $job->log( $status->log );
       $max_level = $status->level;