Documentation Additions

- add CSV plugin output docs
 - add node about migrating from netdisco 1
 - add authZ note to developer docs
 - add doc on X namespace and site_plugins INC dir
 - js and css plugin docs note
 - add port col plugin docs
 - suggest enable housekeeping in docs
 - document Observium plugin
 - add plugin config advice to docs

Netdisco/lib/App/Netdisco.pm

@@ -220,16 +220,21 @@ Run the following command to start the web-app server as a backgrounded daemon
  ~/bin/netdisco-web start
 
 If the Inventory is empty because this is a new installation, you probably
-want to either run some polling jobs from the command-line, or give a web user
+want to run some polling jobs. This can be done from from the web interface or
-some admin rights (TODO: user rights management).
+command-line (see L<netdisco-do>).
 
 Run the following command to start the job control daemon (port control, etc):
 
  ~/bin/netdisco-daemon start
 
-You should take care not to run this Netdisco daemon and the legacy daemon at
+You should take care not to run this Netdisco daemon and the Netdisco 1.x
-the same time. Similarly, if you use the device discovery with Netdisco 2,
+daemon at the same time. Similarly, if you use the device discovery with
-disable your system's cron jobs for the Netdisco 1 poller.
+Netdisco 2, disable your system's cron jobs for the Netdisco 1.x poller.
+
+At this point you can revisit the C<~/environments/deployment.yml> file to
+uncomment more configuration. Check out the community string settings, and
+C<housekeeping> which enables the automatic periodic device discovery. See
+L<Configuration|App::Netdisco::Manual::Configuration> for further details.
 
 =head1 Upgrading
 
@@ -297,13 +302,6 @@ within the L<Developer|App::Netdisco::Manual::Developing> documentation.
 
 =head1 Caveats
 
-Some sections are not yet implemented, e.g. the I<Device Module> tab.
-
-Some menu items on the main black navigation bar go nowhere.
-
-None of the Reports yet exist (e.g. searching for wireless devices, or duplex
-mismatches). These will be implemented as a plugin bundle.
-
 The Wireless, IP Phone and NetBIOS Node properies are not yet shown.
 
 =head1 AUTHOR

Netdisco/lib/App/Netdisco/Manual/Developing.pod

@@ -36,20 +36,20 @@ codebase, hopefully in enough detail that you can get hacking yourself :-)
 
 This is Netdisco major version 2. The minor version has six digits, which are
 split into two components of three digits each. It's unlikely that the major
-version number (2) will increment. Each "feature" release to CPAN will
+version number (2) will increment. Each "significant" release to CPAN will
-increment the first three digits of the minor version. Each "bug fix" release
+increment the first three digits of the minor version. Each "trivial" release
 will increment the second three digits of the minor version.
 
-Stable releases will have an even "feature" number. Beta releases will have an
+Beta releases will have a a suffix with an underscore, to prevent CPAN
-odd "feature" number and also a suffix with an underscore, to prevent CPAN
 indexing the distribution. Some examples:
 
- 2.002002     - "feature" release 2, "bug fix" release 2
+ 2.002002     - "significant" release 2, second "trivial" release
- 2.002003     - another bug was found and fixed, hence "bug fix" release 3
+ 2.002003     - a bug was found and fixed, hence "trivial" release 3
- 2.003000_001 - first beta for the next "feature" release
+ 2.003000_001 - first beta for the next "significant" release
  2.003000_002 - second beta
- 2.004000     - the next "feature" release
+ 2.004000     - the next "significant" release
 
+The words "significant" and "trivial" are entirely subjective, of course.
 
 =head1 Global Configuration
 
@@ -98,7 +98,6 @@ need to do to access the Netdisco database, with no additional setup, is:
 
  schema('netdisco')->resultset(...)->search({...});
 
-
 =head1 DBIx::Class Layer
 
 DBIx::Class, or DBIC for short, is an Object-Relational Mapper. This means it
@@ -174,15 +173,11 @@ using the C<nd-dbic-versions> script which also ships with the distribution.
 
 =head2 Foreign Key Constraints
 
-We have not yet deployed any FK constraints into the Netdisco schema. This is
+We have not deployed any FK constraints into the Netdisco schema. This is
 partly because the current poller inserts and deletes entries from the
 database in an order which would violate such constraints, but also because
 some of the archiving features of Netdisco might not be compatible anyway.
 
-Regardless, a lack of FK constraints doesn't upset DBIx::Class. The
-constraints can easily be deployed in a future release of Netdisco.
-
-
 =head1 Web Application
 
 The Netdisco web app is a "classic" Dancer app, using most of the bundled
@@ -235,31 +230,23 @@ server so you can fire-and-forget the Netdisco web app without much trouble at
 all. This script in turn calls C<netdisco-web-fg> which is the real Dancer
 application, that runs in the foreground if called on its own.
 
-All web app code lives below L<App::Netdisco::Web>, but there are also some
-helper routines in L<App::Netdisco::Util::Web> (for example sorting device
-port names).
-
-If you're working on code in the web application, it's possible to have the
-app restart itself each time you save a file in your editor. The following
-command will watch the C<lib> and C<share> folder trees for changes, and you
-probably want to edit your C<deployment.yml> configuration to enable
-additional logging (see L<App::Netdisco::Manual::Configuration>):
-
- DANCER_ENVIRONMENT=development DBIC_TRACE_PROFILE=console DBIC_TRACE=1 \
-     ~/bin/localenv starman -R lib,share bin/netdisco-web-fg
-
 =head2 Authentication
 
-Dancer includes (of course) good session management using cookies and a memory
-database. You should change this to a disk database if using a proper forking
-web server installation so that sessions are available to all instances.
-
 Session and authentication code lives in L<App::Netdisco::Web::AuthN>. It is
 fully backwards compatible with the existing Netdisco user management, making
 use of the database users and their MD5 passwords.
 
 There is also support for unauthenticated access to the web app (for instance
 if you have some kind of external authentication, or simply trust everyone).
+See L<App::Netdisco::Manual::Configuration> for further details.
+
+=head2 Authorization
+
+Every Dancer route handler must have proper role based access control enabled,
+to prevent unauthorized access to Netdisco's data, or admin features. This is
+done with the L<Dancer::Plugin::Auth::Extensible> module. It handles both the
+authentication using Netdisco's database, and then protects each route
+handler. See L<App::Netdisco::Manual::WritingPlugins> for details.
 
 =head2 Templates
 
@@ -336,22 +323,6 @@ they're loaded within the page body by the templates. These files contain a
 function C<inner_view_processing> which is called each time AJAX delivers new
 content into a tab in the page (think of it like a callback, perhaps).
 
-Also in the C<share/public/javascripts/...> folder are the other public
-libraries loaded by the Netdisco application:
-
-The Toastr library is used for "Growl"-like notifications which appear in the
-corner of the web browser and then fade away. These notify the user of
-successful background job submission, and jos results.
-
-The d3 library is a graphics toolkit used to display the NetMap feature. This
-works differently from the old Netdisco in that everything is generated
-on-the-fly using SQL queries (C<DeviceLinks> resultset) and this d3 library
-for rendering.
-
-Finally Twitter Bootstrap also ships with a toolkit of helpful Javascript
-driven features such as the tooltips and collapsers.
-
-
 =head1 Job Daemon
 
 The old Netdisco has a job control daemon which processes "port control"
@@ -361,12 +332,9 @@ web application. However, it still makes use of the Dancer configuration and
 database connection management features mentioned above.
 
 The job daemon is backwards compatible with the old Netdisco database job
-requests table, although it doesn't yet log results in the same way. Most
+requests table. All code for the job daemon lives under the
-important, it cannot yet poll any devices for discovery or macsuck/arpnip,
+L<App::Netdisco::Daemon> namespace and like the rest of Netdisco is broken
-although that's next on the list!
+down into manageable chunks.
-
-All code for the job daemon lives under the L<App::Netdisco::Daemon> namespace
-and like the rest of Netdisco is broken down into manageable chunks.
 
 =head2 Running the Job Daemon
 
@@ -384,7 +352,7 @@ runs in the foreground if called on its own.
 The job daemon is based on the L<MCE> library, which handles the forking and
 management of child processes doing the actual work. This actually runs in the
 foreground unless wrapped with Daemon::Control, as mentioned above. MCE
-handles three flavours of "worker" for different tasks.
+handles four flavours of "worker" for different tasks.
 
 One goal that we had designing the daemon was that sites should be able to run
 many instances on different servers, with different processing capacities.
@@ -406,10 +374,8 @@ the "interactive" jobs of workers we have two types of worker.
 
 The Interactive worker picks jobs from the local job queue relating to device
 and port reconfiguration only. It submits results directly back to the central
-Netdisco database.
+Netdisco database. The Poller worker similarly picks job from the local queue,
-
+this time relating to device discovery and polling.
-The Poller worker (is not yet written!) and similarly picks job from the local
-queue, this time relating to device discovery and polling.
 
 There is support in the daemon for the workers to pick more than one job at a
 time from the local queue, in case we decide this is worth doing. However the
@@ -419,6 +385,11 @@ daemons on other servers). The user is free to configure the number of
 Interactive and Poller workers in their C<config.yml> file (zero or more of
 each).
 
+The fourth kind of worker is called the Scheduler and takes care of adding
+discover, macsuck, and arpnip jobs to the queue (which are in turn handled by
+the Poller worker). This worker is automatically started only if the user has
+enabled the "C<housekeeping>" section of their C<deployment.yml> site config.
+
 =head2 SNMP::Info
 
 The daemon obviously needs to use L<SNMP::Info> for device control. All the
@@ -434,9 +405,6 @@ Configuration for SNMP::Info comes from the YAML files, of course. This means
 that our C<mibhome> and C<mibdirs> settings are now in YAML format. In
 particular, the C<mibdirs> list is a real list within the configuration.
 
-Other libraries will be added to this namespace in due course, as we add more
-functionality to the Job Daemon.
-
 =head2 DBIx::Class Layer
 
 The local job queue for each Job Daemon is actually an SQLite database running

Netdisco/lib/App/Netdisco/Manual/ReleaseNotes.pod

@@ -8,6 +8,33 @@ This document will list only the most significant changes with each release of
 Netdisco. You are B<STRONGLY> recommended to read this document each time you
 install and upgrade. Also see the Changes file, for more information.
 
+=head1 Migrating from Netdisco 1.x
+
+This distribution (App::Netdisco) is a complete rewrite of the Netdisco
+application. Users often ask whether they can run both versions at the same
+time, and whether the database must be copied. Here are the guidelines for
+migrating from Netdisco 1.x:
+
+=over 4
+
+=item *
+
+You can run both Netdisco 1.x and App::Netdisco web frontends at the same
+time, using the same database.
+
+=item *
+
+Only enable the backend daemon and discovery jobs from I<either> Netdisco 1.x
+I<or> App::Netdisco.
+
+=item *
+
+You can share a single database between Netdisco 1.x and App::Netdisco. The
+deploy script for App::Netdisco will make some schema changes to the database,
+but they are backwards compatible.
+
+=back
+
 =head1 2.016000
 
 =head2 General Notices

Netdisco/lib/App/Netdisco/Manual/WritingPlugins.pod

@@ -14,7 +14,7 @@ See L<App::Netdisco::Web::Plugin> for more general information about plugins.
 =head1 Developing Plugins
 
 A plugin is simply a Perl module which is loaded. Therefore it can do anything
-you like, but most usefully for the App::Netdisco web application, the module
+you like, but most usefully for the App::Netdisco web application the module
 will install a L<Dancer> route handler subroutine, and link this to a web user
 interface (UI) component.
 
@@ -29,15 +29,15 @@ Here's the boilerplate code for our example plugin module:
  package App::Netdisco::Web::Plugin::MyNewFeature
  
  use Dancer ':syntax';
- use Dancer::Plugin::Ajax;
  use Dancer::Plugin::DBIC;
+ use Dancer::Plugin::Auth::Extensible;
  
  use App::Netdisco::Web::Plugin;
  
  # plugin registration code goes here, ** see below **
  
- # your Dancer route handler, for example:
+ # your Dancer route handler
- get '/mynewfeature' => sub {
+ get '/mynewfeature' => require_login sub {
    # ...lorem ipsum...
  };
  
@@ -91,17 +91,16 @@ Therefore your plugin module should look like the following:
  package App::Netdisco::Web::Plugin::Search::MyNewFeature
  
  use Dancer ':syntax';
- use Dancer::Plugin::Ajax;
  use Dancer::Plugin::DBIC;
+ use Dancer::Plugin::Auth::Extensible;
  
  use App::Netdisco::Web::Plugin;
  
  register_search_tab({tag => 'newfeature', label => 'My New Feature'});
  
- ajax '/ajax/content/search/newfeature' => sub {
+ get '/ajax/content/search/newfeature' => require_login sub {
    # ...lorem ipsum...
  
-   content_type('text/html');
    # return some HTML content here, probably using a template
  };
  
@@ -168,13 +167,39 @@ the following URL:
 
  /ajax/content/report/<report tag>
 
-Therefore you should implement in your plugin an AJAX handler for this path.
+Therefore you should implement in your plugin a route handler for this path.
 The handler must return the HTML content for the report. It can also process
 any query parameters which might customize the report search.
 
 See the L<App::Netdisco::Web::Plugin::Report::DuplexMismatch> module for a
 simple example of how to implement the handler.
 
+=head1 CSV Response
+
+Most pages in Netdisco are a table with data. It's possible to have the
+application add a link to download a CSV version of the same data. To do this,
+include the following option in your call to C<register_search_tab>,
+C<register_device_tab>, or C<register_report>:
+
+ provides_csv => 1
+
+The other thing you need to do is adjust your Dancer route handler to return
+either HTML or CSV data. Here's the typical way to do it:
+
+ get '/ajax/content/search/newfeature' => require_login sub {
+   # build some kind of dataset here (e.g. a DBIx::Class query)
+
+   if (request->is_ajax) {
+     template 'mytemplate', { data => $mydataset }, { layout => undef };
+   }
+   else {
+     header( 'Content-Type' => 'text/comma-separated-values' );
+     template 'mytemplate_csv', { data => $mydataset }, { layout => undef };
+   }
+ };
+
+Note that the C<is_ajax> call is part of the standard Dancer featureset.
+
 =head1 Admin Tasks
 
 These components appear in the black navigation bar under an Admin menu, but only
@@ -193,6 +218,64 @@ Note that this won't work for any target link - the path must be an
 App::Netdisco Dancer route handler. Please bug the App::Netdisco devs if you
 want arbitrary links supported.
 
+=head1 Device Port Columns
+
+You can also add columns to the Device Ports page. The canonical example of
+this is to add hyperlinks (or embedded images) of traffic graphs for each
+port, however the plugin is a regular Template::Toolkit template so can be any
+HTML output.
+
+The column plugin has a name (used internally to locate files on disk), label
+(the heading for the column in the table or CSV output), position in the table
+(three options: left, mid, right), and finally a flag for whether the column
+is displayed by default.
+
+To register the column call the following helper routine:
+
+ register_device_port_column({
+   name  => 'myportcolumnplugin',
+   label => 'My Port Column Heading',
+   position => 'left', # or "mid" or "right"
+   default  => 'on',   # or undef
+ });
+
+App::Netdisco searches for one Template::Toolkit file in the regular template
+include paths (see also C<register_template_path>, below). The template must
+be called "C<device_port_column.tt>" on disk and live in the directory:
+
+ plugin/myportcolumnplugin/device_port_column.tt
+
+For a good example of this, see the L<App::NetdiscoX::Web::Plugin::Observium>
+distribution.
+
+=head1 User Authorization
+
+All Dancer route handlers must have proper authorization configured. This is
+not difficult. Make sure that your module loads the
+L<Dancer::Plugin::Auth::Extensible> module (as shown above).
+
+For each route handler you either simply require that a user be logged in, or
+that the user is an administrator.
+
+To require a logged in user, include the C<require_login> wrapper:
+
+ get '/ajax/content/search/newfeature' => require_login sub {
+    # etc .....
+
+To require an administrator, specify their role:
+
+ get '/ajax/control/admin/newfeature' => require_role admin => sub {
+    # etc .....
+
+Finally in case you need it, the other role a user can have is
+C<port_control>:
+
+ ajax '/ajax/portcontrol' => require_role port_control => sub {
+    # etc .....
+
+Take care over the subtle differences in syntax, especially the placement of
+the fat comma ("C<< => >>").
+
 =head1 Templates
 
 All of Netdisco's web page templates are stashed away in its distribution,
@@ -215,7 +298,7 @@ templates along with your plugin, as shown.
 Each path added using C<register_template_path> is searched I<before> any
 existing paths in the template config.
 
-=head3 Template Variables
+=head2 Template Variables
 
 Some useful variables are made available in your templates automatically by
 App::Netdisco:
@@ -260,5 +343,78 @@ current page with some additional parameters, for example:
 
 =back
 
+=head1 Javascript and Stylesheets
+
+A simple mechanism exists for loading additional Javascript and CSS documents.
+This is done in the C<< <head> >> section of the web page.
+
+Within a Template include path (see C<register_template_path>, above) create a
+directory called "C<plugin>" and within that another directory named after
+your plugin (e.g. "C<mynewfeature>"). The Javascript and/or CSS files must
+then be named "C<mynewfeature.js>" and "C<mynewfeature.css>" respectively.
+For example:
+
+  plugin/mynewfeature/mynewfeature.js
+  plugin/mynewfeature/mynewfeature.css
+
+Tell App::Netdisco that you wish to load one or the other using the following
+helper routines:
+
+  register_javascript('mynewfeature');
+  register_css('mynewfeature');
+
+Note that this searches all template include paths, both those built into the
+application and those configured in your plugin(s) with
+C<register_template_path>.
+
+=head1 Naming and File Location
+
+There are several options for how you name, distribute and install your
+App::Netdisco plugin.
+
+=head2 Namespaces
+
+As mentioned in L<App::Netdisco::Web::Plugin>, official Netdisco plugins live
+in the C<App::Netdisco::Web::Plugin::> namespace. You can use this namespace
+and submit the product to the Netdisco developer team for consideration for
+inclusion in the official distribution.
+
+Alternatively you can release the plugin to CPAN under your own account. In
+that case we request that you instead use the C<App::NetdiscoX::Web::Plugin::>
+namespace (note the "X"). Users can load such modules by using the
+abbreviated form "X::MyPluginName" which is then expanded to the full package.
+
+=head2 File Location
+
+If writing your own plugins, Netdisco supports a local include path which is
+usually C<~/site_plugins> (or C<${NETDISCO_HOME}/site_plugins>).
+
+This means if your plugin is called
+"App::NetdiscoX::Web::Plugin::MyPluginName" then it could live at:
+
+ ~/site_plugins/App/NetdiscoX/Web/Plugin/MyPluginName.pm
+
+This feature should make development of new plugins or maintenance of local
+plugins much more straighforward.
+
+=head1 Plugin Configuration
+
+You can support new configuration items which the user should add to their
+C<~/environments/deployment.yml> file. Please use a single Hash-Ref option
+named "C<plugin_mypluginname>" (if your plugin's called C<mypluginname>). For
+example:
+
+ plugin_observium:
+   webhost: "web-server.example.com"
+   open_in_new_window: true
+
+You can then refer to this configuration in your plugin module:
+
+ my $webhost = setting('plugin_observium')->{'webhost'};
+
+Or in templates via Dancer's C<settings> key:
+
+ <a href="http://[% settings.plugin_observium.webhost | uri %]/>Observium</a>
+
 =cut
 

Netdisco/lib/App/NetdiscoX/Web/Plugin/Observium.pm

@@ -18,4 +18,75 @@ register_device_port_column({
 register_css('observium');
 register_javascript('observium');
 
+=head1 NAME
+
+App::NetdiscoX::Web::Plugin::Observium - Port Traffic Links and Thumbnail Graphs from Observium
+
+=head1 SYNOPSIS
+
+ # in your ~/environments/deployment.yml file
+  
+ extra_web_plugins:
+   - X::Observium
+ 
+ plugin_observium:
+   webhost: "web-server.example.com"
+   open_in_new_window: true
+
+=head1 Description
+
+This is a plugin for the L<App::Netdisco> network management application. It
+adds a column to the Device Ports table named "Traffic" with a link and
+thumbnail graph for the port, taken from a local Observium installation.
+
+=head1 Configuration
+
+Create an entry in your C<~/environments/deployment.yml> file named
+"C<plugin_observium>", containing the following settings:
+
+=head2 webhost
+
+Value: String, Required.
+
+Name of the server hosting your local Observium installation.
+
+=head2 open_in_new_window
+
+Value: Boolean. Default: false.
+
+If set to true, the hyperlink is configured to open the port's Observium page
+in  a new browser window or tab.
+
+=head1 AUTHOR
+
+Oliver Gorwits <oliver@cpan.org>
+
+=head1 COPYRIGHT AND LICENSE
+ 
+This software is copyright (c) 2013 by The Netdisco Developer Team.
+ 
+ Redistribution and use in source and binary forms, with or without
+ modification, are permitted provided that the following conditions are met:
+     * Redistributions of source code must retain the above copyright
+       notice, this list of conditions and the following disclaimer.
+     * Redistributions in binary form must reproduce the above copyright
+       notice, this list of conditions and the following disclaimer in the
+       documentation and/or other materials provided with the distribution.
+     * Neither the name of the Netdisco Project nor the
+       names of its contributors may be used to endorse or promote products
+       derived from this software without specific prior written permission.
+ 
+ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+ ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+ WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+ DISCLAIMED. IN NO EVENT SHALL THE NETDISCO DEVELOPER TEAM BE LIABLE FOR ANY
+ DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+ (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+=cut
+
 true;

Netdisco/share/config.yml

@@ -51,7 +51,7 @@ web_plugins:
   - Device::Modules
   - Device::Neighbors
   - Device::Addresses
-web_plugins_extra: []
+extra_web_plugins: []
 
 # -------------
 # NETDISCO CORE