135/netdisco
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

update docs to use GitHub wiki

Browse Source
This commit is contained in:
Oliver Gorwits
2017-11-19 13:43:08 +00:00
parent 5ff7d6fe47
commit 8b1b97f58c
13 changed files with 25 additions and 4054 deletions
Download Patch File Download Diff File Expand all files Collapse all files

45
lib/App/Netdisco.pm
View File

@@ -53,15 +53,15 @@ See the demo at: L<https://netdisco2-demo.herokuapp.com/>
=back
We have several other pages with tips for
L<alternate deployment scenarios|App::Netdisco::Manual::Deployment>,
L<understanding and troubleshooting Netdisco|App::Netdisco::Manual::Troubleshooting>,
L<tips and tricks for specific platforms|App::Netdisco::Manual::Vendors>,
and L<all the configuration options|App::Netdisco::Manual::Configuration>.
L<alternate deployment scenarios|https://github.com/netdisco/netdisco/wiki/Install-Tips>,
L<understanding and troubleshooting Netdisco|https://github.com/netdisco/netdisco/wiki/Troubleshooting>,
L<tips and tricks for specific platforms|https://github.com/netdisco/netdisco/wiki/Vendor-Tips>,
and L<all the configuration options|https://github.com/netdisco/netdisco/wiki/Configuration>.
You can also speak to someone in the C<#netdisco@freenode> IRC channel, or on
the L<community email list|https://lists.sourceforge.net/lists/listinfo/netdisco-users>.
Before installing or upgrading please always review the latest
L<Release Notes|App::Netdisco::Manual::ReleaseNotes>.
L<Release Notes|https://github.com/netdisco/netdisco/wiki/Release-Notes>.
=head1 Dependencies
@@ -77,7 +77,7 @@ On Fedora/Red-Hat:
root:~# yum install perl-core perl-DBD-Pg net-snmp-perl net-snmp-devel openssl-devel make automake gcc
On BSD systems please see L<our BSD tips|App::Netdisco::Manual::BSDInstall>.
On BSD systems please see L<our BSD tips|https://github.com/netdisco/netdisco/wiki/BSD-Install>.
With those installed, please check that your system's clock is correct.
@@ -118,8 +118,8 @@ The following is a general guide which works well in most circumstances. It
assumes you have a user C<netdisco> on your system, that you want to perform
an on-line installation, and have the application run self-contained from
within that user's home. There are alternatives: see the
L<Deployment|App::Netdisco::Manual::Deployment> documentation for further
details.
L<Deployment|https://github.com/netdisco/netdisco/wiki/Install-Tips>
documentation for further details.
To avoid muddying your system, use the following script to download and
install Netdisco and its dependencies into the C<netdisco> user's home area
@@ -160,7 +160,7 @@ uncomment the C<schedule> setting to enable SNMP data gathering from
devices (this replaces cron jobs in Netdisco 1).
Have a quick read of the other settings to make sure you're happy, then move
on. See L<Configuration|App::Netdisco::Manual::Configuration> for further
on. See L<Configuration|https://github.com/netdisco/netdisco/wiki/Configuration> for further
details.
=head1 Initialisation
@@ -174,7 +174,7 @@ script will take care of all this for you:
If this is a new installation of Netdisco 2, answer yes to all questions. If
you wish to deploy without Internet access, see the
L<Deployment|App::Netdisco::Manual::Deployment> documentation.
L<Deployment|https://github.com/netdisco/netdisco/wiki/Install-Tips> documentation.
=head1 Startup
@@ -196,15 +196,15 @@ C<~netdisco/perl5> directory and re-run the C<curl> command above, to update
Netdisco's C library bindings.
We have several other pages with tips for
L<alternate deployment scenarios|App::Netdisco::Manual::Deployment>,
L<understanding and troubleshooting Netdisco|App::Netdisco::Manual::Troubleshooting>,
L<tips and tricks for specific platforms|App::Netdisco::Manual::Vendors>,
and L<all the configuration options|App::Netdisco::Manual::Configuration>.
L<alternate deployment scenarios|https://github.com/netdisco/netdisco/wiki/Install-Tips>,
L<understanding and troubleshooting Netdisco|https://github.com/netdisco/netdisco/wiki/Troubleshooting>,
L<tips and tricks for specific platforms|https://github.com/netdisco/netdisco/wiki/Vendor-Tips>,
and L<all the configuration options|https://github.com/netdisco/netdisco/wiki/Configuration>.
You can also speak to someone in the C<#netdisco@freenode> IRC channel, or on
the L<community email list|https://lists.sourceforge.net/lists/listinfo/netdisco-users>.
Before installing or upgrading please always review the latest
L<Release Notes|App::Netdisco::Manual::ReleaseNotes>.
L<Release Notes|https://github.com/netdisco/netdisco/wiki/Release-Notes>.
=head1 Upgrading from 2.x
@@ -212,9 +212,8 @@ If you're running a version of Netdisco prior to 2.x then you should follow
the full installation instructions, above. This process is for upgrading
version 2.x only.
Before upgrading please review the latest L<Release
Notes|App::Netdisco::Manual::ReleaseNotes>. Then the process below should be
run for each installation:
Before upgrading please review the latest L<Release Notes|https://github.com/netdisco/netdisco/wiki/Release-Notes>.
Then the process below should be run for each installation:
# upgrade Netdisco
~/bin/localenv cpanm --notest App::Netdisco
@@ -262,18 +261,20 @@ run:
Bundled with this distribution is a L<DBIx::Class> layer for the Netdisco
database. This abstracts away all the SQL into an elegant, re-usable OO
interface. See the L<Developer|App::Netdisco::Manual::Developing>
interface. See the L<Developer|https://github.com/netdisco/netdisco/wiki/Developing>
documentation for further information.
=head2 Plugins
Netdisco includes a Plugin subsystem for customizing the web user interface.
See L<App::Netdisco::Web::Plugin> for further information.
Netdisco includes a Plugin subsystem for customizing the web user interface and backend daemon.
See L<Web Plugins|https://github.com/netdisco/netdisco/wiki/Web-Plugins>
and L<Backend Plugins|https://github.com/netdisco/netdisco/wiki/Backend-Plugins>
for further information.
=head2 Developing
Lots of information about the architecture of this application is contained
within the L<Developer|App::Netdisco::Manual::Developing> documentation.
within the L<Developer|https://github.com/netdisco/netdisco/wiki/Developing> documentation.
=head1 AUTHOR

75
lib/App/Netdisco/Manual/BSDInstall.pod
View File

@@ -1,75 +0,0 @@
=head1 NAME
App::Netdisco::Manual::BSDInstall - BSD Install Instructions
=head1 Introduction
This document is compiled from suggestions and comments on the Netdisco mail
lists. We're grateful for the help, and if you have any additions please do
let the project staff know.
You could also look at the following guide for FreeBSD 11: L<http://www.davidbolton.com/?p=681>
=head1 Ports Installs
=over 4
=item *
Perl (if not already installed)
=item *
C<p5-DBD-Pg> (will also pull in C<postgresqlXX-client>)
=item *
C<postgresqlXX-server>
=item *
C<net-snmp> (should install the Perl binding C<SNMP.pm>)
=item *
C<openssl>
=back
=head1 Additional Steps
=head2 Netdisco User
pw useradd netdisco -N -m -s /bin/sh -w no
=head2 PostgreSQL Setup
/usr/local/etc/rc.d/postgresql initdb
/usr/local/etc/rc.d/postgresql start
Make sure PostgreSQL starts at boot by adding the following to
"C</etc/rc.conf>":
postgresql_enable="YES"
When installing C<postgresqlXX-server> port, it creates the C<pgsql> user with
"C<nologin>" shell. As root, do C<vipw> and change the shell to C</bin/sh> or
whichever shell you want, so that you can do the C<createuser> step from the
main instructions.
C<pgtune> doesn't seem to work on NetBSD. This needs looking into. Nothing
will break, but it just means the server probably isn't tuned to your system's
hardware (RAM, etc).
=head1 Run Control Script
# PROVIDE: netdiscoweb
# REQUIRE: DAEMON
# BEFORE: LOGIN
# KEYWORD: shutdown
rcvar="netdisco_enable"
See also L<https://www.freebsd.org/doc/en/books/porters-handbook/rc-scripts.html>
=cut

1505
lib/App/Netdisco/Manual/Configuration.pod
View File

File diff suppressed because it is too large Load Diff

199
lib/App/Netdisco/Manual/Deployment.pod
View File

@@ -1,199 +0,0 @@
=head1 NAME
App::Netdisco::Manual::Deployment - Tips and Tricks for Deployment
=head1 Init and Run Control Scripts
The Netdisco applications will generate RC scripts suitable for Linux systems:
bin/netdisco-web get_init_file
bin/netdisco-backend get_init_file
On C<systemd>-based systems please see L<our systemd
guide|App::Netdisco::Manual::Systemd>.
On BSD systems please see L<our BSD tips|App::Netdisco::Manual::BSDInstall>,
and submit patches against the L<Daemon::Control> distribution.
=head1 Enable MD5 authentication to PostgreSQL
Some installations of PostgreSQL don't have MD5 authentication enabled by
default, which blocks database connections with the default Netdisco
configuration.
If your database and Netdisco are on the same system, then the easiest
solution is to comment out the "C<host:>" line in your C<deployment.yml> file.
Alternatively, reconfigure PostgreSQL to permit MD5 auth for TCP connections
by adding the following to your system's "C<pg_hba.conf>" file (and restarting
the database service):
# TYPE DATABASE USER ADDRESS METHOD
host all all 127.0.0.1/32 md5
=head1 Run multiple poller daemons
A common scenario is that the network is split into security domains (or
zones) and one Netdisco poller daemon cannot see all devices.
You can run multiple pollers, as long as they all connect back to the same
PostgreSQL database. Use the C<devices_only> or C<devices_no> configuration
settings to control which devices are "seen" by each poller. You can also
include only the necessary minimum SNMP community/authentication settings in
each poller's configuration.
Of course you will also need to start the web server somewhere, as well.
=head1 Connect to PostgreSQL database on non-standard port
The standard port for PostgreSQL is 5432. To connect on a different port you
need to use the C<host> option under C<database:> config in your
C<~/environments/deployment.yml> file. For example if connecting to a database
on the local server:
database:
name: 'netdisco'
user: 'changeme'
pass: 'changeme'
host: 'localhost;port=5432'
Obviously, substitute the C<user>, C<pass>, and actual port number for your
local values. Note the separator character is a semicolon.
=head1 Deploy without Internet access
The C<netdisco-deploy> script asks for Internet access but it is possible to
install off-line. You should download the following two files:
=over 4
=item *
https://raw.githubusercontent.com/netdisco/upstream-sources/master/ieee/oui.txt
=item *
https://github.com/netdisco/netdisco-mibs/releases/latest (download link is on the page)
=back
Run the C<netdisco-deploy> script but pass the OUI file name as a parameter on
the command line, for example:
~/bin/netdisco-deploy ./oui.txt
Then answer yes to questions, even though you're not connected to the
Internet.
For the MIBs you can simply extract the downloaded archive to the home
directory of Netdisco, and change the name of the directory to
C<netdisco-mibs>.
=head1 Relocating the Installation
The installation process installs Netdisco self-contained to your home
directory. The target directory can easily be changed by setting the
C<NETDISCO_HOME> environment variable, for example:
export NETDISCO_HOME=/opt/netdisco
Obviously, you'll need to substitute this wherever you see "C<~>" in the
installation instructions. The Netdisco application will use this setting
itself to locate files and configuration.
=head1 Pass Options to the Web Frontend Daemon
Simply add any options after the "C<start>" command. See other sections of
this document for some examples.
=head1 Non-root Hosting
Netdisco will assume its web site is hosted at the apex of your server - that
is, the document root. To relocate the web application, pass the C<--path>
parameter to the web startup script:
~/bin/netdisco-web start --path=/netdisco2
Alternatively, can set the C<path> configuration option in your
C<deployment.yml> file:
path: '/netdisco2'
=head1 Listening Port for the Web Frontend
Pass the C<--port> parameter to any of the web scripts. For example:
~/bin/netdisco-web start --port=8080
=head1 Listening Address for the Web Frontend
Pass the C<--host> parameter to any of the web scripts. For example:
~/bin/netdisco-web start --host=127.0.0.1
=head1 Behind a Proxy
By default the web application daemon starts listening on port 5000 and goes
into the background. This is ideal for hosting behind a web proxy (e.g. Apache
with C<mod_proxy>).
After enabling the C<headers>, C<proxy> and C<proxy_http> modules in Apache, a
suitable configuration would be:
ProxyPreserveHost On
ProxyPass / http://localhost:5000/ retry=0 timeout=60
ProxyPassReverse / http://localhost:5000/
ProxyRequests Off
<Proxy *>
Order allow,deny
Allow from all
</Proxy>
To combine this with Non-root Hosting as above, simply change the paths
referenced in the configuration, and set C<path> in your C<deployment.yml> as
discussed above. Note there is no trailing slash in the Apache config:
ProxyPass /netdisco2 http://localhost:5000/netdisco2 retry=0 timeout=60
ProxyPassReverse /netdisco2 http://localhost:5000/netdisco2
To delegate user authentication to Apache, use the C<trust_remote_user> or
C<trust_x_remote_user> settings. See L<App::Netdisco::Manual::Configuration>
for more details.
=head1 SSL Support
There is no SSL support in the built-in web server. This is because it's not
straightforward to support all the SSL options, and using port 443 requires
root privilege, which the Netdisco application should not have.
You are instead recommended to run C<netdisco-web> behind a reverse proxy as
described elsewhere in this document. Apache can easily act as an SSL reverse
proxy.
=head1 Database Backups
We recommend you backup the Netdisco database regularly. You could put the
following commands into a shell script and call it nightly from C<cron>:
DATE=`date +%Y%m%d`
/usr/bin/pg_dump -F c --create -f /path/to/backups/netdisco-pgsql-$DATE.dump netdisco
gzip -9f /path/to/backups/netdisco-pgsql-$DATE.dump
/usr/bin/find /path/to/backups/ -type f -ctime +30 -exec rm {} \;
This will keep 30 days of backups. You don't need to stop Netdisco during the
backup.
=head1 Display all Table Rows as Default
Add the following to your configuration:
table_pagesize: -1
=head1 Further Reading...
Other ways to run and host the web application can be found in the
L<Dancer::Deployment> page. See also the L<plackup> and L<starman>
documentation.
=cut

501
lib/App/Netdisco/Manual/Developing.pod
View File

@@ -1,501 +0,0 @@
=head1 NAME
App::Netdisco::Manual::Developing - Notes for contributors
=head1 DEVELOPER NOTES
This document aims to help developers understand the intent and design of the
code within Netdisco. Patches and feedback are always welcome :-)
=head1 TLDR; I want to clone git and run the web server.
First do a normal App::Netdisco install into a dedicated user's home, as per
the L<documentation|App::Netdisco>. Then:
su - netdisco && cd $HOME
mkdir git && cd git
git clone git://git.code.sf.net/p/netdisco/netdisco-ng netdisco-ng
cd netdisco-ng/Netdisco
DBIC_TRACE=1 ~/bin/localenv plackup -R share,lib -p 5001 bin/netdisco-web-fg
The above creates you a git clone (change the URL if you're a Netdisco
Developer) and runs the web server:
=over 4
=item *
In the foreground
=item *
Using a single process only (no forking)
=item *
With L<DBIx::Class> tracing
=item *
On port 5001 so it won't conflict with any already-running web frontend
=item *
Restarts the web server when you save a file in the C<share> or C<lib>
directory
=back
You might also want to set C<check_userlog> to C<false> in your config to
quieten some of the web client callbacks.
For the backend daemon, it's very similar:
DBIC_TRACE=1 ~/bin/localenv bin/netdisco-backend-fg
You can point at a different database without editing C<deployment.yml>:
NETDISCO_DBNAME=testdb DBIC_TRACE=1 ~/bin/localenv plackup -R share,lib -p 5001 bin/netdisco-web-fg
NETDISCO_DBNAME=testdb DBIC_TRACE=1 ~/bin/localenv bin/netdisco-backend-fg
It's recommended to delete the "C<~/perl5/lib/perl5/App/Netdisco>"
directory to avoid accidentally picking up old Netdisco code. For working on
L<SNMP::Info> you can similarly delete "C<~/perl5/lib/perl5/SNMP/Info*>" and
then symlink from "C<Info.pm>" and "C<Info>" to your git repo. If you pull
from upstream and the dependencies have changed, you can install them without
re-installing Netdisco itself:
cd netdisco-ng/Netdisco
~/bin/localenv cpanm --installdeps .
Happy hacking!
=head1 Introduction
This release of Netdisco is built as a L<Dancer> application, and uses many
modern technologies and techniques. Hopefully this will make the code easier
to manage and maintain in the long term.
Although Dancer is a web application framework, it provides very useful tools
for command line applications as well, namely configuration file management
and database connection management. We make use of these features in the
daemon and deployment scripts.
Overall the application tries to be as self-contained as possible without also
needing an excessive number of CPAN modules to be installed. However, Modern
Perl techniques have made dependency management almost a non-issue, and
Netdisco can be installed by and run completely within an unprivileged user's
account, apart from the PostgreSQL database setup.
Finally the other core component of Netdisco is now a L<DBIx::Class> layer for
database access. This means there is no SQL anywhere in the code, but more
important, we can re-use the same complex queries in different parts of
Netdisco.
The rest of this document discusses each "interesting" area of the Netdisco
codebase, hopefully in enough detail that you can get hacking yourself :-)
=head1 Versioning
This is Netdisco major version 2. The minor version has six digits, which are
split into two components of three digits each. It's unlikely that the major
version number (2) will increment. Each "significant" release to CPAN will
increment the first three digits of the minor version. Each "trivial" release
will increment the second three digits of the minor version.
Beta releases will have a a suffix with an underscore, to prevent CPAN
indexing the distribution. Some examples:
2.002002 - "significant" release 2, second "trivial" release
2.002003 - a bug was found and fixed, hence "trivial" release 3
2.003000_001 - first beta for the next "significant" release
2.003000_002 - second beta
2.004000 - the next "significant" release
The words "significant" and "trivial" are entirely subjective, of course.
=head1 Global Configuration
Dancer uses YAML as its standard configuration file format, which is flexible
enough for our needs, yet still simple to edit for the user. We no longer need
a parser as in the old version of Netdisco.
At the top of scripts you'll usually see something like:
use App::Netdisco;
use Dancer ':script';
First, this uses C<App::Netdisco>, which is almost nothing more than a
placeholder module (contains no actual application code). What it does is set
several environment variables in order to locate the configuration files.
Then, when we call "C<use Dancer>" these environment variables are used to
load two YAML files: C<config.yml> and C<< <environment>.yml >> where
C<< <environment> >> is typically either C<deployment> or C<development>.
The concept of "environments" allows us to have some shared "master" config
between all instances of the application (C<config.yml>), and then settings
for specific circumstances. Typically this might be logging levels, for
example. The default file which C<App::Netdisco> loads is C<deployment.yml>
but you can override it by setting the "C<DANCER_ENVIRONMENT>" environment
variable.
The file is located in an C<environments> folder which defaults to being in
the user's home directory. The name (or full path) of the folder can be
overriden using the "C<DANCER_ENVDIR>" environment variable. The location of
the folder alone can be overridden using the "C<NETDISCO_HOME>" environment
variable.
Dancer loads the config using YAML, merging data from the two files. Config is
made available via Dancer's C<setting('foo')> subroutine, which is exported.
So now the C<foo> setting in either config file is easily accessed.
Another line commonly seen in scripts is this:
use Dancer::Plugin::DBIC 'schema';
This plugin saves a lot of effort by taking some database connection
parameters from the configuration file, and instantiating DBIx::Class database
connections with them. The connections are managed transparently so all we
need to do to access the Netdisco database, with no additional setup, is:
schema('netdisco')->resultset(...)->search({...});
=head1 DBIx::Class Layer
DBIx::Class, or DBIC for short, is an Object-Relational Mapper. This means it
abstracts away the SQL of database calls, presenting a Perl object for each
table, set of results from a query, table row, etc. The advantage is that it
can generate really smart SQL queries, and these queries can be re-used
throughout the application.
The DBIC layer for Netdisco is based at L<App::Netdisco::DB>. This is the
global schema class and below that, under L<App::Netdisco::DB::Result> is a
class for each table in the database. These contain metadata on the columns
but also several handy "helper" queries which can be called. There are also
C<ResultSet> classes which provide additional "pre-canned" queries.
Netdisco's DBIx::Class layer has excellent documentation which you are
encouraged to read, particularly if you find it difficult to sleep.
=head2 Results and ResultSets
In DBIC a C<Result> is a table and a C<ResultSet> is a set of rows retrieved
from the table as a result of a query (which might be all the rows, of
course). This is why we have two types of DBIC class.
Items in the C<Result> generally relate to the single table
directly, and simply. In the C<ResultSet> class are more complex search
modifiers which might synthesize new "columns" of data (e.g. formatting a
timestamp) or subroutines which accept parameters to customize the query.
However, regardless of the actual class name, you access them in the same way.
For example the C<device> table has an L<App::Netdisco::DB::Result::Device>
class and also an L<App::Netdisco::DB::ResultSet::Device> class. DBIC merges
the two:
schema('netdisco')->resultset('Device')->get_models;
=head2 Virtual Tables (VIEWs)
Where we want to simplify our application code even further we can either
install a VIEW in PostgreSQL, or use DBIx::Class to synthesize the view
on-the-fly. Put simply, it uses the VIEW definition as the basis of an SQL
query, yet in the application we treat it as a real table like any other.
Some good examples are a fake table of only the active Nodes (as opposed to
all nodes), or the more complex list of all ports which are connected together
(C<DeviceLink>).
All these tables live under the
L<App::Netdisco::DB::Result::Virtual> namespace, and so you
access them like so (for the C<ActiveNode> example):
schema('netdisco')->resultset('Virtual::ActiveNode')->count;
=head2 Versioning and Deployment
To manage the Netdisco schema in PostgreSQL we use DBIx::Class's deployment
feature. This attaches a version to the schema and provides all the code to
check the current version and do whatever is necessary to upgrade.
The schema version is stored in a new table called
C<dbix_class_schema_versions>, although you should never touch it.
The C<netdisco-db-deploy> script included in the distribution performs the
following services:
* Installs the dbix_class_schema_versions table
* Upgrades the schema to the current distribtion's version
This works both on an empty, new database, and a legacy database from the
existing Netdisco release, in a non-destructive way. For further information
see L<DBIx::Class::Schema::Versioned> and the C<netdisco-db-deploy> script.
The files used for the upgrades are shipped with this distribution and stored
in the C<.../App/Netdisco/DB/schema_versions> directory. They are generated
using the C<nd-dbic-versions> script which also ships with the distribution.
=head2 Foreign Key Constraints
We have not deployed any FK constraints into the Netdisco schema. This is
partly because the current poller inserts and deletes entries from the
database in an order which would violate such constraints, but also because
some of the archiving features of Netdisco might not be compatible anyway.
=head1 Web Application
The Netdisco web app is a "classic" Dancer app, using most of the bundled
features which make development really easy. Dancer is based on Ruby's Sinatra
framework. Its style is for many "helper" subroutines to be exported into the
application namespace, to do things such as access request parameters,
navigate around the "handler" subroutines, manage response headers, and so on.
Pretty much anything you want to do in a web application has been wrapped up
by Dancer into a neat helper routine that does the heavy lifting. This
includes configuration and database connection management, as was discussed
above. Also, templates can be executed and Netdisco uses the venerable
L<Template::Toolkit> engine for this.
Like most web frameworks Dancer has a concept of "handlers" which are
subroutines to which a specific web request is routed. For example if the user
asks for "C</device>" with some parameters, the request ends up at the
L<App::Netdisco::Web::Device> package's "C<get '/device'>" handler. All this
is done automatically by Dancer according to some simple rules. There are also
"wrapper" subroutines which we use to do tasks such as setting up data lookup
tables, and handling authentication.
Dancer also supports AJAX very well, and it is used to retrieve most of the
data in the Netdisco web application in a dynamic way, to respond to search
queries and avoid lengthy page reloads. You will see the handlers for AJAX
look similar to those for GET requests but do not use Template::Toolkit
templates.
Compared to the current Netdisco, the handler routines are very small. This is
because (a) they don't include any HTML - this is delegated to a template, and
(b) they don't include an SQL - this is delegated to DBIx::Class. Small
routines are more manageable, and easier to maintain. You'll also notice use
of modules such as L<NetAddr::MAC> and L<NetAddr::IP::Lite> to simplify and make
more robust the handling of data.
In fact, many sections of the web application have been factored out into
separate Plugin modules. For more information see the
L<App::Netdisco::Web::Plugin> manual page.
=head2 Running the Web App
Dancer apps conform to the "PSGI" standard interface for web applications,
which makes for easy deployment under many stacks such as Apache, FCGI, etc.
See L<Dancer::Deployment> for more detail.
At a minimum Netdisco can run from within its own user area as an unprivileged
user, and actually ships with a fast, preforking web server engine. The
C<netdisco-web> script uses L<Daemon::Control> to daemonize this simple web
server so you can fire-and-forget the Netdisco web app without much trouble at
all. This script in turn calls C<netdisco-web-fg> which is the real Dancer
application, that runs in the foreground if called on its own.
=head2 Authentication
Session and authentication code lives in L<App::Netdisco::Web::AuthN>. It is
fully backwards compatible with the existing Netdisco user management, making
use of the database users and their MD5 passwords.
There is also support for unauthenticated access to the web app (for instance
if you have some kind of external authentication, or simply trust everyone).
See L<App::Netdisco::Manual::Configuration> for further details.
=head2 Authorization
Every Dancer route handler must have proper role based access control enabled,
to prevent unauthorized access to Netdisco's data, or admin features. This is
done with the L<Dancer::Plugin::Auth::Extensible> module. It handles both the
authentication using Netdisco's database, and then protects each route
handler. See L<App::Netdisco::Manual::WritingWebPlugins> for details.
=head2 Templates
In the C<share/views> folder of this distribution you'll find all the
Template::Toolkit template files, with C<.tt> extensions. Dancer first loads
C<share/views/layouts/main.tt> which is the main page wrapper, that has the HTML
header and so on. It then loads other templates for sections of the page body.
This is a typical Template::Toolkit "wrapper" configuration, as noted by the
C<[% content %]> call within C<main.tt> that loads the template you actually
specified in your Dancer handler.
All templates (and Javascript and Stylesheets) are shipped in the
L<App::Netdisco> distribution and located automatically by the application
(using the environment variables which App::Netdisco set up). The user doesn't
have to copy or install any files.
There's a template for the homepage called C<index.tt>, then separate
templates for searching, displaying device details, and showing inventory.
These are, pretty much, all that Netdisco ever does.
Each of these pages is designed in a deliberately similar way, with re-used
features. They each can have a "sidebar" with a search form (or additional
search parameters). They also can have a tabbed interface for sub-topics.
Here's where it gets interesting. Up till now the page content has been your
typical synchronous page load (a single page comprised of many templates) in
response to a GET request. However the content of the tabs is not within this.
Each tab has its content dynamically retrieved via an AJAX request back to the
web application. Javscript triggers this automatically on page load.
This feature allows the user to search and search again, each time refreshing
the data they see in the tab but without reloading the complete page with all
its static furniture. AJAX can, of course, return any MIME type, not only JSON
but also HTML content as in this case. The templates for the tabs are
organised below C<share/views/ajax/...> in the distribution.
=head2 Stylesheets
The main style for Netdisco uses Twitter Bootstrap, which is a modern library
of CSS and javascript used on many websites. It does a lot of heavy lifting,
providing simple CSS classes for all of the standard web page furniture
(forms, tables, etc). Check out the documetation at the Twitter Bootstrap web
site for more information.
These stylesheets are of course customised with our own C<netdisco.css>. We
try to name all CSS classes with a prefix "C<nd_>" so as to be distinct from
Twitter Bootstrap and any other active styles.
All stylesheets are located in the C<share/public/css> folder of the
distribution and, like the templates, are automatically located and served by
the Netdisco application. You can also choose to serve this content statically
via Apache/etc for high traffic sites.
Although Twitter Bootstrap ships with its own set of icons, we use an
alternative library called Fontawesome. This plugs in easily to Bootstrap and
provides a wider range of scaleable vectored icons which are easy to use.
=head2 Javascript
Of course many parts of the Netdisco site use Javascript, beginning with
retrieving the page tab content itself. The standard library in use is jQuery,
and the latest version is shipped with this distribution.
Many parts of the Netdisco site have small Javscript routines. The code for
these, using jQuery as mentioned, lives in two places. The main C<netdisco.js>
file is loaded once in the page HTML header, and lives in
C<share/public/javascripts/netdisco.js>. There's also a
C<netdisco_portcontrol.js> which is included only if the current user has Port
Control rights.
Netdisco also has Javascript routines specific to the device search or device
details pages, and these files are located in C<share/views/js/...> because
they're loaded within the page body by the templates. These files contain a
function C<inner_view_processing> which is called each time AJAX delivers new
content into a tab in the page (think of it like a callback, perhaps).
=head1 Job Daemon
The old Netdisco has a job control daemon which processes "port control"
actions and also manual requests for device polling. The new Netdisco also has
a daemon, although it is a true separate process and set of libraries from the
web application. However, it still makes use of the Dancer configuration and
database connection management features mentioned above.
The job daemon is backwards compatible with the old Netdisco database job
requests table. All code for the job daemon lives under the
L<App::Netdisco::Backend> namespace and like the rest of Netdisco is broken
down into manageable chunks.
=head2 Running the Job Daemon
Like the web application, the job daemon is fully self contained and runs via
two simple scripts shipped with the distribution - one for foreground and one
for background execution (see the user docs for instructions).
The C<netdisco-backend> script uses L<Daemon::Control> to daemonize so you can
fire-and-forget the Netdisco job daemon without much trouble at all. This
script in turn calls C<netdisco-backend-fg> which is the real application,
that runs in the foreground if called on its own.
=head2 Daemon Engineering
The job daemon is based on the L<MCE> library, which handles the forking and
management of child processes doing the actual work. This actually runs in the
foreground unless wrapped with Daemon::Control, as mentioned above. MCE
handles four flavours of "worker" for different tasks.
One goal that we had designing the daemon was that sites should be able to run
many instances on different servers, with different processing capacities.
This is both to take advantage of more processor capability, but also to deal
with security zones where you might only be able to manage a subset of devices
from certain locations. Netdisco has always coped well with this via its
C<discover_*> and similar configuration, and the separate poller process.
So, the single Manager "worker" in the daemon is responsible for contacting
the central Netdisco database and booking out jobs which it's able to service
according to the local configuration settings. Jobs are "locked" in the
central queue and then copied to a local job queue within the daemon.
There is support in the daemon for the workers to pick more than one job at a
time from the local queue, in case we decide this is worth doing. However the
Manager won't ever book out more jobs from the central Netdisco job queue than
it has workers available (so as not to hog jobs for itself against other
daemons on other servers). The user is free to configure the number of
workers in their C<config.yml> file (zero or more).
The fourth kind of worker is called the Scheduler and takes care of adding
discover, macsuck, arpnip, and nbtstat jobs to the queue (which are in turn
handled by the Poller worker). This worker is automatically started only if
the user has enabled the "C<schedule>" section of their
C<deployment.yml> site config.
=head2 SNMP::Info
The daemon obviously needs to use L<SNMP::Info> for device control. All the
code for this has been factored out into the L<App::Netdisco::Util> namespace.
The L<App::Netdisco::Util::SNMP> package provides for the creation of
SNMP::Info objects along with connection tests. So far, SNMPv3 is not
supported. To enable trace logging of the SNMP::Info object simply set the
C<INFO_TRACE> environment variable to a true value. The Connect library also
provides routines to map interface and PoE IDs.
Configuration for SNMP::Info comes from the YAML files, of course. This means
that our C<mibhome> and C<mibdirs> settings are now in YAML format. In
particular, the C<mibdirs> list is a real list within the configuration.
=head1 Other Noteable Technology
=head2 C<local::lib>
This is the system used to install Netdisco and all its Perl dependencies into
a folder independent of the system's Perl libraries. It means Netdisco can be
self-contaned and at the same time relocated anywhere. The L<local::lib>
module is responsible for re-setting Perl's environment to point at the new
library.
=head2 C<App::cpanminus>
This is simply a sane replacement for the CPAN shell. Don't ever bother with
the CPAN shell again, just use the L<cpanm> client which comes with this
distribution. We install Netdisco using C<cpanm>.
=head2 C<App::local::lib::helper>
This is a companion to C<local::lib> which provides the C<localenv> script you
see referenced in the documentation. It's run automatically by Netdisco to
locate its C<local::lib> folder (that is, works around the bootstrapping
problem where the shipped app doesn't know to where it is relocated). We can
help things along by setting the C<NETDISCO_HOME> environment variable.
=head2 C<Try::Tiny>
A replacement for C<eval> which provides proper C<try/catch> semantics. You
have to take a bit of care unfortunately over things like C<return> statements
though. However it's a lot cleaner than C<eval> in many cases. See the
L<documentation|Try::Tiny> for further details.
=head2 C<Role::Tiny>
Anyone familiar with the concept of an I<interface> from other programming
languages might understand what a role is. It's class functionality, often
also called a "trait", which is composed into a class at run-time. This module
allows the Daemon workers to dynamically assume different roles according to
configuration.
=cut

513
lib/App/Netdisco/Manual/ReleaseNotes.pod
View File

@@ -1,513 +0,0 @@
=head1 NAME
App::Netdisco::Manual::ReleaseNotes - Release Notes
=head1 Introduction
This document will list only the most significant changes with each release of
Netdisco. You are B<STRONGLY> recommended to read this document each time you
install and upgrade. Also see the Changes file, for more information.
=head1 Migrating from Netdisco 1.x
This distribution (App::Netdisco) is a complete rewrite of the Netdisco
application. Users often ask whether they can run both versions at the same
time, and whether the database must be copied. Here are the guidelines for
migrating from Netdisco 1.x:
=over 4
=item *
You can run both Netdisco 1.x and App::Netdisco web frontends at the same
time, using the same database (if "C<safe_password_store>" is set to
"C<false>" in the config file).
=item *
You can share a single database between Netdisco 1.x and App::Netdisco. The
deploy script for App::Netdisco will make some schema changes to the database,
but they are backwards compatible.
=item *
Only enable the backend daemon and discovery jobs from I<either> Netdisco 1.x
I<or> App::Netdisco.
=back
=head1 2.036002
This is a bug fix release since 2.036000. Please also read the 2.036000
release notes below, in full. Notable changes:
=over 4
=item *
Device Port report is much faster when displaying nodes
=item *
C<netdisco-do psql> now supports C<NETDISCO_DBNAME> environment variable
=item *
C<snmp_auth> configuration now supports C<only> and C<no> ACLs per stanza
=item *
New Duplicate Devices Report in case you get these appearing
=item *
Neighbor L2 topology map will show sysName if DNS is not available
=item *
Speed up ACLs featuring regualr expressions
=back
Plus lots more mentioned in the Changes file.
=head1 2.036000
This release has many significant new features and changes. Please read all
the release notes before upgrading.
=over 4
=item *
A new setting C<host_groups> allows for creating named Access Control Lists
which can be referred to in other host groups or in any of the settings taking
an ACL.
=item *
The new setting C<device_identity> allows configuring rules to select the
interface to use as a canonical (friendly) identity of a device in Netdisco.
=item *
The new settings C<devices_no> and C<devices_only> are shorthand for setting
C<discover_*>, C<macsuck_*>, C<arpnip_*>, and C<nbtstat_*> at once.
=item *
Netdisco now tracks SNMP connect failures and after 10 failed attempts will
pause trying to connect, for one week (see the C<max_deferrals> and
C<retry_after> settings). See also the "SNMP Connect Failures" admin report.
=item *
Documentation and support for access control lists has been overhauled. Most
"C<*_no>", "C<*_only>", and "C<only>" settings will accept ACLs as single
items or lists. ACLs now support negation and OR/AND modifier options.
=item *
A new setting C<site_local_files> is a shorthand for confguring paths in which
to install local Perl, template, javascript, and images files for overriding
or enhancing Netdisco.
=item *
The topology import script (C<nd-import-topology>) will now queue a "discover"
job for each new device it imports.
=item *
The C<netdisco-daemon> and C<netdisco-daemon-fg> scripts have
been renamed to C<netdisco-backend> and C<netdisco-backend-fg> respectively.
The old commands will still work but we recommend packagers to use the new
names to remain consistent with documentation. Run the following on upgrade:
ln -s ~/perl5/bin/{localenv,netdisco-*} ~/bin/
~/bin/netdisco-daemon stop
~/bin/netdisco-backend restart
=item *
SSL library headers are required to build Netdisco now that we retrieve
support files via HTTPS.
On Ubuntu/Debian:
root:~# apt-get install libssl-dev
On Fedora/Red-Hat:
root:~# yum install openssl-devel
On BSD these headers are usually installed with the openssl port itself.
Netdisco will otherwise fail to upgrade/install (it will fail building
L<IO::Socket::SSL> or L<Net::SSLeay>). If you get stuck or confused, you are
looking for the package including the file C<openssl/err.h>.
=head1 2.034000
This release changes the way the application tracks web sessions for logged-in
users, from on-disk files, to encrypted browser cookies. As a result, on
upgrade (after running C<netdisco-deploy> and restarting C<netdisco-web>), all
users will need to log in again.
There may be a pause after restarting C<netdisco-web> as old web session files
on disk are purged.
=head1 2.032003
The algorithm for selecting the canonical IP/name of a device has changed in
this release. No longer is the OSPF Router ID taken into account. The default
IP/name of a device will be either the IP specified for manual discovery, or
the IP reported to a neighbor port during automatic discovery. For the latter
you can often influence this through device configuration (LLDP advertise...).
=head1 2.032000
The identification of IP Phone hansets and Wireless APs is now configurable,
using the CDP/LLDP information from the device. See
L<documentation|App::Netdisco::Manual::Configuration> for:
phone_capabilities
phone_platforms
wap_capabilities
wap_platforms
=head1 2.031006
When displaying device ports, Netdisco will now avoid showing VLAN Membership
if it looks like there are a large number of VLANs on many ports. This is an
average of the VLANs per port, configurable in C<devport_vlan_limit>. The
default is 150.
=head1 2.031005
The C<netdisco-do> command's C<delete> option now uses the C<-p> parameter to
set node archive mode (previously it was a hack on C<-e>). For example:
~netdisco/bin/netdisco-do delete -d 192.0.2.1 -e 'older than the sun' -p yes
=head1 2.031003
=head2 Health Advice
This release will I<once again> remove from the database spurious Node
(workstation, printer, etc) entries on vlan 0, which were causing dupliate
entries in the web interface. We advise that you back up the database prior to
upgrade:
/usr/bin/pg_dump -F c --create -f netdisco-pgsql.dump netdisco
=head2 General Notices
The database schema can be fully redeployed (even over an existing
installation, in a safe way) using the following command:
~netdisco/bin/netdisco-db-deploy --redeploy-all
=head1 2.031002
Netdisco web and backend daemons will now rotate their log files
("C<~netdisco/logs/netdisco-{web,daemon}.log>"). This happens when they reach
about 10MB in size and seven historical log files will be maintained in the
same directory. The first time this happens you may notice the daemons
restarting due to having to deal with the large initial logfile.
Two missing features from Netdisco 1 have been implemented: CLI device delete
and renumber (canonical IP change). They are available using the
C<netdisco-do> utility.
The Device Port Log comment feature from 2.030000 has been disabled as it is
incomplete, pending a review of how to handle authorization to the feature.
=head1 2.029014
The node archiving behaviour of Netdisco 2 has until now been accidentally
different to that in Netdisco 1. This has now been fixed. See the new
"C<node_freshness>" configuration setting if you wish to revert or tune this
behaviour.
=head1 2.029010
When upgrading you will encounter a current incompatibility between Netdisco
and one of its components. To work around this, issue the following command:
~/bin/localenv cpanm --notest --force Dancer@1.3126 DBIx::Class@0.08270
=head1 2.029008
When upgrading you will encounter a current incompatibility between Netdisco
and one of its components. To work around this, issue the following command:
~/bin/localenv cpanm --notest --force Dancer@1.3126
=head1 2.029002
The backend polling daemon has been rewritten and as a result your
configuration can be simplified. Some keys have also been renamed. Our advice
is to remove (or comment out) the complete C<workers> configuration which
enables auto-tuning. If you do wish to control the number of worker
processes, follow this pattern:
workers:
tasks: 'AUTO * 2' # this is the default, twice the number of CPUs
=head1 2.029001
=head2 Health Advice
This release will remove from the database spurious Node (workstation,
printer, etc) entries on vlan 0, which were causing dupliate entries in the
web interface. We advise that you back up the database prior to upgrade:
/usr/bin/pg_dump -F c --create -f netdisco-pgsql.dump netdisco
=head2 General Notices
The configuration item C<reports> is now a list (used to be a dictionary).
Each item in the list must have a C<tag> entry which was previously the
dictionary key. For example, now use:
reports:
- tag: power_inventory
category: Device
label: 'Power Supply Inventory'
columns:
- {name: 'Name'}
- {ps1_type: 'PS1 Type'}
- {ps1_status: 'PS1 Status'}
query: |
SELECT d.name, d.ps1_type, d.ps1_status
FROM device d
WHERE d.ps1_type IS NOT NULL
ORDER BY name
Old configuration will be continue to work, but we recommend you reconfigure
anyway.
=head1 2.028000
=head2 Incompatible Changes
The daemons can be started from init scripts, as root. They will drop back
from the root user to C<netdisco> before opening logs. However a limitation is
that the web frontend might temporarily keep root status to bind to a specific
port (e.g. 80) - the logs will then be created as root user. Sorry about that.
You might also find when upgrading that previous logs were owned by root and
Netdisco now wants to write to them as non-root (C<netdisco>) user. Please
either remove the logs before restarting, or alter their ownership.
Logs can be found in the C<logs> subdirectory of Netdisco's home area.
=head2 General Notices
The configuration item C<housekeeping> has been renamed to C<schedule>. Old
configuration will continue to work, but we recommend you now rename this key
in your configuration anyway.
=head1 2.025001
The Web and Backend daemons (C<netdisco-web> and C<netdisco-daemon>
respectively) will now watch your C<deployment.yml> configuration file, and
restart themselves whenever it is changed.
The Web and Backend daemons will also now drop privilege to the same user and
group as their files on disk. This allows use of run control (init) scripts
whilst maintaining non-root privilege status (see
L<Deployment|App::Netdisco::Manual::Deployment> documentation for details).
The housekeeping task C<expiry> has been renamed to C<expire>. Old
configuration will continue to work, but we recommend you rename this part of
your C<housekeeping> configuration anyway.
=head1 2.023000
=head2 Incompatible Changes
This release will automatically migrate user passwords to have stronger
hashing in the database (a good thing!). This is incompatible with Netdisco
1.x web frontend, so if you must maintain backward-compatibility, set the
following in your C<deployment.yml> file:
safe_password_store: false
=head2 General Notices
The number of parallel DNS queries running during node discovery has been
reduced to 10 for maximum safety, but resulting in lower macsuck performance.
If you have a robust DNS infrastructure, you can probably put it back up to
something like 50 or 100:
dns:
max_outstanding: 100
=head1 2.021000
=head2 Incompatible Changes
SNMP community strings provided in the C<community_rw> configuration setting
will I<no longer> be used for I<read> actions on a device (despite having
"C<rw>" in the setting name).
If you have the same community string for read and write access, then you must
set both C<community> and C<community_rw> in your C<deployment.yml> file. In
any case, we recommend using the new C<snmp_auth> configuration format which
supercedes both these settings.
=head2 Health Advice
This release includes support for Device and Node expiry from your database.
This is an important part of housekeeping for your installation, and our
recommendation is to enable this feature such that suitably old Devices and
Nodes are expired nightly.
Add the following to your "C<housekeeping>" configuration in
C<deployment.yml>, to have a nightly check at 11:20pm:
housekeeping:
expire:
when: '20 23 * * *'
You should also configure one or more of C<expire_devices>, C<expire_nodes>,
and C<expire_nodes_archive> to a number of days. See the
L<Configuration|App::Netdisco::Manual::Configuration> documentation for
further details.
=head2 General Notices
If you use an Apache reverse proxy, we recomment increasing the timeout from
our previous example of 5 seconds to, perhaps 60. This is because some reports
do take more time to run their queries on the database. See
L<Deployment|App::Netdisco::Manual::Deployment> documentation for details.
=head1 2.020000
If you were using the C<X::Observium> plugin, you'll now need to install
the separate distribution L<App::NetdiscoX::Web::Plugin::Observium>.
=head1 2.019000
This release fixes a number of issues with the poller, and is a recommended
upgrade.
During Arpnip, Node IPs are resolved to DNS names in parallel. See the C<dns>
configuration option for details. Note that the C<nodenames> configuration
items from release C<2.018000> are no longer available.
This release includes new support for SNMPv3 via the C<snmp_auth>
configuration option. Please provide feedback to the developers on your
experience.
=head1 2.018000
The previous mentioned bug in Macsuck is now fixed.
=head1 2.017000
There is a bug in Macsuck whereby in rare circumstances some invalid SQL is
generated. The root cause is known but we want to take more time to get the
fix right. It should only be a few more days.
The C<no_port_control> configuration setting is now called C<check_userlog>
and its logic is inverted. Don't worry if this is not familiar to you - the
option is only used by Netdisco Developers.
=head1 2.016000
The dangerous action log messages are now saved to the database. In a future
version there will be a way to display them in the web interface.
=head1 2.015000
Some of the "dangerous action" confirmation dialogs offer to take a log
message (e.g. Port Control, Device Delete). Currently the log messages are
B<not saved>. This feature will be added in the next release.
=head1 2.014000
The backend poller daemon is now considered stable. You can uncomment the
C<housekeeping> section of the example configuration and thereby enable
regular device (re-)discovery, arpnip and macsuck.
=head1 2.013000
You can now configure LDAP authentication for users.
=head1 2.012000
The read-write SNMP community is now stored in the database, when used for the
first time on a device. If you don't want the web frontend to be able to
access this, you need to:
=over 4
=item *
Have separate C<deployment.yml> files for web frontend and daemon, such that
only the daemon config contains any community strings.
=item *
Use separate PostgreSQL users for web frontend and daemon, such that the web
frontend user cannot SELECT from the C<community> DB table.
=back
=head1 2.011000
Users can be managed through the web interface (by admins only).
=head1 2.010000
You can now simplify database configuration to just the following, instead of
the more verbose C<plugins/DBIC> setting which was there before:
database:
name: 'netdisco'
host: 'localhost'
user: 'someuser'
pass: 'somepass'
Also, the C<REMOTE_USER> environment variable and C<X-REMOTE_USER> HTTP Header
are now supported for delegating authentication to another web server. See the
Deployment and Configuration documentation for further details.
=head1 2.008000
=head2 Health Advice
This release contains the first version of our new poller, which handles
device and node discovery. Please make sure to backup any existing Netdisco
database before trying it out.
=head2 General Notices
You can remove any settings from C<~/environments/deployment.yml> which you
didn't edit or add to the file yourself. All defaults are now properly
embedded within the application. See the new C<deployment.yml> sample which
ships with this distribution for an example.
=head1 2.006000
=head2 Incompatible Changes
The default environment configuration file C<develpment.yml> has been renamed
to C<deployment.yml>. This better reflects that users are not developers, and
also fits with the default for PSGI compatible cloud deployment services.
Please B<rename or copy> your environment file:
mv ~/environments/development.yml ~/environments/deployment.yml
=head2 General Notices
The installation is now relocateable outside of a user's home directory by
setting the C<NETDISCO_HOME> environment variable. This defaults to your own
home directory.
=cut

70
lib/App/Netdisco/Manual/Systemd.pod
View File

@@ -1,70 +0,0 @@
=head1 NAME
App::Netdisco::Manual::Systemd - Systemd install tips
=head1 Introduction
This page documents Netdisco running under systemd. Thanks to Aurelien Guerson
and Stuart Kendrick for offering this solution. Please check these
instructions apply to your local installation and use at your own risk.
=head1 Files
=head2 C<< /etc/systemd/system/netdisco-backend.service >>
This should be set mode 644 and owned by user and group C<root>.
[Unit]
Description=Netdisco Backend Service
AssertFileIsExecutable=/home/netdisco/bin/netdisco-backend
After=syslog.target network-online.target
[Service]
Type=forking
User=netdisco
Group=netdisco
ExecStart=/home/netdisco/bin/netdisco-backend start
ExecStop=/home/netdisco/bin/netdisco-backend stop
Restart=on-failure
RestartSec=60
[Install]
WantedBy=multi-user.target
=head2 C<< /etc/systemd/system/netdisco-web.service >>
This should be set mode 644 and owned by user and group C<root>.
[Unit]
Description=Netdisco Web Service
AssertFileIsExecutable=/home/netdisco/bin/netdisco-web
After=syslog.target network-online.target netdisco-backend.service
[Service]
Type=forking
User=netdisco
Group=netdisco
ExecStart=/home/netdisco/bin/netdisco-web start
ExecStop=/home/netdisco/bin/netdisco-web stop
Restart=on-failure
RestartSec=60
[Install]
WantedBy=multi-user.target
=head1 Commands
To enable Netdisco in systemd:
systemctl enable netdisco-backend.service
systemctl enable netdisco-web.service
To start Netdisco:
systemctl start netdisco-backend.service
systemctl start netdisco-web.service
May also need to run C<systemctl netdisco-backend reload> depending on the
order you do these steps.
=cut

189
lib/App/Netdisco/Manual/Troubleshooting.pod
View File

@@ -1,189 +0,0 @@
=head1 NAME
App::Netdisco::Manual::Troubleshooting - Tips and Tricks for Troubleshooting
=head1 Understanding Nodes and Devices
The two basic components in Netdisco's world are Nodes and Devices. Devices
are your network hardware, such as routers, switches, and firewalls. Nodes are
the end-stations connected to Devices, such as workstations, servers,
printers, and telephones.
Devices respond to SNMP, and therefore can report useful information about
themselves such as interfaces, operating system, IP addresses, as well as
knowledge of other systems via MAC address and ARP tables. Devices are
actively contacted by Netdisco during a discover (and other polling jobs such
as macsuck, arpnip).
Netdisco discovers Devices using "neighbor protocols" such as CDP and LLDP. We
assume your Devices are running these protocols and learning about their
connections to each other. If they aren't, you'll need to configure manual
topology within the web interface (or simply have standalone Devices).
Nodes, on the other hand, are passive as far as Netdisco is concerned. The
only job that contacts a Node is nbtstat, which makes NetBIOS queries. Nodes
are learned about via the MAC and ARP tables on upstream Devices.
Because Netdisco only learns about Devices through a neighbor protocol, it's
possible to run an SNMP agent on a Node. Only if the Node is also advertising
itself via a neighbor protocol will Netdisco treat it as a Device. This can
account for undesired behaviour, such as treating a server (Node) as a Device,
or vice versa only recognising a switch (Device) as a Node.
To prevent discovery of devices, use the C<devices_no> configuration setting.
If you don't see links between Devices in Netdisco, it might be because
they're not running a neighbor protocol, or for some reason not reporting the
relationships to Netdisco. Use the C<show> command to troubleshoot this:
~netdisco/bin/netdisco-do show -d 192.0.2.1 -e c_id
=head1 Understanding Netdisco Jobs
Please read the section above, if you've not yet done so.
Netdisco has four principal job types:
=over 4
=item discover
Gather information about a Device, including interfaces, vlans, PoE status,
and chassis components (modules). Also learns about potential new Devices via
neighbor protocols and adds jobs for their discovery to the queue.
=item macsuck
Gather MAC to port mappings from known Devices reporting Layer 2 capability.
Wireless client information is also gathered from Devices supporting the
802.11 MIBs.
=item arpnip
Gather MAC to IP mappings from known Devices reporting layer 3 capability.
=item nbtstat
Poll a Node to obtain its NetBIOS name.
=back
The actions as named above will operate on one device only. Complimentary job
types C<discoverall>, C<macwalk>, C<arpwalk>, and C<nbtwalk> will enqueue one
corresponding single-device job for each known device. The Netdisco backend
daemon will then process the queue (in a random order).
=head1 My Device details look all wrong!
See the tips at L<Vendors Guide|App::Netdisco::Manual::Vendors>, or else
contact the L<community email
list|https://lists.sourceforge.net/lists/listinfo/netdisco-users>.
=head1 Devices are not being discovered
Besides reading the whole of this manual page for general tips, take a look at
the "SNMP Connect Failures" report under the Admin menu. Any devices listed
have had multiple SNMP connect failures, indicating a possible configuration
error on the device or in Netdisco's configuration.
=head1 Devices have the wrong names
Netdisco uses neighbor protocols to discover devices and will use as the
default identity for a device the interface IP advertised over those neighbor
protocols. You can use the C<device_identity> configuration setting to steer
Netdisco towards using a different interface for the canonical device name.
=head1 After OS update or upgrade, Netdisco fails
If you upgrade the operating system then your system libraries will change and
Netdisco needs to be rebuilt (specifically, C library bindings).
The safest way to do this is set up a new user and follow the same install
instructions, connecting to the same database. Stop the web and backend daemon
for the old user, and start them for the new user. Then delete the old user
account.
Alternatively, if you do not mind the downtime: stop the web and backend
daemons then delete the C<~netdisco/perl5> directory and reinstall from
scratch. The configuration file, database, and MIBs can all be reused
in-place.
=head1 Run a C<netdisco-do> Task with Debugging
The C<netdisco-do> command has several debug flags which will show what's
going on internally. Usually you always add C<-D> for general Netdisco
debugging, then C<-I> for L<SNMP::Info> logging and C<-Q> for SQL tracing. For
example:
~netdisco/bin/netdisco-do discover -d 192.0.2.1 -DIQ
You will see that SNMP community strings and users are hidden by default, to
make the output safe for sending to Netdisco developers. To show the community
string and SNMPv3 protocols, set the C<SHOW_COMMUNITY> environment variable:
SHOW_COMMUNITY=1 ~netdisco/bin/netdisco-do discover -d 192.0.2.1 -DIQ
=head1 Dump an SNMP object for a Device
This is useful when trying to work out why some information isn't displaying
correctly (or at all) in Netdisco. It may be that the SNMP response isn't
understood. Netdisco can dump any leaf or table, by name:
~netdisco/bin/netdisco-do show -d 192.0.2.1 -e interfaces
~netdisco/bin/netdisco-do show -d 192.0.2.1 -e Layer2::HP::interfaces
You can combine this with SNMP::Info debugging, shown above (C<-I>).
=head1 Interactive SQL terminal on the Netdisco Database
Start an interactive terminal with the Netdisco PostgreSQL database. If you
pass an SQL statement in the "-e" option then it will be executed.
~netdisco/bin/netdisco-do psql
~netdisco/bin/netdisco-do psql -e 'SELECT ip, dns FROM device'
~netdisco/bin/netdisco-do psql -e 'COPY (SELECT ip, dns FROM device) TO STDOUT WITH CSV HEADER'
The last example above is useful for sending data to Netdisco developers, as
it's more compact and readable than the standard tabular output (second
example).
=head1 Database Schema Redeployment
The database schema can be fully redeployed (even over an existing
installation), in a safe way, using the following command:
~netdisco/bin/netdisco-db-deploy --redeploy-all
=head1 Debug HTTP Requests and Configuration
You can see HTTP Headers received by Netdisco, and other information such as
how it's parsing the config file, by enabling the Dancer debug plugin. First
download the plugin:
~netdisco/bin/localenv cpanm --notest Dancer::Debug
Then run the web daemon with the environment variable to enable the feature:
DANCER_DEBUG=1 ~/bin/netdisco-web restart
A side panel appears in the web page with debug information. Be sure to turn
this off when you're done (stop and start without the environment variable)
otherwise secrets could be leaked to end users.
=head1 Change the SNMP commnuity string for a Device
If you change the SNMP community string in use on a Device, and update
Netdisco's configuration to match, then everything will continue to work fine.
However, if the Device happens to support two community strings then Netdisco
can become "stuck" on the wrong one, as it caches the last-known-good
community string to improve performance. To work around this, delete the
device (either in the web GUI or using C<netdisco-do> at the command line),
and then re-discover it.
=head1 Installation on SLES 11 SP4
Try running the following command for installation:
curl -L http://cpanmin.us/ | CFLAGS="-DPERL_ARGS_ASSERT_CROAK_XS_USAGE" perl - --notest --local-lib ~/perl5 App::Netdisco
=cut

130
lib/App/Netdisco/Manual/Vendors.pod
View File

@@ -1,130 +0,0 @@
=head1 NAME
App::Netdisco::Manual::Vendors - Tips and Tricks for Vendor Platforms
=head1 Neighbor Relations on Juniper EX
The LLDP configuration should look like:
lldp {
management-address 10.0.0.1;
port-id-subtype interface-name;
interface all;
}
=head1 Neighbor Relations on D-Link
Add the following to your devices (changing the port numbers appropriately):
config lldp ports 1-28 mgt_addr ipv4 enable
or
config lldp ports 1-28 mgt_addr ipv4 1.2.3.4 enable
Which you use will depend on the device OS version.
=head1 VRFs and NXOS
Netdsico at this time does not support VRFs. In particular, overlapping IP
address spaces will not be shown in the interface.
However if you're running Cisco NXOS and do not have overlapping IP address
space, then you can use the NXOS SSHCollector profile for that platform.
=head1 Report Cisco as Single Device Instead of Stacked (37xx/29xx/etc)
Add this to your device config:
no snmp-server sysobjectid type stack-oid
=head1 SNMP Support on Huawei Quidway and CloudEngine
Where C<mycommunity> is your community string. Note C<iso> means I<everything>
is visible to readers!
snmp-agent mib-view included all iso
snmp-agent community read cipher mycommunity mib-view all
snmp-agent packet max-size 17940
snmp-agent extend error-code enable
=head1 SNMP Support on Linksys and Cisco Linksys
Where C<mycommunity> is your community string. Note this results in I<everything>
being visible to readers!
snmp-server view test iso included
snmp-server view test system included
snmp-server view test interfaces included
snmp-server view test ip included
snmp-server view test icmp included
snmp-server view test tcp included
snmp-server view test udp included
snmp-server view test transmission included
snmp-server view test snmp included
snmp-server view test rmon included
snmp-server view test dot1dBridge included
snmp-server view test ifMIB included
snmp-server view test dns included
snmp-server view test radiusMIB included
snmp-server view test traceRouteMIB included
snmp-server view test powerEthernetMIB included
snmp-server community mycommunity ro view test
=head1 SNMPv3 Support on Cisco IOS
To access per-VLAN MAC address tables we use SNMPv3 contexts. In Cisco IOS
the access control is per-context so for each context (VLAN) you need to permit
access from the poller.
You should already have something like the following to enable SNMPv3 from Netdisco at 192.0.2.1:
snmp-server view myv3view iso included
snmp-server group myv3group v3 priv read myv3view
snmp-server user myv3user myv3group v3 auth md5 PASSWORD priv des PASSWORD
snmp-server host 192.0.2.1 version 3 auth myv3user
Then set the authorization:
snmp-server group myv3group v3 auth
snmp-server group myv3group v3 auth context vlan- match prefix
If the second command above is rejected, you have an older version of IOS and must
enter a statement for each active VLAN on the device:
snmp-server group myv3group v3 priv context vlan-1
snmp-server group myv3group v3 priv context vlan-2
snmp-server group myv3group v3 priv context vlan-3
... etc
=head1 Linux SNMP Service (Agent)
Install the C<snmpd> (SNMP agent) and C<lldpd> (neighbor discovery) packages.
Edit the C</etc/snmp/snmpd.conf> file:
# AGENT BEHAVIOUR
# comment out: agentAddress udp:127.0.0.1:161
agentAddress udp:161,udp6:[::1]:161
# ACCESS CONTROL
rocommunity <your-secret> <management-device-IP/net>
# SYSTEM INFORMATION
sysServices 76
# (default is 72, 74 is layer2 bridge/switch, 76 for layer3 router/gateway)
If running a firewall, allow SNMP traffic in on UDP port 161.
Edit the C</etc/default/lldpd> file:
DAEMON_ARGS="-k -x -l -m <Mgmt-IP>"
# <Mgmt-IP> is the IP to advertise for Netdisco to connect
Restart C<snmpd> and C<lldpd> services when you have configured them.
This assumes you're using LLDP on your network. If you use CDP then the
C<lldpd> daemon can support that protocol - see the manual page for details.
=cut

521
lib/App/Netdisco/Manual/WritingWebPlugins.pod
View File

@@ -1,521 +0,0 @@
=head1 NAME
App::Netdisco::Manual::WritingWebPlugins - Documentation on Web Plugins for Developers
=head1 Introduction
L<App::Netdisco>'s plugin subsystem allows developers to write and test web
user interface (UI) components without needing to patch the main Netdisco
application. It also allows the end-user more control over the UI components
displayed in their browser.
See L<App::Netdisco::Web::Plugin> for more general information about plugins.
=head1 Developing Plugins
A plugin is simply a Perl module which is loaded. Therefore it can do anything
you like, but most usefully for the App::Netdisco web application the module
will install a L<Dancer> route handler subroutine, and link this to a web user
interface (UI) component.
Explaining how to write Dancer route handlers is beyond the scope of this
document, but by examining the source to the plugins in App::Netdisco you'll
probably get enough of an idea to begin on your own.
App::Netdisco plugins should load the L<App::Netdisco::Web::Plugin> module.
This exports a set of helper subroutines to register the new UI components.
Here's the boilerplate code for our example plugin module:
package App::Netdisco::Web::Plugin::MyNewFeature;
use Dancer ':syntax';
use Dancer::Plugin::DBIC;
use Dancer::Plugin::Auth::Extensible;
use App::Netdisco::Web::Plugin;
# plugin registration code goes here, ** see below **
# your Dancer route handler
get '/mynewfeature' => require_login sub {
# ...lorem ipsum...
};
true;
=head1 Navigation Bar items
These components appear in the black navigation bar at the top of each page,
as individual items (i.e. not in a menu). The canonical example of this is the
Inventory link.
To register an item for display in the navigation bar, use the following code:
register_navbar_item({
tag => 'newfeature',
path => '/mynewfeature',
label => 'My New Feature',
});
This causes an item to appear in the Navigation Bar with a visible text of "My
New Feature" which when clicked sends the user to the C</mynewfeature> page.
Note that this won't work for any target link - the path must be an
App::Netdisco Dancer route handler. Please bug the App::Netdisco devs if you
want arbitrary links supported.
=head1 Search and Device page Tabs
These components appear as tabs in the interface when the user reaches the
Search page or Device details page. Note that Tab plugins usually live in
the C<App::Netdisco::Web::Plugin::Device> or
C<App::Netdisco::Web::Plugin::Search> namespace.
To register a handler for display as a Search page Tab, use the following
code:
register_search_tab({tag => 'newfeature', label => 'My New Feature'});
This causes a tab to appear with the label "My New Feature". So how does
App::Netdisco know what the link should be? Well, as the
L<App::Netdisco::Developing> documentation says, tab content is retrieved by
an AJAX call back to the web server. This uses a predictable URL path format:
/ajax/content/<search or device>/<feature tag>
For example:
/ajax/content/search/newfeature
Therefore your plugin module should look like the following:
package App::Netdisco::Web::Plugin::Search::MyNewFeature
use Dancer ':syntax';
use Dancer::Plugin::DBIC;
use Dancer::Plugin::Auth::Extensible;
use App::Netdisco::Web::Plugin;
register_search_tab({tag => 'newfeature', label => 'My New Feature'});
get '/ajax/content/search/newfeature' => require_login sub {
# ...lorem ipsum...
# return some HTML content here, probably using a template
};
true;
If this all sounds a bit daunting, take a look at the
L<App::Netdisco::Web::Plugin::Search::Port> module which is fairly
straightforward.
To register a handler for display as a Device page Tab, the only difference is
the name of the registration helper sub:
register_device_tab({tag => 'newfeature', label => 'My New Feature'});
=head1 Reports
Report components contain pre-canned searches which the user community have
found to be useful. Before you go further, it might be the case that Netdisco
can generate the report for you without any Perl or HTML: see the L<Reports
Configuration|App::Netdisco::Manual::Configuration/reports> for details.
Otherwise, the typical implementation is very similar to one of the Search
and Device page Tabs, so please read that documentation above, first.
Report plugins usually live in the C<App::Netdisco::Web::Plugin::Report>
namespace. To register a handler for display as a Report, you need to pick the
I<category> of the report. Here are the pre-defined categories:
=over 4
=item *
Device
=item *
Port
=item *
IP
=item *
Node
=item *
VLAN
=item *
Network
=item *
Wireless
=back
Once your category is selected, use the following registration code:
register_report({
category => 'Port', # pick one from the list
tag => 'newreport',
label => 'My New Report',
});
You will note that like Device and Search page Tabs, there's no path
specified in the registration. The reports engine will make an AJAX request to
the following URL:
/ajax/content/report/<report tag>
Therefore you should implement in your plugin a route handler for this path.
The handler must return the HTML content for the report. It can also process
any query parameters which might customize the report search.
See the L<App::Netdisco::Web::Plugin::Report::DuplexMismatch> module for a
simple example of how to implement the handler.
An additional feature allows you to create Reports which do not appear in the
Navbar menu. This is useful if the page is only linked directly from another
(for example Port Log). To enable this feature add the C<hidden> key:
register_report({
tag => 'newfeature',
label => 'My New Feature',
hidden => true,
});
=head1 CSV Response
Most pages in Netdisco are a table with data. It's possible to have the
application add a link to download a CSV version of the same data. To do this,
include the following option in your call to C<register_search_tab>,
C<register_device_tab>, or C<register_report>:
provides_csv => 1
The other thing you need to do is adjust your Dancer route handler to return
either HTML or CSV data. Here's the typical way to do it:
get '/ajax/content/search/newfeature' => require_login sub {
# build some kind of dataset here (e.g. a DBIx::Class query)
if (request->is_ajax) {
template 'mytemplate', { data => $mydataset }, { layout => undef };
}
else {
header( 'Content-Type' => 'text/comma-separated-values' );
template 'mytemplate_csv', { data => $mydataset }, { layout => undef };
}
};
Note that the C<is_ajax> call is part of the standard Dancer featureset.
=head1 Admin Tasks
These components appear in the black navigation bar under an Admin menu, but only
if the logged in user has Administrator rights in Netdisco.
To register an item for display in the Admin menu, use the following code:
register_admin_task({
tag => 'newfeature',
label => 'My New Feature',
});
This causes an item to appear in the Admin menu with a visible text of "My New
Feature" which when clicked sends the user to the C</admin/mynewfeature> page.
Note that this won't work for any target link - the path must be an
App::Netdisco Dancer route handler. Please bug the App::Netdisco devs if you
want arbitrary links supported.
An additional feature allows you to create Admin Tasks which do not appear in
the Navbar menu. This is useful if the page is only linked directly from
another. To enable this feature add the C<hidden> key:
register_admin_task({
tag => 'newfeature',
label => 'My New Feature',
hidden => true,
});
=head1 Device Port Columns
You can also add columns to the Device Ports page. The canonical example of
this is to add hyperlinks (or embedded images) of traffic graphs for each
port, however the plugin is a regular Template::Toolkit template so can be any
HTML output.
The column plugin has a name (used internally to locate files on disk), label
(the heading for the column in the table or CSV output), position in the table
(three options: left, mid, right), and finally a flag for whether the column
is displayed by default.
To register the column call the following helper routine:
register_device_port_column({
name => 'myportcolumnplugin',
label => 'My Port Column Heading',
position => 'left', # or "mid" or "right"
default => 'on', # or undef
});
App::Netdisco searches for Template::Toolkit files in the regular template
include paths: either its internal locations, or those configured with the
C<site_local_files> setting or the C<register_template_path> helper (see
below). The template must be called "C<device_port_column.tt>" on disk and
live in the directory:
plugin/myportcolumnplugin/device_port_column.tt
For a good example of this, see the L<App::NetdiscoX::Web::Plugin::Observium>
distribution.
=head1 Device Details
You can add items to the Device Details tab as well. A good example of this is
to add a link to the RANCID backup of the device in a WebSVN app somewhere.
Like Device Port Columns plugins, the plugin is a regular Template::Toolkit
snippet so can be any HTML output.
The details plugin has a name (used internally to locate files on disk) and
label (the heading for the row in the table).
To register the column call the following helper routine:
register_device_details({
name => 'mydevicedetailsplugin',
label => 'My Device Details Heading',
});
App::Netdisco searches for Template::Toolkit files in the regular template
include paths: either its internal locations, or those configured with the
C<site_local_files> setting or the C<register_template_path> helper (see
below). The template must be called "C<device_port_column.tt>" on disk and
live in the directory:
plugin/mydevicedetailsplugin/device_details.tt
For a good example of this, see the L<App::NetdiscoX::Web::Plugin::RANCID>
distribution.
=head1 User Authorization
All Dancer route handlers must have proper authorization configured. This is
not difficult. Make sure that your module loads the
L<Dancer::Plugin::Auth::Extensible> module (as shown above).
For each route handler you either simply require that a user be logged in, or
that the user is an administrator.
To require a logged in user, include the C<require_login> wrapper:
get '/ajax/content/search/newfeature' => require_login sub {
# etc .....
To require an administrator, specify their role:
get '/ajax/control/admin/newfeature' => require_role admin => sub {
# etc .....
Finally in case you need it, the other role a user can have is
C<port_control>:
ajax '/ajax/portcontrol' => require_role port_control => sub {
# etc .....
Take care over the subtle differences in syntax, especially the placement of
the fat comma ("C<< => >>").
=head1 Database Connections
The Netdisco database is available via the C<netdisco> schema key, as below.
You can also use the C<external_databases> configuration item to set up
connections to other databases.
# possibly install another database driver
~netdisco/bin/localenv cpanm --notest DBD::mysql
# deployment.yml
external_databases:
- tag: externaldb
dsn: 'dbi:mysql:dbname=myexternaldb;host=192.0.2.1'
user: oliver
password: letmein
# plugin code
use Dancer::Plugin::DBIC;
schema('netdisco')->resultset('Devices')->search({vendor => 'cisco'});
schema('externaldb')->resultset('MyTable')->search({field => 'foobar'});
You'll need to install a L<DBIx::Class> Schema and also set C<schema_class> to
its name within the C<external_databases> setting, for the second example
above.
=head1 Templates
All of Netdisco's web page templates are stashed away in its distribution,
probably installed in your system's or user's Perl directory. It's not
recommended that you mess about with those files.
So in order to replace a template with your own version, or to reference a
template file of your own in your plugin, you need a new path.
If you don't plan on redistributing the plugin via CPAN, then configuring the
"C<site_local_files>" setting to be true will enable "C</nd-site-local/lib>"
for Perl code and "C</nd-site-local/share>" for tmplates in your Netdisco home
location. You will need to create these directories.
Alternatively, shipping templates within a CPAN distribution, the following
code would be appropriate:
package App::Netdisco::Web::Plugin::Search::MyNewFeature
use File::ShareDir 'dist_dir';
register_template_path(
dist_dir( 'App-Netdisco-Web-Plugin-Search-MyNewFeature' ));
The "C<views>" subdirectory of the registered path will be searched before the
built-in C<App::Netdisco> path. We recommend use of the L<File::ShareDir>
module to package and ship templates along with your plugin, as shown.
Each path added using C<register_template_path> is searched I<before> any
existing paths in the template config. See the
L<App::NetdiscoX::Web::Plugin::Observium> distribution for a working example.
=head2 Template Variables
Some useful variables are made available in your templates automatically by
App::Netdisco:
=over 4
=item C<search_node>
A path and query string which links to the Node tab of the Search page,
together with the correct default search options set.
=item C<search_device>
A path and query string which links to the Device tab of the Search page,
together with the correct default search options set.
=item C<device_ports>
A path and query sting which links to the Ports tab of the Device page,
together with the correct default column view options set.
=item C<uri_base>
Used for linking to static content within App::Netdisco safely if the base of
the app is relocated, for example:
<link rel="stylesheet" href="[% uri_base %]/css/toastr.css"/>
=item C<uri_for>
Simply the Dancer C<uri_for> method. Allows you to do things like this in the
template safely if the base of the app is relocated:
<a href="[% uri_for('/search') %]" ...>
=item C<self_options>
Available in the Device tabs, use this if you need to refer back to the
current page with some additional parameters, for example:
<a href="[% uri_for('/device', self_options) %]&foo=bar" ...>
=back
=head1 Javascript and Stylesheets
A simple mechanism exists for loading additional Javascript and CSS documents.
This is done in the C<< <head> >> section of the web page.
Netdisco searches all template include paths, both those built into the
application and those configured in your plugin(s) with "C<site_local_files>"
or C<register_template_path>.
Within the template location, create a directory called "C<plugin>" and within
that another directory named after your plugin (e.g. "C<mynewfeature>"). The
Javascript and/or CSS files must then be named "C<mynewfeature.js>" and
"C<mynewfeature.css>" respectively. For example:
plugin/mynewfeature/mynewfeature.js
plugin/mynewfeature/mynewfeature.css
Tell App::Netdisco that you wish to load one or the other using the following
helper routines:
register_javascript('mynewfeature');
register_css('mynewfeature');
=head1 Naming and File Location
There are several options for how you name, distribute and install your
App::Netdisco plugin.
=head2 Namespaces
As mentioned in L<App::Netdisco::Web::Plugin>, official Netdisco plugins live
in the C<App::Netdisco::Web::Plugin::> namespace. You can use this namespace
and submit the product to the Netdisco developer team for consideration for
inclusion in the official distribution.
Alternatively you can release the plugin to CPAN under your own account. In
that case we request that you instead use the C<App::NetdiscoX::Web::Plugin::>
namespace (note the "X"). Users can load such modules by using the
abbreviated form "X::MyPluginName" which is then expanded to the full package.
=head2 File Location
If writing your own plugins that are not for redistribution or packaging on
CPAN, Netdisco can enable local include paths for Perl, templates, and static
content such as javascript and images.
Configuring the "C<site_local_files>" to be "true" enables:
# perl code
$ENV{NETDISCO_HOME}/nd-site-local/lib
# templates and static content
$ENV{NETDISCO_HOME}/nd-site-local/share
Note that you still need to create the directories yourself, and templates may
need to have a further "C<views>" subdirectory created within "C<share>".
As an example, if your plugin is called
"App::NetdiscoX::Web::Plugin::MyPluginName" then it could live at:
~netdisco/nd-site-local/lib/App/NetdiscoX/Web/Plugin/MyPluginName.pm
=head1 Plugin Configuration
You can support new configuration items which the user should add to their
C<~/environments/deployment.yml> file. Please use a single Hash-Ref option
named "C<plugin_mypluginname>" (if your plugin's called C<mypluginname>). For
example:
plugin_observium:
webhost: "web-server.example.com"
open_in_new_window: true
You can then refer to this configuration in your plugin module:
my $webhost = setting('plugin_observium')->{'webhost'};
Or in templates via Dancer's C<settings> key:
<a href="http://[% settings.plugin_observium.webhost | uri %]/>Observium</a>
=cut

226
lib/App/Netdisco/Manual/WritingWorkers.pod
View File

@@ -1,226 +0,0 @@
=head1 NAME
App::Netdisco::Manual::WritingWorkers - Developer Documentation on Worker Plugins
=head1 Introduction
L<App::Netdisco>'s plugin system allows users to write I<workers> to gather
information from network devices using different I<transports> and store
results in the database.
For example, transports might be SNMP, SSH, or HTTPS. Workers might be
combining those transports with application protocols such as SNMP, NETCONF
(OpenConfig with XML), RESTCONF (OpenConfig with JSON), eAPI, or even CLI
scraping. The combination of transport and protocol is known as a I<driver>.
Workers can be restricted to certain vendor platforms using familiar ACL
syntax. They are also attached to specific actions in Netdisco's backend
operation (discover, macsuck, etc).
See L<App::Netdisco::Worker::Plugin> for more information about worker
plugins.
=head1 Developing Workers
A worker is Perl code which is run. Therefore it can do anything you like, but
typically it will make a connection to a device, gather some data, and store
it in Netdisco's database.
App::Netdisco plugins must load the L<App::Netdisco::Worker::Plugin> module.
This exports a helper subroutine to register the worker. Here's the
boilerplate code for our example plugin module:
package App::Netdisco::Worker::Plugin::Discover::Wireless::UniFi;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
# worker registration code goes here, ** see below **
true;
=head1 Registering a Worker
Use the C<register_worker> helper from L<App::Netdisco::Worker::Plugin> to
register a worker:
register_worker( $coderef );
# or
register_worker( \%workerconf, $coderef );
For example (using the second form):
register_worker({
driver => 'unifiapi',
}, sub { "worker code here" });
The C<%workerconf> hashref is optional, and described below. The C<$coderef>
is the main body of your worker. Your worker is run in a L<Try::Tiny>
statement to catch errors, and passed the following arguments:
$coderef->($job, \%workerconf);
The C<$job> is an instance of L<App::Netdisco::Backend::Job>. Note that this
class has a C<device> slot which may be filled, depending on the action, and
if the device is not yet discovered then the row will not yet be in storage.
The C<\%workerconf> hashref is the set of configuration parameters you used
to declare the worker (documented below).
=head2 Package Naming Convention
The package name used where the worker is declared is significant. Let's look
at the boilerplate example again:
package App::Netdisco::Worker::Plugin::Discover::Wireless::UniFi;
The package name B<must> contain C<Plugin::> and the namespace component after
that becomes the action. For example workers registered in the above package
will be run during the I<discover> backend action (that is, during a
C<discover> job). You can replace C<Discover> with other actions such as
C<Macsuck>, C<Arpnip>, C<Expire>, and C<Nbtstat>, or create your own.
The component after the action is known as the I<phase> (C<Wireless> in this
example), and is the way to override a Netdisco built-in worker, by using the
same name (plus an entry in C<%workerconf>, see below). Otherwise you can use
any valid Perl bareword for the phase.
Workers may also be registered directly to the action (C<Discover>, in this
example), without any phase. This is used for very early bootstrapping code
(such as first inserting a device into the database so it can be used by
subsequent phases) or for very simple, generic actions (such as C<netdisco-do
psql>).
=head2 C<%workerconf> Options
=over 4
=item ACL Options
Workers may have C<only> and C<no> parameters configured which use the
standard ACL syntax described in L<the settings
guide|App::Netdisco::Manual::Configuration>. The C<only> directive is
especially useful as it can restrict a worker to a given device platform or
operating system (for example Cisco IOS XR for the C<restconf> driver).
=item C<driver> (string)
The driver is a label associated with a group of workers and typically refers
to the combination of transport and application protocol. Examples include
C<snmp>, C<netconf>, C<restconf>, C<eapi>, and C<cli>. The convention is for
driver names to be lowercase.
Users will bind authentication configuration settings to drivers in their
configuration. If no driver is specified when registering a worker, it will be
run for every device and phase (such as during Expire jobs).
=item C<primary> (boolean)
When multiple workers are registered for the same phase, they will all be run.
However there is a special "I<primary>" slot for each phase in which only one
worker (the first that succeeds) is used. Most of Netdisco's built-in worker
code is registered in this way, so to override it you can use the same package
namespace and set C<primary> to be C<true>.
=back
=head1 Worker Execution and Return Code
Workers are configured as an ordered list. They are grouped by C<action> and
C<phase> (as in Package Naming Convention, above).
Workers defined in C<extra_worker_plugins> are run before those in
C<worker_plugins> so you have an opportunity to override built-in workers by
adding them to C<extra_worker_plugins> and setting C<primary> to C<true> in
the worker configuration.
The return code of the worker is significant for those configured with
C<primary> as C<true>: when the worker returns true, no other C<primary> hooks
are run for that phase. You should always use the aliased
L<App::Netdisco::Worker::Status> helper (loaded as in the boilerplate code
above) when returning a value, such as:
return Status->done('everything is good');
# or
return Status->error('something went wrong');
# or
return Status->defer('this device cannot be processed right now');
Remember that a worker is only run if it matches the hardware platform of the
target device and the user's configuration, and is not also excluded by the
user's configuration. This filtering takes place before inspecting C<primary>.
=head1 Accessing Transports
From your worker you will want to connect to a device to gather data. This is
done using a transport protocol session (SNMP, SSH, etc). Transports are
singleton objects instantiated on demand, so they can be shared among a set of
workers that are accessing the same device.
See the documentation for each transport to find out how to access it:
=over 4
=item *
L<App::Netdisco::Transport::SNMP>
=back
=head1 Database Connections
The Netdisco database is available via the C<netdisco> schema key, as below.
You can also use the C<external_databases> configuration item to set up
connections to other databases.
# plugin package
use Dancer::Plugin::DBIC;
my $set =
schema('netdisco')->resultset('Devices')
->search({vendor => 'cisco'});
=head1 Review of Terminology
In summary, Worker code is defined in a package namespace specifying the
Action and Phase, and registered as a plugin with configuration which may
specify the Driver and whether it is in the Primary slot. Access Control Lists
determine which Workers are permitted to run, and when. Here are more complete
definitions:
=over 4
=item C<action>
The highest level grouping of workers, corresponding to a Netdisco command
such as C<discover> or C<macsuck>. Workers can be registered at this level to
do really early bootstrapping work.
=item C<phase>
The next level down from C<action> for grouping workers. Phases have arbitrary
names and are visited in the order defined in the C<extra_worker_plugins>
setting list, followed by the C<worker_plugins> setting list. Workers are
usually registered at this level.
=item C<worker>
A lump of code you write which does a single clearly defined task. The package
namespace of the worker identifies the action and optionally the phase.
Workers are typically registered with some configuration settings.
=item C<driver>
A label associated with a group of workers which refers to a combination of
transport and application protocol used to connect to and communicate with the
target device. Users attach authentication configuration to specific drivers.
=item C<primary> (defaults to C<false>)
Indicates that the worker will only be run if no other C<primary> worker for
this phase has already succeeded. In this way, you can override Netdisco code
by setting this option and returning true from your worker.
=back
=cut

60
lib/App/Netdisco/Web/Plugin.pm
View File

@@ -232,65 +232,7 @@ Admin Menu function (job control, manual topology, pseudo devices)
=back
This document explains how to configure which plugins are loaded. See
L<App::Netdisco::Manual::WritingWebPlugins> if you want to develop new
plugins.
=head1 Application Configuration
Netdisco configuration supports a C<web_plugins> directive along with the
similar C<extra_web_plugins>. These list, in YAML format, the set of Perl
module names which are the plugins to be loaded. Each item injects one part of
the Netdisco web user interface.
You can override these settings to add, change, or remove entries from the
default lists. Here is an example of the C<web_plugins> list:
web_plugins:
- Inventory
- Report::DuplexMismatch
- Search::Device
- Search::Node
- Search::Port
- Device::Details
- Device::Ports
Any change should go into your local C<deployment.yml> configuration file. If
you want to view the default settings, see the C<share/config.yml> file in the
C<App::Netdisco> distribution.
=head1 How to Configure
The C<extra_web_plugins> setting is empty, and used only if you want to add
new plugins but not change the set enabled by default. If you do want to add
to or remove from the default set, then create a version of C<web_plugins>
instead.
Netdisco prepends "C<App::Netdisco::Web::Plugin::>" to any entry in the list.
For example, "C<Inventory>" will load the
C<App::Netdisco::Web::Plugin::Inventory> module.
Such plugin modules can either ship with the App::Netdisco distribution
itself, or be installed separately. Perl uses the standard C<@INC> path
searching mechanism to load the plugin modules.
If an entry in the list starts with a "C<+>" (plus) sign then Netdisco attemps
to load the module as-is, without prepending anything to the name. This allows
you to have App::Netdiso web UI plugins in other namespaces:
web_plugins:
- Inventory
- Search::Device
- Device::Details
- +My::Other::Netdisco::Web::Component
The order of the entries is significant. Unsurprisingly, the modules are
loaded in order. Therefore Navigation Bar items appear in the order listed,
and Tabs appear on the Search and Device pages in the order listed, and so on.
Finally, you can also prepend module names with "C<X::>", to support the
"Netdisco extension" namespace. For example, "C<X::Observium>" will load the
L<App::NetdiscoX::Web::Plugin::Observium> module.
See L<https://github.com/netdisco/netdisco/wiki/Web-Plugins> for details.
=cut

45
lib/App/Netdisco/Worker/Plugin.pm
View File

@@ -115,50 +115,7 @@ Workers can be restricted to certain vendor platforms using familiar ACL
syntax. They are also attached to specific actions in Netdisco's backend
operation (discover, macsuck, etc).
=head1 Application Configuration
See L<https://github.com/netdisco/netdisco/wiki/Backend-Plugins> for details.
The C<worker_plugins> and C<extra_worker_plugins> settings list in YAML format
the set of Perl module names which are the plugins to be loaded.
Any change should go into your local C<deployment.yml> configuration file. If
you want to view the default settings, see the C<share/config.yml> file in the
C<App::Netdisco> distribution.
=head1 How to Configure
The C<extra_worker_plugins> setting is empty, and used when you want to add
new plugins and not change the set enabled by default. If you do want to add
to or remove from the default set, then create a version of C<worker_plugins>
instead.
Netdisco prepends "C<App::Netdisco::Worker::Plugin::>" to any entry in the
list. For example, "C<Discover::Wireless::UniFi>" will load the
C<App::Netdisco::Worker::Plugin::Discover::Wireless::UniFi> package.
You can prepend module names with "C<X::>" as shorthand for the "Netdisco
extension" namespace. For example, "C<X::Macsuck::WirelessNodes::UniFi>" will
load the L<App::NetdiscoX::Worker::Plugin::Macsuck::WirelessNodes::UniFi>
module.
If an entry in the list starts with a "C<+>" (plus) sign then Netdisco attemps
to load the module as-is, without prepending anything to the name. This allows
you to have worker plugins in any namespace.
Plugin modules can either ship with the App::Netdisco distribution itself, or
be installed separately. Perl uses the standard C<@INC> path searching
mechanism to load the plugin modules. See the C<include_paths> and
C<site_local_files> settings in order to modify C<@INC> for loading local
plugins.
As an example, if you set C<site_local_files> to be true, set
C<extra_worker_plugins> to be C<'X::MyPluginName'> (the plugin package is
"App::NetdiscoX::Worker::Plugin::MyPluginName") then your plugin lives at:
~netdisco/nd-site-local/lib/App/NetdiscoX/Worker/Plugin/MyPluginName.pm
The order of the entries is significant, workers being executed in the order
which they appear in C<extra_worker_plugins> followed by C<worker_plugins>.
See L<App::Netdisco::Manual::WritingWorkers> for further details.
=cut