diff --git a/DEVELOPING.pod b/DEVELOPING.pod index 45d2b100..d281d0db 100644 --- a/DEVELOPING.pod +++ b/DEVELOPING.pod @@ -1,6 +1,245 @@ =head1 DEVELOPER NOTES -This document will help developers understand the intent and design of the -code within Netdisco. +This document aims to help developers understand the intent and design of the +code within Netdisco. Patches and feedback are always welcome :-) + +=head1 Introduction + +This release of Netdisco is built as a L application, and uses many +modern technologies and techniques. Hopefully this will make the code easier +to manage and maintain in the long term. + +Although Dancer is a web application framework, it provides very useful tools +for command line applications as well, namely configuration file management +and database connection management. We make use of these features in the +daemon and deployment scripts. + +Overall the application tries to be as self-contained as possible without also +needing an excessive number of CPAN modules to be installed. However, Modern +Perl techniques have made dependency management almost a non-issue, and +Netdisco can be installed by and run completely within an unprivileged user's +account, apart from the PostgreSQL database setup. + +Finally the other core component of Netdisco is now a L layer for +database access. This means there is no SQL anywhere in the code, but more +important, we can re-use the same complex queries in different parts of +Netdisco. + +The rest of this document discusses each "interesting" area of the Netdisco +codebase, hopefully in enough detail that you can get hacking yourself :-) + +=head1 Versioning + +This is Netdisco major version 2. The minor version has six digits, which are +split into two components of three digits each. It's unlikely that the major +version number (2) will increment. Each "feature" release to CPAN will +increment the first three digits of the minor version. Each "bug fix" release +will increment the second three digits of the minor version. + +Stable releases will have an even "feature" number. Beta releases will have an +odd "feature" number and also a suffix with an underscore, to prevent CPAN +indexing the distribution. Some examples: + + 2.002002 - "feature" release 2, "bug fix" release 2 + 2.002003 - another bug was found and fixed, hence "bug fix" release 3 + 2.003000_001 - first beta for the next "feature" release + 2.003000_002 - second beta + 2.004001 - the next "feature" release + + +=head1 Global Configuration + +Dancer uses YAML as its standard configuration file format, which is flexible +enough for our needs, yet still simple to edit for the user. We no longer need +a parser as in the old version of Netdisco. + +At the top of scripts you'll usually see something like: + + use App::Netdisco; + use Dancer ':script'; + +First, this uses C, which is almost nothing more than a +placeholder module (contains no actual application code). What it does is set +several environment variables in order to locate the configuration files. + +Then, when we call "C" these environment variables are used to +load two YAML files: C and C<< .yml >> where +C<< >> is typically either C or C. + +The concept of "environments" allows us to have some shared "master" config +between all instances of the application (C), and then settings +for specific circumstances. Typically this might be logging levels, for +example. The default file which C loads is C +but you can override it by setting the "C" environment +variable. + +Dancer loads the config using YAML, merging data from the two files. Config is +made available via Dancer's C subroutine, which is exported. +So now the C setting in either config file is easily accessed. + +Another line commonly seen in scripts is this: + + use Dancer::Plugin::DBIC 'schema'; + +This plugin saves a lot of effort by taking some database connection +parameters from the configuration file, and instantiating DBIx::Class database +connections with them. The connections are managed transparently so all we +need to do to access the Netdisco database, with no additional setup, is: + + schema('netdisco')->resultset(...)->search({...}); + + +=head1 DBIx::Class Layer + +DBIx::Class, or DBIC for short, is an Object-Relational Mapper. This means it +abstracts away the SQL of database calls, presenting a Perl object for each +table, set of results from a query, table row, etc. The advantage is that it +can generate really smart SQL queries, and these queries can be re-used +throughout the application. + +The DBIC layer for Netdisco is based at L. This is the +global schema class and below that, under L is a +class for each table in the database. These contain metadata on the columns +but also several handy "helper" queries which can be called. There are also +C classes which provide additional "pre-canned" queries. + +Netdisco's DBIx::Class layer has excellent documentation which you are +encouraged to read, particularly if you find it difficult to sleep. + +=head2 Results and ResultSets + +In DBIC a C is a table and a C is a set of rows retrieved +from the table as a result of a query (which might be all the rows, of +course). This is why we have two types of DBIC class. +Items in the C generally relate to the single table +directly, and simply. In the C class are more complex search +modifiers which might synthesize new "columns" of data (e.g. formatting a +timestamp) or subroutines which accept parameters to customize the query. + +However, regardless of the actual class name, you access them in the same way. +For example the C table has an L +class and also an L class. DBIC merges +the two: + + schema('netdisco')->resultset('Device')->get_models; + +=head2 Virtual Tables (VIEWs) + +Where we want to simplify our application code even further we can either +install a VIEW in PostgreSQL, or use DBIx::Class to synthesize the view +on-the-fly. Put simply, it uses the VIEW definition as the basis of an SQL +query, yet in the application we treat it as a real table like any other. + +Some good examples are a fake table of only the active Nodes (as opposed to +all nodes), or the more complex list of all ports which are connected together +(C). + +All these tables live under the +L namespace, and so you +access them like so (for the C example): + + schema('netdisco')->resultset('Virtual::ActiveNode')->count; + +=head2 Versioning and Deployment + +To manage the Netdisco schema in PostgreSQL we use DBIx::Class's deployment +feature. This attaches a version to the schema and provides all the code to +check the current version and do whatever is necessary to upgrade. +The schema version is stored in a new table called +C, although you should never touch it. + +The C script included in the distribution performs the +following services: + + * Installs the dbix_class_schema_versions table + * Upgrades the schema to the current distribtion's version + +This works both on an empty, new database, and a legacy database from the +existing Netdisco release, in a non-destructive way. For further information +see L and the C script. + +The files used for the upgrades are shipped with this distribution and stored +in the C<.../App/Netdisco/DB/schema_versions> directory. They are generated +using the C script which also ships with the distribution. + +=head2 Foreign Key Constraints + +We have not yet deployed any FK constraints into the Netdisco schema. This is +partly because the current poller inserts and deletes entries from the +database in an order which would violate such constraints, but also because +some of the archiving features of Netdisco might not be compatible anyway. + +Regardless, a lack of FK constraints doesn't upset DBIx::Class. The +constraints can easily be deployed in a future release of Netdisco. + + +=head1 Web Application + +The Netdisco web app is a "classic" Dancer app, using most of the bundled +features which make development really easy. Dancer is based on Ruby's Sinatra +framework. The theme is that many "helper" subroutines are exported into the +application namespace, to do things such as access request parameters, +navigate around your "handler" subroutines, manage response headers, and so +on. + +Pretty much anything you want to do in a web application has been wrapped up +into a neat helper routine that does the heavy lifting. This includes +configuration and database connection management, as was discussed above. +Also, templates can be executed and Netdisco uses the venerable +L engine for this. + +Like most web frameworks Dancer has a concept of "handlers" which are +subroutines to which a specific web request is routed. For example if Netdisco +asks for "C" with some parameters the request ends up at the +L package's "C" handler. All this +is done automatically by Dancer according to some simple rules. There are also +"wrapper" subroutines which we use to do tasks such as setting up data lookup +tables, and handling authentication. + +Dancer also supports AJAX very well, and it is used to retrieve most of the +data in the Netdisco web application in a dynamic way, to respond to search +queries and avoid lengthy page reloads. You will see the handlers for AJAX +look similar to those for GET requests but do not use Template::Toolkit +templates. + +=head2 Running the Web App + +=head2 Authentication + +=head2 Templates + +=head2 Javascript + + +=head1 Job Daemon + +=head2 SNMP::Info + +=head2 DBIx::Class Layer + +=head2 Running the Job Daemon =cut + + + + + + + + + + + + + + + + + + + + + + + diff --git a/Netdisco/lib/App/Netdisco.pm b/Netdisco/lib/App/Netdisco.pm index 9ff8e925..19023fbe 100644 --- a/Netdisco/lib/App/Netdisco.pm +++ b/Netdisco/lib/App/Netdisco.pm @@ -40,8 +40,8 @@ interactive requests such as changing port or device properties. There is not yet a device poller, so please still use the old Netdisco's discovery, arpnip, and macsuck. -If you have any trouble getting the frontend running, please speak to someone -in the C<#netdisco> IRC channel (on freenode). +If you have any trouble getting the frontend running, speak to someone in the +C<#netdisco> IRC channel (on freenode). =head1 Dependencies