I S G T O O L S v0.0.2
-------------------------
This software is Copyright (c) Robert Koliha, 2001-2002.
Robert Koliha grants you the rights to distribute and use
this software as governed by the terms of the GNU General
Public License (GPL).
(((((((( C O N T E N T S ))))))))
Section 1 - Introduction & Features
Section 2 - Requirements & Dependencies
Section 3 - Installation
Section 4 - Files
Section 5 - License Agreement
(((((((( I N T R O & F E A T U R E S ))))))))
What is ISG Tools?
ISG Tools is a small set of scripts and html that allow you to perform most of
the commands that are utilized in a telnet session to a uBR via a web based
interface. Modem searches (by partial or full mac address) take about 15
seconds on a 7 uBR system.
ISG Tools also comes with several backend monitoring scripts that do
things like ip address logging.
Current and historical ip data can be searched thru using two included cgi
scripts.
ISG Tools is designed for use with Cisco uBRs. Other equipment may require
manual editing of the scripts and some of the commands used within them.
There are a couple of configuration requirements for both the host machine
and the uBRs. It also may be useful to note that this was developed and is
in operation on a redhat 7.x system.
ISG Tools was developed in a real world enviorment. This package is
currently being used to assist in supporting a 15,000+ user system.
What used to take 3-8 minutes over the phone can now be done in a matter
of seconds.
CGI Features:
* Search for a modem (searches all uBR's) using partial mac address
Modem Search ping tests the modem and both hosts behind the modem.
It also lists the modem, host, previous days flap, and live
flap statistics. After the search is complete you can reset the modem,
clear either host, clear the modem from the flaplist, "watch" the signal
levels (signal level page that auto refreshes every 1-3 seconds), or pull
the modems logfile with a single click. There is also a "text version"
link that has the same information in text format for easy copying and
pasting into email or an internal ticket based support system
* Two simplified versions of the above search script. One that is geared
for a dispatch or tech oriented enviorment where you don't need to
reveal more than basic signal information and you want to hide client
ip addresses, and one that is a self service cgi for customers to
access to test their own modems. The second script is in plain language,
has basic mac and ip information, modem name/uptime, ping tests, pass or
fail upstream/downstream/snr test, if it detects more than 30 flaps
it will display 3 different warning messages.
* Pull live statistics from any of the uBR's. Includes: IOS Version, Uptime
Information, SNR information for every blade, and modem statistics
* Pull the daily flaplist from any uBR
* Pull a live flaplist from any uBR
* Look at modems currently offline from any uBR
* Look at modems currently in the init state
* Look at modems that are currenlty max screamers
* Realtime search for an ip address (shows modem & hosts behind it)
* Historical (Month/Day/Hour) search for an ip address
Backend Features:
* Creates daily flaplist, clears the flaplist, and emails the results to
selected mail recipients.
* Logs IP addresses from all uBR's. Keeps a 1 year compressed backlog.
* Searches ip logs daily for invalid ip addresses and emails the results
to selected mail recipients.
There are example html forms located in the examples/ directory. These
examples are also installed to www.yourserver.net/isgtools/ (index included)
IMPORTANT NOTE: All of the cgi scripts require that variables be passed
to them in a certain order. This means that the order the
form fields are listed in the example html is the same way
that you need to design custom forms. It's also important
that the variable names such as macid= and ubr= aren't
changed (espically the length!)
(((((((( R E Q U I R E M E N T S & D E P E N D E N C I E S ))))))))
[software]
GNU Textutils - http://www.gnu.org/software/textutils/textutils.html
grep - http://www.gnu.org/software/grep/grep.html
Sun's JRE - http://java.sun.com
NetCat - http://www.freshmeat.net/projects/netcat
DocsDiag - http://homepage.ntlworld.com/robin.d.h.walker/docsdiag/
(DocsDiag is INCLUDED w/ISG Tools 0.0.2+)
Thanks to Robin Walker with v0.0.2+ of ISG Tools DocsDiag (docsdiag.jar)
is included with this distribution.
DocsDiag is a program for reading diagnostic data from DOCSIS cable modems.
It should work with any DOCSIS-compliant cable modem that has SNMP access
avalible to the end-user (in this case we are using it on the provider
side). DocsDiag is a text-mode console program without any gui. It will
run on any platform that supports Java console applications.
I'm pretty sure everything else comes with a full install of RedHat and
possibly other linux distributions. If in doubt and your distribution
supports .rpm packages try: rpm -a -q | grep insert-program-name-here
[router requirements]
You will need to enable the following commands for the non-enabled account
on the uBRs so that the scripts can interact with the router:
(I reccomend doing this BEFORE installation)
1) clear cable flap
2) clear cable host
3) clear cable modem 0000.0000.0000 reset
4) show cable modem
5) show cable flap
6) show proc
7) show ver
8) show cont
9) show cab mod sum tot
10) show int c?/0 mod 0
You will also need to make sure that there are NO LOGIN BANNERS!!
When you telnet to the uBR it should just and ask for a username or
password.. Any line with the word 'User' (case sensitive) is stripped
off by the scripts. If you absolutely have to have a login banner you
will need to add the appropriate ' | grep -v keyword ' for every line
so the scripts will strip it properly.
(((((( I N S T A L L A T I O N O V E R V I E W ))))))
*** MAKE SURE YOU ARE SIGNED IN AS ROOT ***
1) Open isgtools.conf with your favorite text editor (jed) and customize
the variables for your system.
2) Open iplogger/invalid-scan and edit it to suit your system if you want
to use the invalid ip scanner
3) Run ./install
This script installs everything but the cron scripts
4) Read the cron script installation instructions below
or optionally: (if you didn't do it at the end of step 3)
cd cron/
./install.redhat
** NOTE: After the installation is complete the config file for
ISG Tools will be located at /etc/isgtools.sh
(((((( C R O N ))))))
There are 8 cron scripts that are associated with ISG Tools
1) cron/weather-image
A bonus script I wrote to pull a simple image with just the
current temperature from weather underground.
CUSTOMIZE & READ COMMENTS BEFORE USE
2) cron/cron.2359/iplog-rotate
This script compresses and rotates the current days iplogs.
!!THIS MUST BE RUN BETWEEN 11:01-11:59 EVERY NIGHT FOR THE
IP LOGGER AND IP SEARCH CGI COMPONENTS TO WORK PROPERLY!!
3) cron/cron.daily/cgi-cleanup
This script removes all of the previous days text based modem
search results and removes any stale snmp temp files in the
cgi-bin/ directory
4) cron/cron.daily/flaplist
This pulls the daily flaplist from the uBR's, clears the list
and emails the results of the pull to the email address(es)
configured in isgtools.conf
5) cron/cron.daily/invalid-scan
This runs the script to check for invalid ip addresses that
are reported from the uBRs and emails the results to the email
address(es) configured in isgtools.conf
6) cron/cron.hourly/iplogger
This script grabs & stores all of the modem and cpe ip addresses
!!THIS MUST BE RUN HOURLY FOR THE IP LOGGER AND IP SEARCH CGI
COMPONENTS TO WORK PROPERLY!!
7) cron/cron.hourly/checkhttpd
Checks to make sure httpd is running, if it's not then it shuts
the service down (/etc/init.d/httpd stop) and then restarts it.
It will also generate a cron-style error email stating when it
restarted the service
8) cron/cron.minute/pollubrs
This script pulls all of the modems from all of the uBR's for
use with the modemsearch and other scripts. It is highly reccomended
that this script be run every minute.
ie: If you run it every 5 minutes, when I plug in a brand new modem
it won't show up with the tool for 5 minutes..
9) cron/cron.weekly/cgi-logrotate
This script rotates the log files created by some of the cgi scripts
By default only 5 weeks worth of logs are stored in the cgi-bin
Reccomended run times for these scripts..
iplog-rotate - !!BETWEEN 11:30-11:59 NIGHTLY!!
cgi-cleanup - daily
flaplist - daily
invalid-scan - daily (optional to install)
iplogger - !!HOURLY!!
checkhttpd - every minute / every hour (up to you)
pollubrs - every minute (HIGHLY RECCOMENDED)
cgi-logrotate - weekly
(((((( F I L E S ))))))
cgi-bin/clearflap - cgi script that users interact w/to clear a modem
cgi-bin/cflap - backend script that clears a modem from the flaplist
cgi-bin/clearhost - cgi script that users interact w/to clear a host
cgi-bin/chost - backend script that clears a host
cgi-bin/clearmodem - cgi script that users interact w/to reset a modem
cgi-bin/cmodem - backend script that resets a modem
cgi-bin/modemsearch - modem search cgi script
cgi-bin/cmsearch - backend script that actually does the search
cgi-bin/ubr-poll - backend script that pulls live host information
cgi-bin/modemcheck - simplified modem search cgi script
cgi-bin/cmsearch-simplified - backend script that actually does the search
cgi-bin/modemtest - self serve modem test cgi script
cgi-bin/cmt - backend script that actually does the search
cgi-bin/modemwatch - modem signal level poll every 1-3 seconds
cgi-bin/modemw - script to pull info from modem and determine proper
graphics to display in output html
cgi-bin/modemlog - script that pulls a selected modems logfile
cgi-bin/flap - cats the archived flaplist to stdout (used as a cgi & elsewhere)
cgi-bin/liveflap - pulls the live flaplist from a selected ubr
cgi-bin/lflap - backend script that actually pulls the info
cgi-bin/init - shows initalizing modems on a selected ubr
cgi-bin/init-main - backend script that actually pulls the info
cgi-bin/offline - shows offline modems on a selected ubr
cgi-bin/offline-main - backend script that actually pulls the info
cgi-bin/maxscream - shows initalizing modems on a selected ubr
cgi-bin/maxscream-main - backend script that actually pulls the info
cgi-bin/ubrcheck - script to pull ubr diagnostic info
cgi-bin/ubrchk - backend script that actually pulls the info
cgi-bin/ubrdata - this is the directory where the once a minute uBR
poll is stored for use in multiple cgi scripts
There are also several logfiles located in the cgi-bin like
cmsearch.log or chost.log.
cmsearch:
[date and time] ip address - search string
clear scripts (like cflap, chost, etc):
[date and time] ip address - uBR #/macid
Logfiles are rotated weekly and the .4 logfile is rotated to /dev/null
so we don't fill any unnessicary space (5 weeks is plenty of log).
flap/flaplist - script to pull, store, and clear the flaplist
(only meant to be ran once daily)
flap/mailresults - script to mail the flaplist to the recipients
specified in /etc/isgtools.sh
flap/results - results storage
iplogger/ip-grab - hourly script to pull & store ip address information
iplogger/compress-current - daily script to compress the current days data
(runs at 11:59, must run between 11:30-11:59)
iplogger/invalid-scan - script to scan for invalid ip addresses nightly
(this is the file that has to be manually edited)
iplogger/logs - stores current day and archive
iplogger/archive - stores archived (directory structure by month)
icons/clear.gif - Clear modem button
icons/dormant.gif - USB 'dormant' status indicator
icons/down(2).gif - USB/ETH 'down' status indicator
icons/fail.gif - Self Serve CGI 'FAILED' graphic
icons/green.gif - Modem signal green status indicator
icons/logfile.gif - Logfile button
icons/pass.gif - Self Serve CGI 'PASSED' graphic
icons/red.gif - Modem signal red status indicator
icons/reset.gif - Reset modem button
icons/text.gif - Text based results button
icons/unknown.gif - unknown/modem unreachable signal indicator
icons/up(1).gif - USB/ETH 'up' status indicator
icons/watch.gif - Watch modem button
icons/yellow.gif - Modem signal yellow status indicator
(((((( L I C E N S E A G R E E M E N T ))))))
GNU General Public License
Version 2, June 1991
Copyright (C) 1989, 1991 Free Software Foundation, Inc.
59 Temple Place - Suite 330, Boston, MA 02111-1307, USA
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The licenses for most software are designed to take away your freedom
to share and change it. By contrast, the GNU General Public License is
intended to guarantee your freedom to share and change free
software--to make sure the software is free for all its users. This
General Public License applies to most of the Free Software Foundation's
software and to any other program whose authors commit to using it.
(Some other Free Software Foundation software is covered by the GNU
Library General Public License instead.) You can apply it to your
programs, too.
When we speak of free software, we are referring to freedom, not price.
Our General Public Licenses are designed to make sure that you have the
freedom to distribute copies of free software (and charge for this service
if you wish), that you receive source code or can get it if you want it,
that you can change the software or use pieces of it in new free programs;
and that you know you can do these things. To protect your rights, we need
to make restrictions that forbid anyone to deny you these rights or to ask
you to surrender the rights. These restrictions translate to certain
responsibilities for you if you distribute copies of the software, or if
you modify it. For example, if you distribute copies of such a program,
whether gratis or for a fee, you must give the recipients all the rights
that you have. You must make sure that they, too, receive or can get the
source code. And you must show them these terms so they know their rights.
We protect your rights with two steps:
(1) copyright the software, and (2) offer you this license which gives you
legal permission to copy, distribute and/or modify the software.
Also, for each author's protection and ours, we want to make certain that
everyone understands that there is no warranty for this free software.
If the software is modified by someone else and passed on, we want its
recipients to know that what they have is not the original, so that any
problems introduced by others will not reflect on the original authors'
reputations.
Finally, any free program is threatened constantly by software patents.
We wish to avoid the danger that redistributors of a free program will
individually obtain patent licenses, in effect making the program
proprietary. To prevent this, we have made it clear that any patent must be
licensed for everyone's free use or not licensed at all.
The precise terms and conditions for copying, distribution and modification
follow.
TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
0. This License applies to any program or other work which contains a
notice placed by the copyright holder saying it may be distributed
under the terms of this General Public License. The "Program", below,
refers to any such program or work, and a "work based on the Program"
means either the Program or any derivative work under copyright law: that
is to say, a work containing the Program or a portion of it, either verbatim or
with modifications and/or translated into another language. (Hereinafter,
translation is included without limitation in the term "modification".)
Each licensee is addressed as "you".
Activities other than copying, distribution and modification are not covered
by this License; they are outside its scope. The act of running the Program
is not restricted, and the output from the Program is covered only if
its contents constitute a work based on the Program (independent of having
been made by running the Program). Whether that is true depends on what the
Program does.
1. You may copy and distribute verbatim copies of the Program's source code
as you receive it, in any medium, provided that you conspicuously and
appropriately publish on each copy an appropriate copyright notice and
disclaimer of warranty; keep intact all the notices that refer to this License
and to the absence of any warranty; and give any other recipients of the
Program a copy of this License along with the Program. You may charge a fee
for the physical act of transferring a copy, and you may at your option offer
warranty protection in exchange for a fee.
2. You may modify your copy or copies of the Program or any portion of it,
thus forming a work based on the Program, and copy and distribute such
modifications or work under the terms of Section 1 above, provided that you
also meet all of these conditions:
a) You must cause the modified files to carry prominent notices stating that
you changed the files and the date of any change.
b) You must cause any work that you distribute or publish, that in whole or
in part contains or is derived from the Program or any part thereof, to be
licensed as a whole at no charge to all third parties under the terms of
this License.
c) If the modified program normally reads commands interactively when run,
you must cause it, when started running for such interactive use in the most
ordinary way, to print or display an announcement including an appropriate
copyright notice and a notice that there is no warranty (or else, saying that
you provide a warranty) and that users may redistribute the program under
these conditions, and telling the user how to view a copy of this License.
(Exception: if the Program itself is interactive but does not normally print
such an announcement, your work based on the Program is not required to print
an announcement.)
These requirements apply to the modified work as a whole. If identifiable
sections of that work are not derived from the Program, and can be reasonably
considered independent and separate works in themselves, then this License,
and its terms, do not apply to those sections when you distribute them as
separate works. But when you distribute the same sections as part of a whole
which is a work based on the Program, the distribution of the whole must be
on the terms of this License, whose permissions for other licensees extend to
the entire whole, and thus to each and every part regardless of who wrote it.
Thus, it is not the intent of this section to claim rights or contest your
rights to work written entirely by you; rather, the intent is to exercise the
right to control the distribution of derivative or collective works based on
the Program. In addition, mere aggregation of another work not based on the
Program with the Program (or with a work based on the Program) on a volume of
a storage or distribution medium does not bring the other work under the
scope of this License.
3. You may copy and distribute the Program (or a work based on it, under
Section 2) in object code or executable form under the terms of Sections 1
and 2 above provided that you also do one of the following:
a) Accompany it with the complete corresponding machine-readable source code,
which must be distributed under the terms of Sections 1 and 2 above on a
medium customarily used for software interchange; or,
b) Accompany it with a written offer, valid for at least three years, to give
any third party, for a charge no more than your cost of physically performing
source distribution, a complete machine-readable copy of the corresponding
source code, to be distributed under the terms of Sections 1 and 2 above on a
medium customarily used for software interchange; or,
c) Accompany it with the information you received as to the offer to
distribute corresponding source code. (This alternative is allowed only for
noncommercial distribution and only if you received the program in object
code or executable form with such an offer, in accord with Subsection b above.)
The source code for a work means the preferred form of the work for making
modifications to it. For an executable work, complete source code means
all the source code for all modules it contains, plus any associated interface
definition files, plus the scripts used to control compilation and installation
of the executable. However, as a special exception, the source code distributed
need not include anything that is normally distributed (in either source or
binary form) with the major components (compiler, kernel, and so on) of the
operating system on which the executable runs, unless that component itself
accompanies the executable.
If distribution of executable or object code is made by offering access to
copy from a designated place, then offering equivalent access to copy the
source code from the same place counts as distribution of the source code,
even though third parties are not compelled to copy the source along with the
object code.
4. You may not copy, modify, sublicense, or distribute the Program except as
expressly provided under this License. Any attempt otherwise to copy, modify,
sublicense or distribute the Program is void, and will automatically terminate
your rights under this License. However, parties who have received copies, or
rights, from you under this License will not have their licenses terminated
so long as such parties remain in full compliance.
5. You are not required to accept this License, since you have not signed it.
However, nothing else grants you permission to modify or distribute
the Program or its derivative works. These actions are prohibited by law if
you do not accept this License. Therefore, by modifying or distributing the
Program (or any work based on the Program), you indicate your acceptance of
this License to do so, and all its terms and conditions for copying,
distributing or modifying the Program or works based on it.
6. Each time you redistribute the Program (or any work based on the Program),
the recipient automatically receives a license from the original licensor to
copy, distribute or modify the Program subject to these terms and conditions.
You may not impose any further restrictions on the recipients' exercise of
the rights granted herein. You are not responsible for enforcing compliance
by third parties to this License.
7. If, as a consequence of a court judgment or allegation of patent
infringement or for any other reason (not limited to patent issues),
conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot distribute so
as to satisfy simultaneously your obligations under this License and any other
pertinent obligations, then as a consequence you may not distribute the
Program at all. For example, if a patent license would not permit
royalty-free redistribution of the Program by all those who receive copies
directly or indirectly through you, then the only way you could satisfy
both it and this License would be to refrain entirely from distribution
of the Program.
If any portion of this section is held invalid or unenforceable under any
particular circumstance, the balance of the section is intended to apply and
the section as a whole is intended to apply in other circumstances. It is
not the purpose of this section to induce you to infringe any patents or
other property right claims or to contest validity of any such claims; this
section has the sole purpose of protecting the integrity of the free
software distribution system, which is implemented by public license
practices. Many people have made generous contributions to the wide range of
software distributed through that system in reliance on consistent
application of that system; it is up to the author/donor to decide if he or
she is willing to distribute software through any other system and a
licensee cannot impose that choice. This section is intended to make
thoroughly clear what is believed to be a consequence of the rest of
this License.
8. If the distribution and/or use of the Program is restricted in certain
countries either by patents or by copyrighted interfaces, the original
copyright holder who places the Program under this License may add an
explicit geographical distribution limitation excluding those countries,
so that distribution is permitted only in or among countries not thus
excluded. In such case, this License incorporates the limitation as if
written in the body of this License.
9. The Free Software Foundation may publish revised and/or new versions of
the General Public License from time to time. Such new versions will be
similar in spirit to the present version, but may differ in detail to
address new problems or concerns. Each version is given a distinguishing
version number. If the Program specifies a version number of this License
which applies to it and "any later version", you have the option of
following the terms and conditions either of that version or of any later
version published by the Free Software Foundation. If the Program does not
specify a version number of this License, you may choose any version
everpublished by the Free Software Foundation.
10. If you wish to incorporate parts of the Program into other free programs
whose distribution conditions are different, write to the author to ask for
permission. For software which is copyrighted by the Free Software Foundation,
write to the Free Software Foundation; we sometimes make exceptions for this.
Our decision will be guided by the two goals of preserving the free status
of all derivatives of our free software and of promoting the sharing and
reuse of software generally.
NO WARRANTY
11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR
THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN
OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO
THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM
PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR
CORRECTION.
12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO
LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR
THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.
