What is Nbsp?
The Noaaport Broadcast System Processor (nbsp) is a software suite for receiving, processing and distributing the contents of the noaaport data stream. The program is developed in FreeBSD, and packages exist for FreeBSD, the CentOS, Debian and Ubuntu distributions of Linux.
The core module of nbsp, written in C, receives and unpacks the noaaport data fragments, and stores the complete reassembled files in a ready-to-use format on a local disk. The files are then procesed further by a comprehensive set of scripts and libraries written in Tcl, suitable for particular applications including,
- storing them in a standard directory tree, appropriate for viewing with any web browser and/or ordinary editors and image manipulation programs (bulletins as pure text, satellite images as png or jpg files, and radar images as gif files), including options for creating loop sequences of the images
- or in a gempak-like directory tree structure and file format, ready to be used by the GemPak viewers and tools to produce standard plots with them
- or in a digatmos-friendly form (and suitable naming convention) that is amenable for importing and analyzing with several Windows applications, including Digital Atmosphere, IDV, GRLevel3, Weatherscope and MetarWeather
None of these configurations excludes the others, and these various data directories can be placed under one parent directory such that the different data formats can serve different purposes. This data directory, which resides in the nbsp server machine, can be exported via samba, to windows PC machines, in which the text, sat, rad and digatmos directories can be mounted and accessed as if they were stored in a local drive in the PC.
nbsp is compliant with the Open Nexrad Data Access Specification, which aims to make the raw radar data available to clients in a conventional manner. This means that, by default, nbsp creates a directory layout and provides the infrastructure to support services and clients that have adopted this specification, such as GRLevel3 and Digital Atmosphere.
nbsp can distribute the files through the network in various formats, including standad http and ftp channels, or the QBT (Quick Block Transfer ``byte blaster'') format used by emwin(pure text bulletins as well as jpg satellite images and gif radar images). It has the capability to distribute the files by nntp, via an inn news gateway, and by RSS feeds accessible from the built-in web server.
nbsp has a web interface for monitoring its activities. The built-in web server supplies information about the files received, and can report various statistics that nbsp keeps about its internal state. The web server is based on the tclhttpd library, which is extensible and can be adapted to many uses.
An ``exporter'' module allows client applications to query and retrieve all the data stored by the server, by sending a few simple http commands to the web server.
nbsp can insert the files in an ldm queue, allowing a running ldm to distribute and/or process the files further in the usual way.
nbsp can run standalone, or in a master-slave mode combination. In a master-slave setup, the master instance of nbsp receives the broadcast stream in one computer, and then transmits the complete files to a slave instance of nbsp running in a different computer for further processing and/or distribution of the files.
Extensive and flexible capabilities exist for processing the files further by any number of postprocessors or filters. The filters, which can be written in any language, can be used to rewrite the files in any format, or to save them in different directories or using a particular naming convention, or for distributing the files using any external program. These facilities can be used for configuring the system for specialized tasks, such as installing a notification system when particular files are received, or to fit specific customized storage, backup and retrieval schemes.
What nbsp is not
The nbsp system does not come with graphical tools for manipulating or viewing the data and image files. The text products are stored as text files, and the satellite images can be saved as jpg and/or png files automatically, or manually if desired, with supplied tools. But other types of binary files can be read only by special external programs, such as those of gempak on unix or Digital Atmosphere on windows.
Features
- nbsp takes its orders from a main configuration file and from various run control scripts at different stages. They are written in tcl, which is a well documented and extensible language, and allows for very customizable configurations of the system.
- By default, all the files received from all four noaaport channels are stored on a local disk in ready-to-use format. The set of files processed and the noaaport channels that are opened can be restricted by means of various configuration options. Similarly, the directory where the files are saved as well as many other options are fully configurable without need of recompilation. As a rule, for any parameter where the could be a choice to be made, it has been designed as a configurable option.
- The processed files can be distributed in various ways. A built-in emwin-like server can listen for standard emwin clients connections, and it will serve those clients the files in the expected format. A facility exists for distributing the files via an inn news gateway, and they can also be distributed by a fax or email gateway. A built-in web server can distribute the processed files by http, including RSS feeds. The received files can also be inserted in an ldm queue where they can be further processed and distributed by a running ldm in the usual way.
- Powerfull and extensive filtering or postprocessing facilities are built-in, designed to extend the functionality of the system in very flexible ways. Postprocessing filters can be used for time-stamping and/or backing the files, selectively redistributing them by various means, and for implementing a complete notification facility. The postprocessors and filters can be written in any language, such as perl, C or any other. The language officially supported by the distribution is tcl.
- The program can natively decode the text files, satellite and radar images, and store them in a configurable directory appropriate for viewing with ordinary tools, such a web browser, without the need of any external program. If the gempak tools are installed, the satelite images can be enhanced with the map outlines. A filter is able to process the files and save them in a format and directory tree compatible for viewing and manipulating them with the gempak programs.
- The files can be saved in a digatmos-friendly form (and suitable naming convention) appropriate for importing by Digital Atmosphere. Moreover, the text, sat, rad and digatmos directories can be exported via samba in the nbsp server, such that they can then be mounted and accessed as if they were stored in a local drive in a windows PC.
- Various specialized ``filters'' handle specific data. For example, the gribfilter saves the grib files in useful collections and creates maps and plots in a very user-configurable way. Similarly the ``metarfilter'' handles data from Metar reports. The ``trackfilter'' can do the same with hurricane data and can create tracking plots of past and forecast positions. All these features and more are illustrated in the sample files available elsewhere in this site.
- Technically, nbsp is a multithreaded application. There are four separate reader threads, each connected to one noaaport channel, a network server thread, a filter server thread, and the main processing thread, running concurrently for speed and efficiency. The threads communicate among themselves by means of a queue system built around the Berkeley DB, which is fast and reliable. The threaded model allows the program to make good use of dual processor systems. Alternatively, the system can be configured to split the load between two different computers, a minimal processing unit in one computer and the filtering facility in another. However, this is by no means a necessity, and the threaded model is very effective in a uniprocessor system as well.
- It is free
Future development
Nbsp is subject to continuous development, looking for improvement in overall performance, fixing bugs, and making enhancements compatible with the design goals: configurable and adaptable to different uses.