The Northern Hemisphere Marine Aerosol Characterization Experiment
(ACE-2) of the International Global Atmospheric Chemistry Project (IGAC)
is the second in a series of experiments to study the radiative effects
and controlling processes of anthropogenic aerosols from Europe and desert
dust from the Africa as they are transported over the North Atlantic Ocean.
The intensive field phase occurred from 16 June to 25 July 1997 with a
base of operations in Tenerife, Spain. The experiment involved over 250
research scientists from Europe and the United States. It included 60 coordinated
aircraft missions with 6 aircraft (for a total of 450 flight hours), one
ship, and ground stations on Tenerife, Portugal and Madeira.
During the multi-day experiment, a huge amount of operational data (primarily meteo and satellite data) together with a selected set of research data (chemical and physical parameters) and a list of status reports from the various platforms, were ingested daily and processed in a timely fashion to prepare a comprehensive report that serves as input to the daily operations planning and monitoring achievements evaluation. To support all these data processing activities at the Operational Center at the Tenerife Los Rodeos airport, a World Wide Web (WWW ) system, the so-called On-Line Catalogue (OLC), was developed.
During a day at the ACE-2 Operational Center a timely organized set
of activities were to be performed. Figure 1 depicts a general schema of
the information and workflow within the Operational Center and between
the center and the various platforms.
The Science Team (ST) was responsible to assess monitoring achievements, to plan missions for the next day and to take decisions in general. Each coordinated mission covered a set of tasks, and involved a number of platforms (aircraft's and land based sites) and satellites. The "go no go" decisions for missions were being taken the morning after, mainly based on meteo conditions. At the end of the day, during the evening Science Team Meeting (STM) a set of status information provided by platforms together with a rich set of meteorological data were being used to evaluate missions performed and result achievements. Every day, each Platform must provide a set of status information and data about undertaken actions during the day. These included a set of answers related to planned missions in which the platform was involved, a description of unplanned actions undertaken, aircraft flight hours, general status information of instrumentation and expected status of the platform for the day after. Land based sites provided additional measured data about aerosols chemical/physical parameters (scattering, absorption etc.) and wind direction/speed. The Forecast Team (FT) was responsible to collect general meteorological data, nowcasts and forecasts, to be used to complement platforms status information to plan future actions and to evaluate results. The meteo offices KNMI, INM and UKMO provided independently forecast images and bulletins specific for the whole ACE-2 area. The Naval Postgraduate School (NPS) of the US Navy provided real-time images from the NOAA-12 and NOAA-14 satellites, while a rich set of dedicated images were being calculated by other organizations (MEDUSE, NASA, Meteo France).
All operations during the experiment were coordinated by the Operations Management Team (OPSMT), sited at the Operational Center at the Tenerife North airport. The main tasks of the OPSMT were to provide general status reports to the ST and platforms, to off-line process numerical data coming from platforms and to collect and distribute any useful information.
To help the OPSMT to provide everybody involved in the experiment with
up to date information of ST decisions, plans, meteo conditions, to allow
the ST to plan and evaluate missions as well as the platforms and the FT
to input needed data, the WWW based On-Line Catalog system was developed.
The On-Line Catalog (OLC) is an easy to use on-line, real-time, dynamic, safe and WWW distributed information system, that provides a set of tools to collect, process, archive and distribute any useful data in a timely organized fashion.
More in detail the system characteristics are the following :
Using a WWW client/server architecture, the system allows to any user inside or outside the Operational Center to insert and upload data as well as retrieve any information of interest from a central server in a easy and common way.
All data and information entered into the system are immediately processed, timely organized and then distributed to the users. Using a small database engine coupled with a Web server, all the users interfaces and HTML documents are dynamically assembled on demand, to reflect the specific user requirements and containing the latest and newest information. Using the OLC, ACE-2 people feel they contribute directly into the data and information part of the experiment, because as soon as they inserted and uploaded a new piece of data, it is immediately browsable and usable by everybody.
The easiness of HTML Web browser based user interfaces allows an immediately use and understand of the system functionalities. Technical features such as the possibility upload to a central server a set of data coupled with standard HTML form elements permit to collect easily into a common database all relevant data and information provided by various ACE-2 users during a day.
The OLC defines a set of user levels (or categories), each one corresponding to different privileges and different operations can be performed on the system. Platforms, the FT, the ST and the OPSMT use different subsets of the OLC HTML pages, targeted for they needs. Any user can browse all available catalog information. The OPSMT is defined as a privileged user and can insert/delete/modify any type of information. Guest users can only browse the catalog (e.g. external visitors to the system can look at data and reports).
The main goal of the OLC system is to automate all the ACE-2 daily activities, providing a set of WWW based tools to assist the users to fulfill their tasks. Mission tools, task evaluation tools and data upload/insert/delete/modify tools have been prepared.
After a careful analysis of ACE-2 daily activities and the general information flow, the following logical structure was given :
A report is a collection of related data and information. The are three
kinds of reports : Status Report, Science Team Report and Platforms Report.
Every day each platform compiles a Status Report containing a debriefing
of undertaken actions and specific status information. Land based platforms
provide additional on-line data relative to observed chemical/physical
parameters. The Science Team Report consists of general status information,
a debriefing, the mission plan for day N, and a rich set of meteorological
information provided by the FT. The ST evaluates tasks fulfillment for
day N and plans missions for day N+1. A mission covers a set of tasks (or
activities) and involves a number of platforms. The Platforms Report contains
a summary of fulfilled tasks, the mission plan for day N+1 and a set of
forecasts both general and specific per platform.
The following OLC users have been defined :
Each user has different privileges and can perform different kind of
operations on the system. A summary of users and allowed operations is
given in Table 1.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The OLC aims to provide support for ACE-2 investigators in the following
activities :
This option is used by all members of the FT to insert any meteorological product into the system. The KNMI and the INM meteo offices provide trajectories, numerical fields (HIRLAM, ECMWF), synoptic data, METOSAT satellite images, forecasts text description and a set of predicted values of airmass and cloudcover. The UKMO group provides specific forward and backward trajectories. Satellite images coming from the NOAA-12 and NOAA-14 satellites are inserted by the Durkee user (NPS). An additional set of satellite images (Dulac, TOMS, MeteoFrance) and dust model predictions (MEDUSE) are retrieved from the Internet and then inserted by the OPSMT (or Data Management).
FT, Durkee, UKMO, OPSMT (or DM)
Every platform (aircraft or surface based) insert its daily status report through this functionality. The status report contains a set of status information indicating whether the platform has successfully participated in the tasks in which it was involved, whether instrumentation is/was working correctly, the expected status of the platform for next days and any other observations or description about undertaken actions. Surface based users provide an additional set of numeric data(specific for each platform), concerning cloud cover, type of clouds, type of airmass and a set of data files containing measured values about aerosol number concentration, wind speed and direction, aerosol absorption/scattering, total aerosol mass and aerosol surface area and volume. These data files are attached to the report using the browser upload facility.
Sagres, Hidalgo, Taganana, R/V Vodyanitsky, Izana, C-130, ARAT, Dornier, Merlin, Citation, Pelican, OPSMT
The Science Team uses the mission tool to plan next day missions. The tool has the following options :
ST, OPSMT
After the definition of a new mission the ST has to do a "go-no-go" decision to effectively plan a mission. This decision consequently changes the contents of the system. If the mission can not be started, the mission remains unplanned. To start a mission, the user must select the mission code from a list.
ST, OPSMT
During the Science Team Meeting or immediately after the ST decides which tasks have been fulfilled during the day. The priority for each task is given as M + N, in which:
ST, OPSMT
As soon as platforms upload the raw data files (on-line data),the OPSMT needs to off-line process these in a set of times series describing parameters along a specific period of time. To assist the user in this task, this option allows to directly access the inserted raw data files and to upload the respective processed files.
OPSMT
During the day every user can look at latest and newest data and information using this option. The user can display the following information :
To display all available meteorological data files (satellite images, numerical fields, dust model predictions and trajectories) the user must select the third item. All the timeserie plots derived from the aerosol parameters (scattering, surface, absorption etc.) can be display by selecting the fourth option. By selecting the last item from the list, each platform user can display a specific report containing all missions performed, tasks fulfilled, mission plans and a set of detailed forecasts for the platform; a general report containing all platforms information (platforms report) can be visualized too.
guest users and any other
All information stored into the OLC can be visualized through this option. The user can select exactly the same set of options of the "View today?s information" functionality, but all data displayed will refer to a specific day. By selecting a date from a list, the user is able to easily access and display a specific piece of information. It is possible to browse a specific data file or report per day, to see changes, evolutions and to do comparisons between them. This is a powerful tool!
guest users and any other
A complete logbook of performed missions can be displayed with this option. By selecting a mission code the full mission content is visualized.
guest users and any other
This option displays the status of ACE-2 tasks. For each task both the description and priorities are represented.
guest users and any other
Operations planning and decisions making require more information than the one stored into the OLC. All needed additional information are put into a common document accessible by selecting this option. The document is put in a public FTP area on the main server, and can be easily modified.
guest users and any other
To correct errors within meteo data files, on-line data or any other uploaded file, the user must select this option. To replace an existing file is necessary to remove it and re-insert it afterwards.
OPSMT
To correct typing errors inside status reports and to change numerical values of airmass and cloudcover, the user must select this option.
OPSMT
The ST user can define the time of the next Science Team Meeting (STM) by selecting the right time from a list of possibilities. By changing this option, the user can postpone or leave early the daily meeting and consequently modify the time thresholds for getting and distributing data (e.g. all the status reports should be inserted 2 hours before the start of the STM)
ST, OPSMT
After the daily STM, the ST edits a small text file describing the outcome of the day. A briefing of fulfilled tasks, fault missions, planned tasks and any other useful information is included. To insert such an information into the OLC, the user must use this option.
ST, OPSMT
7. Hardware and software configuration
The OLC is designed to run in a UNIX environment, with the goal to provide a robust, stable and complete software framework to all the ACE-2 activities.
The system is PC based, running the free Linux 2.0 operating system and the free Apache Web server connected with a small relational database. The database stores all data and information needed, providing dynamic HTML page contents. The mediator between the Web server and the database is a CGI wrapper module embedded directly into the HTTP server. The Personal Home Page/Form Interpreter (PHP/FI) is a CGI wrapper that provides a large and powerful set of tools to make easy CGI programming and database management. It allows to embed special HTML tags into a HTML document to query a database and process query results. When a user accesses a PHP/FI HTML document, it is pre-processed by the wrapper, passed to the WWW server CGI interface as standard HTML document and then served to the WWW client.
Using such an architecture, the OLC has been organized as a set of PHP/FI modules, each one providing a specific functionality to the user. Each module holds both the user interface logic, the data and information processing, organization and distribution. User interfaces contain only standard HTML elements. No frames, no Java applets or ActiveX controls are used. The interface to the system is essential, easy to use and understandable by everybody. Just a little bit of JavaScript is used to allow a smart navigation between HTML pages (go back, go forward, go to the home page through the document history). To allow the user to enter into the system textual information and data files, a set of HTML forms containing both text fields, scroll lists, checkboxes and file upload fields have been employed. Through an upload field the user can attach a file to a report when is submitted to the Web server.
A basic safety model has been set up between the WWW client and the WWW server. When a user starts to use the system he/she must provide a user name and password to authenticate him/her to the server. Different user names/passwords have been prepared (OPSMT, ST, FT etc.). A user session will last 4 hours ; after that time, the user has to re-log in the system. To implement such a model and to bypass problems due to HTTP statelessness, a set of Netscape cookies have been employed. These cookies are exchanged during each HTTP transaction for all the duration of a user session. Each module starts checking the user name and password ; if it is invalid, the log in page is prompted to the user.
The OLC has shown to be a valid and powerful tool to support decision making, operations planning, data collection and distribution in a large field experiment such the ACE-2.
The OLC is a very effective system based on the actual standard WWW technology. HTML pages provide an intuitive and user friendly interface to the system. Server side CGI programming coupled with a relation database implement a dynamic document model, where each page changes respect the newest and latest information inserted. The WWW client/server model allows to collect and distribute data and information in a straightforward way.
On the other side, the heavy use of ad-hoc CGI programming to implement the whole software system, has shown some limits during software development. The development process has been slowed down due the need to develop custom software solutions to bypass the limits of the WWW model. The maintenance and the evolution of the system still require a big effort to take consistent all the modules.
Using the actual analysis, the OLC could be implemented using a set of modern off-the-shelf software tools, without danger to drown in it.
The OLC is still on-line for data browsing at the following address :