Submitting Data via Scripts

GMRI  is submitting real-time data for NERACOOS Buoys A, B, E, F, I, M and N.

Submitted Data is selected from the NERACOOS PostgreSQL database and the NERACOOS SOS Web service, formatted to the ODPX XML standard and submitted.

We've done this in 4 distinct steps.

  1. Submitted and Organization, Project/GCMD  ODPX XML  file to the node. (This file was hand edited and submitted with NodeClientLite2)
  2. Submitted a Monitoring Location ODPX XML file for each Buoy to the Node (that's 7 files generated by a script  and submitted with NodeClientLite2).
  3. Submitted an Activity ODPX XML file for each Deployment of a Buoy to the Node (that's 142 files generated by a script and submitted with NodeClientLite2 ).
  4. Once an hour grab data from our SOS service, create an ODPX XML file for the results and submit that (7 files generated by a script and submitted by a script).

 

The PerlExamples.zip file contains sample scripts for:

  • README.txt
  • ml_import.pl
  • activity_import.pl
  • GoMOOS_result_import2.pl

http://www.gomoos.org/schema/PerlExamples.zip

 

Examples for using these scripts:


From README.txt (contained in PerlExamples.zip

Example Perl Scripts for automatic XML file generation.

(And one Perl Script for a node submission)

 

NOTE: These examples are for illustration purposes only. They will not "run" as is. Do not hesitate to ask for help, clarification etc. There are many ways of achieving the same results with these "tools".

NOTE 2: The node Submission comes after the 3 examples of ODPX XML file creation.

There are 3 distinct examples. All of these assume that an Organization and Project/GCMD already exist. The examples create XML to add Monitoring Locations to an Organization, Activities to an Organization and Results to an Activity.

The Results script is much more complicated than the other two. It uses SOS GetCapabilities and SOS GetOffering to create an ODPX XML Result.

 

These examples are for adding GoMOOS buoy data to the ODPX node, your needs might be different. 

  • Monitoring Location is very infrequent.
  • Adding Activities happens twice a year ( at buoy re-deployment).
  • Adding Results is hourly.

1) Adding Monitoring Locations to an Organization

ml_import.pl

This script is an example of querying a database to create an ODPX XML document.  It uses odpx_ml_import_template.xml. After the template is "loaded" into the software the results of the database query are used to fill in relevant information.

 

Using MonitoringLocationIdentifier as an example. When the template is loaded contains nothing.

 <MonitoringLocationIdentifier></MonitoringLocationIdentifier>

The script uses the database query result to fill in a value, here "A01"

 <MonitoringLocationIdentifier>A01</MonitoringLocationIdentifier>

 

2) Adding Activities to an Organization

activity_import.pl

This script is an example of querying a database to create an ODPX XML document. It uses odpx_activity_import_template.xml. After the template is "loaded" into the software the results of the database query are used to fill in relevant information.

Using ActivityIdentifier as an example. When the template is loaded contains nothing.

<ActivityIdentifier></ActivityIdentifier>

The script uses the database query result to fill in a value, here "A0123"

<ActivityIdentifier>A0123</ActivityIdentifier>

Additionally this script will compress the resulting files into a "ZIP" archive which can be submitted with the Windsor NodeClientLite2.             

3) Adding Results to an Organization

 GoMOOS_result_import2.pl

This script is an example of querying an SOS services GetCapabilities to determine available Observation Properties ( parameters or characteristics), using those results to build an SOS service GetObservations request and then using those results to create an ODPX XML file for submission.

Additionally  this script will compress the resulting files into a "ZIP" archive which can be submitted with the Windsor NodeClientLite2. It is similar to the preivious 2 scripts in that it uses "fill in the blanks" but it will also "cut" and "paste" xml elements into the resulting XML file.

For example all the "fields" (observed properties) available are added to the template on the fly

     <swe:field name="wind_speed">

    <swe:Quantity definition="http://mmisw.org/cf#wind_speed">

        <swe:uom code="m.s-1"/>                   

    </swe:Quantity>

 

4) Node submission via a perl script.

SubmitFile.pl

This uses Perl and the WSO2 Perl module to submit a file to the Node. INSTALL_NOTES.txt contains information on installing the libraries needed to support this script. The installation can get complicated in a hurry....