How the data are structured

Data in the archive are organised within a hierarchy and identified via strict naming conventions.

• Overview of the data structure
• Hierarchy
  • Site
  • Station
  • Parameter
  • Time series
• Hydrological and Calendar Time intervals
• Long-term values
• Catalogue of time series names
  • Rain
  • Groundwater level
  • Level
  • Flow
  • Tidal level

The structure of data in the archive

The data, representing observed environmental information, are referenced by where, what, and how often, i.e.

• where were they observed - their position in the world,
• what was measured - the conditions they describe, and
• how often were observations made - this may also describe whether observations are regular or irregular.

Accordingly, each time series of individual data measurements may be identified by either of two systems:

1. a unique identifier, the time series ID; ts_id, or
2. a logical hierarchy of text identifiers that specify:
   • the location,
     • the spatial identifier
   • the environmental parameter,
     
     • the categorical identifier
   • how often individual data-points are available
     • this represents the temporal resolution of the data.

Either identification system can be used in an API request to specify which data to return.

N.B. the free text labels are not constrained to be unique; although, in general, station_name and ts_name values are unique.
In the case of stationparameter_name, 'Rain'  is used for both the RE and RS stationparameter_no categories.

The key hierarchical identifiers for data are:

• the Site
• the Station
• the Parameter and
• the Time series (type)

The time series type indicates what sample or statistic of the parameter is represented by the values in the time series, e.g. instantaneous value, day mean, annual maximum, &c.

Together, these key identifiers make up a path to the data - the ts_path. Analogous to the path to a file in a computer filesystem, the time series path can be used in certain API query functions to specify which time series data to return. When used in a data request, the ts_path can include wildcards to access data from a whole class of time series.

Example: a selection of time series data values with both their path identifiers and their time series IDs

Hierarchy

Site

'Site' is a broad category of location containing many separate measurement stations. Unlike SEPA's primary archive, stations in the API duplicate are ascribed to a single site: its site_no is 1 and its site_name is Stations. This renders the getSitelist query function largely redundant.

Station

Measurements are identified by the station at which they are observed. The station's name is  usually based on the name of settlement nearest to the station; it may include the type of gauge at the station, e.g. Hawick, Capenoch Raingauge, and Kingston Tide Gauge.

Explore API requests for station information, including identifiers and metadata amongst the endpoint examples.

Parameter

Each measurable environmental characteristic is identified as a parameter, for instance level, flow, or temperature. Each parameter is ascribed a plain English long name, stationparameter_name, but is identified uniquely by its abbreviated short name, stationparameter_no, or by its parameter_id.

Parameter information can be accessed either on its own or as metadata associated with station or time series data.

A list of available stationparameter_name and stationparameter_no values are shown below in the table of time series types.

Each parameter is stored in consistent units; rainfall in mm, level in m, flow in cumecs (m³.s^-1), temperature in Celsius.

Measurements of parameter values are stored in one or more time series.

Time series

The time series are defined by the time interval between the data in the series and the type of observation each measurement represents, either an instantaneous value or an aggregation over the period of the time interval. Time series are ascribed both a descriptive label, ts_name, and unique text identifier, ts_shortname, principally for use in coding.

As an example, instantaneous values measured at a 15 minute interval would be in a time series with ts_name 15minute and ts_shortname 15m.Cmd (in this case the Cmd is an abbreviation for 'continuously measured data'). For rainfall, where the value represents a total over a given interval the highest resolution data have ts_name 15minute.Total with corresponding ts_shortname 15m.Total.

Each parameter has its own structure of time series at various temporal resolution which are generally similar between parameters. The time series resolutions and aggregations are the same at every station for each parameter. A catalogue of the time series types available for each parameter may be found below.

The time series configured under each parameter are broadly the same, so there should be little difference in the available time series between stations per parameter. The exceptions to this rule are:

• For Flow parameters, the time series summarising some high flow data are not available where there is insufficient confidence in the quality
• For Daily observed rainfall data (parameter_no RS), time series for 15-minute and hourly data are unavailable as these data are not measured.

Hydrological and Calendar time intervals

Hydrological science uses the concept of a hydrological day, hydrological month, and hydrological year.

• The hydrological day runs from 09:00 on the given day to 09:00 on the following day
• The hydrological month runs from 09:00 on the 1^st of the month and runs up to 09:00 on the 1^st of the following month.
• The hydrological year starts on the 1^st of October at 09:00 and runs up to 09:00 on the 1^st of October the following year.

With reference to time series naming:

• HydrologicalYear is included at the start of the time series name for annual values calculated on the hydrological year.
  • The ts_shortname for hydrological time series starts H
• CalendarYear is included at the start of the time series name for annual values calculated on the hydrological year.
  
  • The ts_shortname for time series calculated on the calendar year start C, for example CYear.Total
• Hydrological or Calendar is not used in the time series names for monthly or daily time series, but 'H' is used in the ts_shortname.

These concepts have particular significance for aggregated derived time series.

The daily mean flow for a given date is the total discharge in a river between 09:00 on the given date up to 09:00 on the following. Similarly for rainfall the daily total rainfall is the total amount of rain that fell between 09:00 on the given day up to 09:00 on the following day.

Long-term Values

Long-term values are summary statistics available for certain parameters. These time series contain average, maximum, and minimum values for the period of record for each station. The time series have a general date that identifies the broad period that the statistic represents, and where applicable, an occurrence timestamp (sic), which identifies the specific occurrence time within the aggregation period of the statistic.

As an example, the river level Long-term Monthly Maximum represents the highest value recorded in each month of the year across the whole record and, hence, returns 12 values, with the dates on which they occurred.

https://timeseries.sepa.org.uk/KiWIS/KiWIS?service=kisters&type=queryServices&datasource=0&request=getTimeseriesValues&ts_path=1/116011/SG/LTV.HMonth.Max&metadata=true&period=P1Y&returnfields=Timestamp,Value,Occurrance Timestamp,Quality Code,Quality Code Name&dateformat=yyyy-MM-dd HH:mm:ss

Returned values:

Catalogue of time series names

The following tables describe the time series names associated with the available parameters.

• Rain
• Groundwater level
• Level
• Flow
• Tidal level

stationparameter_name : Rain (stationparameter_no : RE or RS)

* Not available for RS

stationparameter_name : GroundwaterLevel (stationparameter_no : GWL)

stationparameter_name : Level (stationparameter_no : SG)

N.B. this includes examples of the ts_path for each timeseries

stationparameter_name : Flow (stationparameter_no : Q)

(* Only available at selected stations) (** Flow is the primary vaue, level is available as the return field Stagesource Value)

stationparameter_name : TidalLevel (stationparameter_no : TL)

As for Level, above, plus the following.