Data extracts

Standard data sample format

Details of file format and data values provided in standard AIS data samples / Historical extracts

Spire Maritime AIS data samples are normally provided in CSV format with data fields as listed in the table of data fields below.

Different AIS Messages:

It is important to understand the data and in particular that the file contains data from 2 different type of AIS messages. AIS positions messages that report the position, speed, heading, rate of turn and status of vessels and AIS static voyage messages that report identity, type, size and voyage information.

These 2 types of data are linked only by the vessels MMSI number which is used as the vessel identifier for both.

Message Types 1, 2, 3, 18, 19, 27 are position messages and contain values in the position related fields.

Message Types 5 & 24 are static voyage messages and contain values in the static details and voyage related fields.

To identify the Name, IMO number or type of ships related to the AIS Position reports, the MMSI number must be used to join together the 2 different sets of AIS data., i.e. if position reports are recorded against MMSI 636018333 in AIS message type 1, then the IMO and name of that vessel is discovered by looking for AIS message type 5 reported using the same MMSI number as shown below.

Static position table

Distribution of values in key fields from different AIS messages:

Value counts AIS

Was this article useful? Yes No

Data columns description

created_at date: ISO 8601 formatted timestamp in UTC of the time the vessel record was created
timestamp string: ISO 8601 formatted timestamp in UTC of the time the AIS message was transmitted
mmsi integer: The Maritime Mobile Service Identity of the vessel transmitting the AIS message; Possible values: 000000000 – 999999999
msg_type integer: AIS message type; Common values: 1, 2, 3, 5, 18, 19, 24, 27
latitude float: Vessel latitude in degrees (North=positive, South=negative); Range: -90 to +90
longitude float: Vessel longitude in degrees (East = positive, West=negative); Range: -180 to +180
speed number: Vessel speed over ground represented in knots; Possible values: 0 - 102.2 knots, 102.3 (not available)
course number: Vessel course over ground in degrees; Possible values: 0 - 359.9 degrees, 360.0 (not available)
heading number: Vessel true heading in degrees; Possible values: 0 - 359 degrees, 511 (not available)
rot integer: Vessel rate of turn; Possible values: -127 - 127; -128 (not available)
imo integer: IMO number of the ship; Unique International Maritime Organization number for the vessel that stays with the ship for it’s life; Valid values: 7 digit number
name string: Vessel name
call_sign string: Vessel call sign
flag string: Vessel flag
draught float: Vessel draught represented in 1/10 meters; Possible values: 0.1 – 255, 0 (not available; default)
ship_and_cargo_type integer: Vessel ship and cargo type code; Some common values: 30 (fishing vessel), 52 (tug boat), 70 (cargo/fishing ship) (read our AIS Essentials article on determining a ship type)
length number: Vessel length extracted from ship dimensions to_bow and to_stern in meters
width number: Vessel width extracted from ship dimensions to_port and to_starboard in meters
eta string: Vessel estimated time of arrival as entered by the captain, represented in ISO 8601 format; Possible values: Month: 1 – 12, 0 (not available; default); Day: 1 – 31, 0 (not available; default); Hour: 0 – 23, 24 (not available; default); Minute: 0 – 59, 60 (not available; default)
destination string: Vessel destination as entered by the vessel captain
status string: Vessel navigation status; Some common values: 0 (under way using engine), 1 (at anchor), 3 (restricted maneuverability), 7 (engaged in fishing), 15 (read the AIS essentials article on navigational status for more information)
maneuver integer: Vessel maneuver code; Valid values: 0 (not available; default), 1 (not engaged in special maneuver), 2 (engaged in special maneuver)
accuracy integer: Vessel GPS geo location accuracy in meters; Possible values: 1 (high, <=10 meters); 0 (low, >10 meters, default)
to_bow integer: Distance from AIS transponder to the bow side edge in meters (Hull dimension A); available from March 2020 onwards
to_stern integer: Distance from AIS transponder to the stern side edge in meters (Hull dimension B); available from March 2020 onwards
to_port integer: Distance from AIS transponder to the port side edge in meters (Hull dimension C); available from March 2020 onwards
to_starboard integer: Distance from AIS transponder to the starbord side edge in meters (Hull dimension D); available from March 2020 onwards
collection_type string: How the message was captured; Possible values: satellite or terrestrial or dynamic

Was this article useful? Yes No

File delivery

Unless a small sample is requested that can be delivered by E-mail then AIS history files are provided through a Google Cloud Storage (GCS) bucket.

Access to GCS requires a google managed account that can be granted access to the bucket and it is advised to use the Google Cloud SDK to download the files.

Was this article useful? Yes No

Expected AIS data volumes

To estimate file sizes for provision of historical AIS data we need to know the period and the sources of AIS being provided.,

As of 2021-08-30 1 day of satellite and terrestrial AIS, with down sampled terrestrial AIS created a set of data files that when compresses were 34Gb in size. Dynamic AIS would add about 12.5% creating a daily file archive of approximately 38Gb.

The system used to spool historical AIS files chunks requested data into multiple files, normally about 300mb in size. So as well as the total file size, the number of files received will be determined by the size of the AIS history provided.

Global data volumes

As of September 2020, Spire Maritime’s AIS volumes were about 180 million records per day as full, non-downsampled, non-cleaned data, and about 50 million messages per day when downsampled, which means up to 64 billion records a year, which is why proper filtering of API calls according to usage is essential.

We ran a 1 day, standard 5 minute downsampling test, including Satellite & Terrestrial AIS, for Tankers & Cargo vessels as identified by AIS which yielded the following results:

Satellite AIS: 8,131,365 total messages; 8,111,980 positions; 29,922 distinct MMSIs
Terrestrial AIS: 13,567,818 total messages; 6,763,977 positions; 53,779 distinct MMSIs
Combined Terrestrial and Satellite AIS: 21,699,183 total messages; 14,875,957 positions; 68,826 distinct MMSIs

The exported the static data as CSV files represented 30 x 300Mb files, which represents about 3Gb for 1 day S+T data.

Were this data pulled from the API, the data volumes would be much larger because the JSON format used for API responses is more expansive than the compressed CSV format used in the test.

In a 24h period test using the Messages API and the MMSI numbers from a selection of 100 tankers returned over 200,000 messages, in pages of up to 20,000 messages each. The file showing that first page of API results is around 19Mb in size.

Vessels API data volumes can be even larger than Messages API, since it also contains additional added value data such as Vessel Characteristics. The results for a 100 vessels (all tankers) test, including Vessel Characteristics, is about 700K.

Maritime 2.0 architecture means you can query and receive much more targeted data and avoid unnecessary data bloat, which is one of the reasons why you should privilege this more scalable GraphQL architecture.

Was this article useful? Yes No