Using the Spire Maritime AIS TCP Feed V2

How to use the Spire Maritime AIS TCP feed V2.

This FAQ outlines how to use the Version 2 (V2) version of the Spire Maritime AIS TCP Feed. This replaces the V1 TCP stream. It is intended to be backwardly compatible with the V1 service but may require different authentication tokens. 

Connect to the Spire Sense TCP Server

To be able to use the Spire AIS TCP Service, an authorization token needed, which is used to authenticate the service.

Initiating the TCP Feed: Initiate Connection.

For TCP connections, client systems must open a TCP socket connection to the Spire service on the     Service Address :  streamingv2.ais.spire.com

using Port number: 56784.

To initiate the stream of messages, and initial authentication string must be sent as below:

A|T|<auth_token_goes_here>

Note: Depending on which system you are using, you might want to send a new line ‘\n’ to make sure that the token is sent in one piece directly

If connection is successful then the stream of AIS data in NMEA format will start as shown in the example below

nc streamingv2.ais.spire.com 56784
A|T|USER_TOKEN_HERE
\s:terrestrial,c:1615991449*52\!AIVDM,1,1,,B,B6:bqEh00B1qq7SBon1Tp0HT0000,0*38
\s:terrestrial,c:1615991451*5B\!AIVDM,1,1,,B,133<6J`P00v:QILEV0=v4?wV25ip,0*20
\s:terrestrial,c:1615991451*5B\!AIVDM,1,1,,A,83aDqqhj2d<dd=v<u0I0N@J53900,0*50

Above shows the service connection and the first 3 results of AIS messages being returned.

Possible initial delay.

Once your connection has been established, AIS messages should begin streaming into your client. Note: It can take up to 60 seconds for the TCP server to start up a new stream after sending your token. 

Recommended Reconnection Strategy

The TCP server should keep your connection alive indefinitely and send keep-alives every 15 minutes to your client, however, exceptions still happen. Occasionally there might still be instances of unexpected disconnects to the TCP feed. In these cases, you can simply reconnect, re-authenticate with your token and the data stream will resume.

In order to minimize loss of data and impact on latency, we recommend setting up a reconnection method that automatically acts within an hour of any unexpected disconnect.

Interruptions continue where left off

The behaviour of the TCP service is to provide a continuous stream of AIS messages in the order that they are received by Spire systems. If the service stops, due to disconnection or client system problems, then if re-connecting within 7 days from disconnection, the stream of data will continue from the time of disconnection and no data will be missed.

If the service is disconnected for more multiple hours or days then it can cause a large backlog of now old data to be streamed before the service catches up to current time and entirely new messages are received. 

If the service is disconnected for more than 7 days then data will be streamed from the current time and only new messages are received.  In this situation please contact Spire Maritime sales engineering here to request a backfill of historical data that has been missed. 

Updating the service checkpoint

In the V2 TCP stream service it is possible to update the service checkpoint  that species where in the stream of data service will continue from.

Setting checkpoint to a specified time:

To reset your TCP stream service to start from a particular point in time, pass the authentication token string as normal followed by a date time string in the format YYYY-MM-DDTHH:MM:SS.MMMZ. Note the time must be in UTC (zulu time)

Example authentication to set the TCP stream checkpoint to a given time

A|T|<your-token>|2021-03-08T11:45:09.840Z` 

Note the checkpoint  can be set up to 7 days in the past and times most be specified in UTC

example below

nc streamingv2.ais.spire.com 56784
A|T|USER_TOKEN_HERE|2021-03-17T16:45:09.840Z

Above shows the service checkpoint reset request after which the connection is reset.
To stream data, connection must be established again

 

Setting checkpoint to the latest message time:

To reset your TCP stream service to start from a the time of the most recent message in your service, pass the authentication token string as normal followed by the label resetToLatest

Example authentication to set the TCP stream checkpoint to the latest message time

A|T|<your-token>|resetToLatest 

example below

nc streamingv2.ais.spire.com 56784
A|T|USER_TOKEN_HERE|resetToLatest 

Above shows the service checkpoint reset request after which the connection is reset.
To stream data, connection must be established again

 

[Still Under Development] Setting checkpoint to a earliest message time:

To reset your TCP stream service to start from the earliest time available in your service, pass the authentication token string as normal followed by the label resetToEarliest

Example authentication to set the TCP stream checkpoint to the latest message time

A|T|<your-token>|resetToEarliest

 

Forced reconnect after updating the service checkpoint

After setting a checkpoint timestamp or using one of the checkpoint reset labels resetToLatest, resetToEarliest the TCP service will update the checkpoint and disconnect.

You will need to connect again and pass auth string as usual `A|T|<your-token>` to get the data from the new checkpoint.

Multipart messages

AIS message 5 is commonly received in multiple parts, generally 2 message sentences. Spire AIS stream v2 provides grouping tags as shown below. Optionally for users who require it, this feature may be disabled upon request to Spire Maritime sales engineering here 

An example of multi part AIS message 5 showing the NMEA compliant group tag

\g:1-2-1,s:terrestrial,c:1615816351*11\!AIVDM,2,1,1,A,53kiP6023Ps@=5Hh000T@60>p00000000000000l1@E:140Ht00000000000,0*3B
\g:2-2-1*6C\!AIVDM,2,2,1,A,00000000000,2*25
\g:1-2-2,s:terrestrial,c:1615816351*12\!AIVDM,2,1,2,B,543Kbi000000DpEJ220M>oS8000000000000001S3HJ366ShN6kRBkk0AE`8,0*2E
\g:2-2-2*6F\!AIVDM,2,2,2,B,88888888880,2*25
\g:1-2-3,s:terrestrial,c:1615816351*13\!AIVDM,2,1,3,B,53mA9<42=Ti`hm<B220<thu:0H4q@5=V2222220t3c3BB6iWNACSklk88888,0*3F
\g:2-2-3*6E\!AIVDM,2,2,3,B,88888888880,2*24
\g:1-2-4,s:terrestrial,c:1615816351*14\!AIVDM,2,1,4,A,53aLlsP0000109U@0008A>10Thu@00000000000j1@63340003@000000000,0*7D
\g:2-2-4*69\!AIVDM,2,2,4,A,00000000000,2*20
\g:1-2-5,s:terrestrial,c:1615816351*15\!AIVDM,2,1,5,A,544alp400001DU`6220<50U@4r1`4@u8u`PqV20l1HB53800094iEPDm3l3k,0*4E
\g:2-2-5*68\!AIVDM,2,2,5,A,88888888880,2*21

Sample TCP Stream results

Example TCP stream results
\s:terrestrial,c:1615816351*52\!AIVDM,1,1,,A,342O;g@P@61tAMRF00EK@8;00>@<,0*34
\s:terrestrial,c:1615816351*52\!AIVDM,1,1,,B,144i@9P0002f57bIPLw>46vv2>@<,0*3E
\s:terrestrial,c:1615816309*5F\!AIVDM,1,1,,B,152blF0PADIGC@t@hw=sO9;R0p:f,0*55
\s:terrestrial,c:1615816351*52\!AIVDM,1,1,,B,13aFdSOP0OPO::6MqC@3ugvv2>`<,0*30
\g:1-2-9,s:terrestrial,c:1615816351*19\!AIVDM,2,1,9,B,53c4vr82<ciI0D<N220DhU<48E@R222222222217=@<99toa0=nQA@TUAiiO,0*16
\g:2-2-9*64\!AIVDM,2,2,9,B,R5C38888880,2*39
\g:1-2-10,s:terrestrial,c:1615816351*21\!AIVDM,2,1,0,B,53c4vr82<ciI0D<N220DhU<48E@R222222222217=@<99toa0=nQA@TUAiiO,0*1F
\g:2-2-10*5C\!AIVDM,2,2,0,B,R5C38888880,2*30
\s:terrestrial,c:1615816351*52\!AIVDM,1,1,,B,13GQ3J?OiewluSJHkUO@HP:v0D10,0*1B
\s:dynamic,c:1615815280*4C\!AIVDM,1,1,,A,B3:GbB006B=KfIU;cUAfcwP00000,0*2C
\s:dynamic,c:1615815299*44\!AIVDM,1,1,,A,15D2E4?P005gV8F7OW``IVEp0000,0*35
\s:dynamic,c:1615815447*41\!AIVDM,1,1,,A,14`Ut8gP00Ll``IjiVgu?a7p0000,0*5A
\s:dynamic,c:1615815539*49\!AIVDM,1,1,,A,17liV?OP008<IrWv5qMtg?wp0000,0*6F
\s:dynamic,c:1615815557*41\!AIVDM,1,1,,A,13:1wR?P301tHkNRF4wIC7Op0000,0*1B
\s:44405,c:1615815822*0A\!AIVDM,1,1,,A,14eG;oh2@0o;eDtL@>37Sn140@Do,0*2B
\s:44405,c:1615815717*03\!AIVDM,1,1,,B,4030p<QvDoeainQABdNN8=7005bT,0*66
\s:44405,c:1615815943*0C\!AIVDM,1,1,,B,14eHUbsP1>o<P8PL6Vt2cww:08EW,0*54
\s:44405,c:1615815898*0B\!AIVDM,1,1,,B,1815;0001lD=tTTM32?SB2eV05bh,0*1F
\s:44405,c:1615815976*0A\!AIVDM,1,1,,B,B4eGR9000=fSWF7?;aoQ0c4QnDNr,0*7F
\s:44405,c:1615815977*0B\!AIVDM,1,1,,B,14QSvp7000l96t8NvGnto`:F0<0B,0*64

Understanding the AIS message stream format above 

group tag \g:1-2-5 and \g:2-2-5
Group tag has 3 parts X-Y-Z.
X is the fragment number
Y is the count of message fragments
Z is the sequence number that joins the fragments together

source tag \s indicating the AIS source of the received AIS message
Values are:
terrestrial for AIS received from terrestrial, land based, AIS receivers.
dynamic for AIS received from dynamic AIS, vessel based, AIS receivers.
<satellite id> for messages received from Spire satellites.
Note satellite id will be the NORAD ID of the satellite
or if the NORAD ID is not yet confirmed then the Spire satellite name

message timestamp and tags checksum \c:1601972308*58
message header !AIVDM
message sentence count 2
message sentence number 1 or 2
message number 5
AIS channel A or B
Encoded AIS data and checksum (the rest of the message record)

Example Python to connect

A simple python program for connecting to the Spire TCP service is available for testing from this Google Drive link https://drive.google.com/drive/folders/1UHH7vGTVZCxqrATQGWsMkFtV0RdPt4Nx?usp=sharing

It can be called  in the following fashion (using a demo or client authentication token)

python call_spire_tcp_feed_parameters.py test_tcp.txt 100 streaming.ais.spire.com 56784 2 <CLIENT AUTHENTICATION TOKEN>

Example

python call_spire_tcp_feed_parameters.py test_tcp.txt 100 streaming.ais.spire.com 56784 2 eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjdXN0b21lciI6eyJpZCI6IjcxMSIsIm5hbWUiOiJTcGlyZSBUQ1AgREVNTyBET1ZFUiIsInV1aWQiOiI3MTEifSwiaXNzIjoic3BpcmUuY29tIiwiaWF0IjoxNTk1NDA0MTUxfQ.kOiZqAYH2-uY6EJZHLKwuqyMhP763CttvFm8DuQZQW8

Sample Results

python3 call_spire_tcp_feed_parameters.py t
est_tcp.txt 100 streaming.ais.spire.com 56784 2 eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjdXN0b21lciI6eyJpZCI6IjcxMSIsIm5hbWUiOiJTcGlyZSBUQ1AgRE
VNTyBET1ZFUiIsInV1aWQiOiI3MTEifSwiaXNzIjoic3BpcmUuY29tIiwiaWF0IjoxNTk1NDA0MTUxfQ.kOiZqAYH2-uY6EJZHLKwuqyMhP763CttvFm8DuQZQW8
Program Start Time: 2020-07-22T09:55:43+02:00
server set to parameter value: streaming.ais.spire.com
Run TCP client for 2 minutes
Debug set to default of False - False
Parameter Debug = : False
Using API Token from parameters
Using API Token: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjdXN0b21lciI6eyJpZCI6IjcxMSIsIm5hbWUiOiJTcGlyZSBUQ1AgREVNTyBET1ZFUiIsInV1aWQiOiI3MTEi
fSwiaXNzIjoic3BpcmUuY29tIiwiaWF0IjoxNTk1NDA0MTUxfQ.kOiZqAYH2-uY6EJZHLKwuqyMhP763CttvFm8DuQZQW8
About to try connecting to:
Server: streaming.ais.spire.com on port: 56784

Writing text data stream to file test_tcp.txt20200722-075543.txt
Writing nmea data stream to file test_tcp.txt20200722-075543_nmea.txt
Setting job message limit : 100
Connecting..
program has run for 0.0 minutes. 20200722-075546 Last Minute captured 1 records.
Stopping as max_records number reached: 100
Stopping as max_records number reached: 100
Stopped after receiving 100 messages 100
API Processing ran from 20200722-075543 to 20200722-095546 received 100 lines of results

Program run from 20200722-075543 to 20200722-095546 took (0.0, 2.779322) minutes

Start time as unix time: 1595397343.0
End time as unix time: 1595397346.0
Elapsed time in seconds: 3

wc -l test_tcp.txt20200722-075543_nmea.txt
100 test_tcp.txt20200722-075543_nmea.txt

Showing the captured results

head -5 test_tcp.txt20200722-075543_nmea.tx
t
20200722-075546,b'\\c:1595382932*50\\!AIVDM,1,1,,B,13Hj;iPP00P7Am0M1`cN4?w02>`<,0*01\r'
20200722-075546,b'\\c:1595382943*56\\!AIVDM,1,1,,A,33QFvD500007=4nM1a0raF5F0>`<,0*3F\r'
20200722-075546,b'\\c:1595382957*53\\!AIVDM,1,1,,A,B3HjFSP0Fh1ba>7A4HFmwwtUoP06,0*1B\r'
20200722-075546,b'\\c:1595382966*51\\!AIVDM,1,1,,B,B3`erch0081Otk7D0MwQ3wS1nE6b,0*00\r'
20200722-075546,b'\\c:1595382969*5E\\!AIVDM,1,1,,B,13aFfS@P00P7Dc8M1VSN4?vB2>`<,0*05\r'

 

For further questions or support please submit questions to the Spire Sales Engineering team using our client web support form here https://spire.com/developers/support/