Basics

Freematics Hub is a telemetry data server which is highly efficient and works on systems of any footprint. The design goal is to seamlessly connect multiple telemetry data logging devices with applications that deals with telemetry data.

Freematics Hub receives data from remote telemetry devices, caches in memory, stores in disk files, and supplies the data in an organized way on request. As it caches data for a long duration, data logging devices and data processing programs can work at a different pace without missing any data samples. Data from multiple sources can also be aggregated and processed together with least efforts. Data are transmitted from device as UDP datagrams or HTTP transactions and available to peers requiring them by simple HTTP/REST APIs.

 

Feeding Data (from device to server)

A device collects data and feeds them to Freematics Hub by sending UDP datagrams or continuous HTTP GET or POST requests. The sequence is straight-forward as following.

  1. Notify server that a device is start data feeding (login)
  2. Send data to server (repeating)
  3. Notify server that a device has stopped data feeding (logout)
  4. Ping server with an interval to inform server that the device is online

A low-overhead payload format is defined for communication between server and device.

Event notification (UDP datagram)

<Device ID>#EV=<event ID>,SK=<server key>,TS=<device timestamp>,VIN=<vehicle ID>*<checksum>

* Server key is currently not used and can be omitted

Data transmission (UDP datagram)

<Device ID>#<PID 1>:<data 1>,<PID 2>:<data 2>,<PID 3>:<data 3 value 1>;<data 3 value 2>;<data 3 value 3>,…$<checksum>

 

Login and Logout

A device notifies server of events like login and logout. An event ID is a numeric value which indicates the type of event. A event notification can be sent by UDP or HTTP GET.  Login event (event ID 1) indicates the data transmission session is about to start and some identifications are submitted including Vehicle identification number (VIN) is used as A unique identifier. Logout event (event ID 2) indicates the data transmission session is about to end, most likely as a trip is over.

Notifying by UDP

A typical notifying request by UDP has the payload like following.

ABCDEFG#EV=1,TS=39539,VIN=A1JC5444R7252367*XX
  • ABCDEFG is device ID
  • EV=1 indicates a login event
  • TS=… the current timestamp (normally the device timer counter)
  • VIN=… specifies the VIN which should be unique and is used to allocate Feed ID
  • *XX is the checksum (in hex) of all previous bytes

After server receives a UDP request, it replies a UDP datagram carrying a payload like following as a reply.

1#EV=1,RX=n,TS=39539*X
  • First number (in Hex) is the feed ID (arbitrary)
  • EV=n is the event ID from request
  • TS=… is the timestamp from the request
  • RX=… is the number of received requests/data transmissions
  • *XX is the checksum (in hex) of all previous bytes

Notifying by HTTP

Notifying request can also be sent by HTTP or more secure HTTPs request with the following request format. The definitions of parameters are identical to that with UDP.

http://<server host>/api/notify/[DEVICE ID]?EV=<event ID>&TS=<device timestamp>&VIN=<vehicle ID number>

HTTP Response on success

{"result":"done","id":n}

n is an arbitrary integer number which is used for subsequent data feeding referred as Feed ID.

HTTP Response on error

{"result":"failed","error":"Invalid server key"}
{"result":"failed","error":"Invalid VIN"}

Feeding data by UDP

Using UDP to feed live data is the most efficient for a device with a not always stable network communication. In this way, payloads containing a group of collected data and checksum are transmitted as UDP datagrams. For details of the payload format, refer to Freematics Packed Data Format.

For UDP transmission, a simple header containing device ID and a tail of checksum are added like following. Both are represented in hexadecimal.

<Device ID>#<PID 1>:<value 1>,<PID 2>:<value 2>,<PID 3>:<value 3>,...*<Checksum>

Example:

ABCDEFG#0:68338,10D:79,30:1010,105:199,10C:4375,104:56,111:62,20:0;-1;95,10:6454200,A:-32.727482,B:150.150301,C:159,D:0,F:5,24:1250*7A

Feeding data by HTTP GET

Using a HTTP GET request to feed live data is easy for testing (even from browser). Multiple PID and value pairs are accepted as URL arguments for one request. PID numbers are in hex format and values are in decimal format (integer, float or semicolon separated values). It is required to transfer data with same timestamp by one HTTP GET transaction.

Request URL

http://<server host>/api/push/[DEVICE ID]?TS=[timestamp]&[pid1]=[value1]&[pid2]=[value2]...

Response (success)

{"result":n}

n is the number of accepted PID/value pairs.

Response (erroneous)

{"result":"failed","error":"Invalid server key"}
{"result":"failed","error":"Invalid FEED ID"}

 

Feeding data by HTTP POST

Using HTTP POST to feed data has the both advantages of security and flexibility. A group of data with different timestamps can be transmitted by one request like UDP.

Request URL

http://<server host>/api/post/[DEVICE ID]

Post Payload

The payload, which is made up of a series of time-stamped data, is formed in Freematics Packed Data Format.

Response (success)

{"result":n}

n is the number of accepted data PID/value pairs.

Response (erroneous)

{"result":"failed","error":"Invalid server key"}
{"result":"failed","error":"Invalid FEED ID"}

 

Fetching Data (from server)

Software which is designed to illustrate, process, analyze or permanently store data, uses a set of HTTP SOAP API requests to fetch data from Freematics Hub. Freematics Hub’s GUI is implemented by using the API.

 

Obtaining list of data feeds

Request URL

http://<server host>/api/channels

Response

The response is an object array in which each element is corresponding to a data feed.

{"channels":[
{"id":"1","devid":"A3HNZMJG","recv":26253,"tick":505269,"age":2250,"flags":1,"csq":22},
{"id":"2","devid":"ES9129GD","recv":432432,"tick":1205269,"age":1125,"flags":1,"csq":25},
...
]}

Properties:

  • id – Feed ID
  • vin – vehicle identification number
  • recv – data received (bytes) during current session/trip
  • tick – device system clock tick (ms)
  • age – time elapsed (ms) since latest data entry received till it’s requested
  • csq – signal strength index (in dB)
  • flag – 1 for active/running state, 0 for inactive/parked state
 

Fetching instant data

This is to request live data from a data feed.

Request URL

http://<server host>/api/get/[DEVICE ID]

Response

{"stats":{"tick":431865078,"devtick":818627,"elapsed":70,"age":{"data":3234,"ping":0},"parked":0},
"data":[[32,[0,0,0],3234],[36,1229,3234],[130,44,18300],[260,87,3234],[261,61,13283],[266,549,3234],[268,7500,3234],[269,0,3234]]}

In the response, stats contains statistics information including:

  • tick – server timestamp (in ms)
  • devtick – device timestamp (in ms)
  • elapsed – trip/session runtime
  • age.data – time (in ms) elapsed since last data transmission
  • age.ping – time (in ms) elapsed since last ping back (when device in standby mode)

The array data contains all the instant data represented in a 3 elements array including:

  • PID – in decimal
  • data value (could be integer, float or array of 3 elements for 3-axis data)
  • age – time (in ms) elapsed since its collection from on device end

Operation Sequence

The basic sequence of fetching instant data is like following.

  1. Getting data feed list (containing ID and some properties of a data feed)
  2. Fetching instant data (with Feed ID)
  3. Parsing response JSON payload and deal with the data
  4. Repeating 2 and 3

Fetching history data

A data queue is a serious of data with time stamp. The request for a data queue can specify time stamp range or time to rollback. The data is organized as a 2-dimension JSON array. This is useful for rendering a data chart or route on map with complete data samples, so the line on chart or the shape of route looks smooth. This is also necessary for caching or storing data from a data feed.

Request URL

http://<server host>/api/pull/<DEVICE ID>?[ts=<ms>][&endts=<ms>][&rollback=<ms>]

Request Arguments

  • ts is used to specify start time of requested data queue. The time is presented in device system clock tick in ms.
  • endts is used to specify end time of requested data queue. The time is presented in device system clock tick in ms.
  • rollback sets start time with relative time in ms without need for knowing device local clock tick. For example, rollback=60000 indicates requesting for data from 1 minute ago.

Response (success)

The response contains statistic data and the requested data queue represented as an array of 3-element sub-array (timestamp, PID, data). Timestamp is from device system clock in ms. PID is in decimal instead of hexadecimal. Value is in string format. A response example is shown below.

{"stats":{"tick":93385265,"recv":3754583,"elapsed":93365,"age":546,"flags":1},
"data":[[92690810,269,"71"],[92690810,260,"74"],[92690810,273,"68"],[92690810,261,"55"],
[92690810,32,"4/1/97"],[92690881,36,"1290"],[92692981,48,"1765263"],[92693184,268,"2500"],
[92693184,269,"71"],[92693184,260,"74"],[92693184,273,"65"],[92693184,271,"97"],[92693184,32,"4/1/97"],
[92693231,16,"3395000"],[92693231,10,"-33.721328"],[92693231,11,"151.163440"],[92693256,36,"1280"],
[92694010,268,"2500"],[92694010,269,"71"],[92694010,260,"78"],[92694010,273,"68"],[92694010,261,"55"],
[92694010,32,"4/1/97"],[92694081,36,"1290"],[92694809,268,"2500"],[92694809,269,"71"],[92694809,260,"74"],
[92694809,273,"68"],[92694809,271,"97"],[92694809,32,"4/1/97"],[92694880,36,"1290"],[92695609,268,"2500"],
[92695609,269,"71"],[92695609,260,"74"],[92695609,273,"65"],[92695609,261,"55"],[92695609,32,"4/1/97"],
[92695680,36,"1290"],[92698173,48,"1765362"],[92698354,268,"2500"],[92698354,269,"71"],
[92698354,260,"78"],[92698354,273,"65"],[92698354,271,"97"],[92698354,32,"4/1/97"],
[92698401,16,"3395500"],[92698401,10,"-33.721328"],[92698401,11,"151.163440"],[92698426,36,"1290"],
[92699224,268,"2500"],[92699224,269,"71"],[92699224,260,"74"],[92699224,273,"65"],[92699224,261,"55"],
[92699224,32,"4/1/97"],[92699294,36,"1290"],[92700001,268,"2500"],[92700001,269,"71"],[92700001,260,"78"],
[92700001,273,"65"],[92700001,271,"97"],[92700001,32,"4/1/97"],[92700071,36,"1290"],[92700823,268,"2500"],
[92700823,269,"71"],[92700823,260,"78"],[92700823,273,"68"],[92700823,261,"55"],[92700823,32,"4/1/97"],
[92700894,36,"1280"],[92703366,48,"1765460"],[92703547,268,"2500"],[92703547,269,"71"],
[92703547,260,"78"],[92703547,273,"68"],[92703547,271,"97"],[92703547,32,"4/1/97"],
[92703594,16,"3400000"],[92703594,10,"-33.721328"],[92703594,11,"151.163440"],[92703619,36,"1290"],
[92704396,268,"2500"],[92704396,269,"71"],[92704396,260,"74"],[92704396,273,"68"],[92704396,261,"55"],
[92704396,32,"4/1/97"],[92704466,36,"1290"]],
"eos":true}

Description of properties:

  • stats.tick – device system clock tick (ms)
  • stats.recv- data received (bytes) in current trip
  • stats.elapsed – time elapsed (seconds) in current trip
  • stats.age – time elapsed (ms) since latest data entry received till it’s requested
  • stats.flag – 1 for driving/active, 0 for parked/inactive
  • data[] – data queue represented as an array of 3-element array consisting of timestamp, data type and data value
  • eos – indicates if this is the end of stream (no more data) at the time the request is made

Response (erroneous)

{"result":"failed","error":"Invalid server key"}
{"result":"failed","error":"Invalid FEED ID"}

Operation Sequence

The sequence of fetching complete data queue is like following.

  1. Getting data feed list (containing Feed  ID and some properties of the data feed)
  2. Fetching a data queue with rollback time specified
  3. Parsing response JSON payload, deal with the data and keep the time stamp of last data sample
  4. Fetching next data queue with time stamp specified (start time set as last data sample plus 1)
  5. Repeating 3 and 4

Command and Query

Commands can be sent to device remotely for performing specific actions or requesting for data. Once a command is sent, a token number is returned. Subsequent requests are to be made for checking the completion of the command and readiness of result data. Multiple commands are queued.

This feature is implemented in Freematics Hub version 0.9.5 and later.

Sending command

Request URL

http://<server host>/api/command/<DEVICE ID>?cmd=<command name>

Response (success)

{"result":"pending","token":<token number>}

Response (erroneous)

{"result":"failed","error":"Command unsent"}

{"result":"failed","error":"Invalid request"}

Checking command status

Request URL

http://<server host>/api/command/<DEVICE ID>?token=<token number>

Response (completed)

{"result":"done","elapsed":<time elapsed for completing command>,"data":"<response message or result data>"}

Response (not completed)

{"result":"pending","elapsed":<time elapsed for the command>}

Response (erroneous)

{"result":"failed","error":"Invalid token"}