Support Portal

Data loss: Possible reasons and detection in the data

Related Products
multiScan100

Table of Contents

Different data loss types

Streaming data can be lost in the system due three different reasons:

  • Queue overflow errors: Queue overflow errors occur when the system tries to put more data items (e.g. scan segments or depth maps for the field evaluation) in the processing queue of an app than the queue can hold. If this is the case the items which do not fit into the queue anymore are discarded. Details on the conditions when queue overflow errors occur can be found here: Details on queue overflow errors
  • ScanData lost errors: It is possible, that problems occur already during the creation of scan segments in the house keeper or even on the rotor. If only the data for few beams are missing, these are filled with zero values. If no valid measurement data is obtained for an angle range larger than 90° no segments are created anymore. In both cases a recoverable error type 2 is raised and resolved as soon as the next data item (without missing beams) arrives.
operating mode motor sender / receiver (e.g. laser) primary sensor data

Aggregation

secondary sensor data

application heating device ready status SICK common device state

recoverable error

category 2

on on off off off active if necessary device NOT ready "7" recoverable error




The notification in case of that error are shown in the table below:

 

Missing measurement data 
  • Package loss: It can happen that data is lost on its way from the sensor to the client. Reasons for that can be:  
    • The sensor is not able to transmit a data package. We expect that the probability of such a problem is very low since the transmit buffer of the sensor is rather large and UDP puts data on the transmission medium without checking for collisions, i.e. there are not re-transmits which could lead to a buffer overflow.
    • The data package is lost on the transmission medium due to collisions or other problems of the transmission medium. Note that in UDP there is no re-transmit mechanism in contrast to TCP.
    • The data package can not be received by the receiver because the receiver is busy. This is likely to occur for receivers with low performance or when using inefficient drivers since the event frequency is 240Hz (20Hz rotation frequency * 12 segments per rotation) in average and peak frequencies can be much higher since the data items in filled queues can be transmitted much faster in some cases.
 

Detecting the losses in the data

To detect data loss in the streaming data the following fields in the output data are important (all fields are available in both, MSGPACK and Compact data):

  • SegmentCounter
  • FrameNumber
  • TelegramCounter

 

Queue overflow errors or ScanData lost errors can be detected when there is a gap in the SegmentCounter/ FrameNumber sequence but NOT in the TelegramCounter.

Examples:

 

Queue overflow errors or ScanData lost errors can be detected when there is a gap in the SegmentCounter/ FrameNumber sequence but NOT in the TelegramCounter.

Examples:

Hint: To easily detect gaps in the SegmentCounter / FrameNumber sequence one can compute a combined counter and search for gaps in that counter: CombinedCounter = FrameNumber* NumberOfSegmentsPerFrame+SegmentCounter

 

Package losses are identified by a gap in both the segment counter/ frame number sequence and the telegram counter sequence. To detect package losses it is enough to check only for gaps in the telegram counter.

Example:

SegmentCounter; FrameNumber; TelegramCounter SegmentCounter; FrameNumber; TelegramCounter

0; 0; 0

1; 0; 1

2; 0; 2

3; 0; 3

4; 0; 4

6; 0; 6

7; 0; 7

8; 0; 8

9; 0; 9

0; 0; 0

1; 0; 1

2; 0; 2

3; 0; 3

4; 0; 4

5; 0; 5

4; 1; 16

5; 1; 17

6; 1; 18

7; 1; 19

8; 1; 20

9; 1; 21

10; 1; 22

11 ; 1; 23

 

Behavior of the 3d object detection in case of data loss

If scan segments are lost due to queue overflows or scan segment lost errors, the object detection will work on images with invalid data but will NOT report an error. That means that one must not rely on the object detection output if an error 910 is active ("Too many functions are active at the same time, so not all data is output anymore. Deactivate functions that are not required.").
Keywords:
data loss, queue overflow, scan data loss, package loss