| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
saw in live data type=1 records with length of 163
right after 168 with same data payload except for
padding bits
type=1 length=163
!AIVDM,1,1,,A,13aIkM@P00PJ@qPNL=e@0?wJ28JO,5*63
1|0|244740981|0|-128|0|1|3442480|31919541|0|511|45|0x0|1|0x434
type=1 length=168
!AIVDM,1,1,,A,13aIkM@P00PJ@qPNL=e@0?wJ28JO,0*66
1|0|244740981|0|-128|0|1|3442480|31919541|0|511|45|0x0|1|0x869f
This could make sense for re-broadcast data but
there may be other issues with type=1,2,3 because the message
lengths vary from 163 up to 700 some bits long and logic
only parses first 168 bits
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Full parsing not guarenteed but it at least parses more of
the data that is pressent for several of these messages
there are 28 bits not being processed out of this message
but it doesn't match up with the 12+4+3+11=30 bits that
the repeated data set is currently using
Looks like other messages of type 20 have data in the
last 28 bits but it's not decoded anywhere right now
186 bits long
!AIVDM,1,1,,2,D02E34iFTg6D000000000000002gjG2,0*75
20|0|002442003|1385|2|7|1125|0|0|0|0|0|0|0|0|0|0|0|0
186 bits long
!AIVDM,1,1,,2,D02=VVA8`N?`>4N01L=Nfp1>AA0,0*75
20|0|002320025|1162|1|7|250|225|1|7|0|1475|5|7|750|19|9|0|1296
shorter
!AIVDM,1,1,,A,D028rqP<QNfp000000000000000,2*0C
20|0|002243302|200|5|7|750|0|0|0|0|0|0|0|0|0|0|0|0
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Found a live sample that was not handled
right by the ais driver for type=16
max size had to be increased to 168 bits from 144
this looks sane as the samples of this type of message
as follows shows that the increment1, mmsi2, offset2,
and increment2 have a value.
It also matches results from http://www.aggsoft.com/ais-decoder.htm
test/sample.aivdm.js.chk
42:{"class":"AIS","device":"stdin","type":16,"repeat":0,"mmsi":2053501,"scaled":true,"mmsi1":224251000,"offset1":200,"increment1":0,"mmsi2":0,"offset2":0,"increment2":0}
92:{"class":"AIS","device":"stdin","type":16,"repeat":0,"mmsi":439952844,"scaled":true,"mmsi1":315920,"offset1":2049,"increment1":681,"mmsi2":230137673,"offset2":424,"increment2":419}
|
| |
|
|
|
| |
had to extend check to include extra size
Also tweaked devtools/ais.py
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
While trying the regression tests on a MacBook (PowerPC), I ran across
some failures in the "Testing AIVDM decoding w/ CSV format" test. The
failing type/dac/fid cases were as follows:
6,1,14
6,1,18
6,1,30
6,1,32
8,200,10
The proximate cause is endian dependence in the CSV output, but the
real problem turns out to be that the code is broken on all platforms,
but only fails on big-endian platforms because the
incorrect-but-presumed-correct output was captured on a little-endian
processor.
The problem arises when the type 6 or type 8 case is supported by the
AIVDM parser but not by the CSV formatter. When the parser doesn't
support the case, it falls back to capturing the raw binary
bitdata. When the formatter doesn't support the case, it falls back to
dumping the raw binary bitdata. But in the offending cases, the raw
data was never correctly captured, and is instead an overlaid view of
the structured data, which is incorrect (as raw data) in all cases as
well as being endian-sensitive due to the presence of multibyte
quantities in host byte order.
The only way to make the CSV fallback work as intended would be to
change the struct ais_t type6 and type8 substructure definitions to
move the bitdata field out of the union and populate it
unconditionally. But this would increase the memory footprint and add
to the core code for the sole purpose of providing more useful output
in these cases of missing CSV formatting support, which doesn't seem
very justifiable.
The alternative is simply not to report the unavailable raw bitdata at
all in these cases, but instead to report something indicating that
the raw data is unknown. I've implemented a version of this where the
formatting remains the same (including the bit count) but the hex data
is replaced by "x" characters (which seemed safer than using "?",
given the absence of the latter in any existing cases.
After implementing the code change and before updating the test data,
I saw one additional miscompare that I'd not seen on the PPC. This was
for the 6,1,12 case, where the existing test data is too incomplete to
populate the endian-sensitive fields in the decoded form, but which
would have failed with a better version of the test case.
After updating the test data as well as the code, the test passes on
both x86_64 and PPC.
P.S.:
There are three additional uses of gpsd_hexdump in the CSV output, for
types 17, 25, and 26, but those all seem to populate the bidata field
unconditionally (and type 17 doesn't even have a "structured" flag),
so I left them alone.
Clearly the right fix would be to implement the missing CSV formatting
cases, which would render the fallback code unreachable, but it could
become reachable again if new cases were implemented in the parser and
not in the CSV formatter, so having a well-behaved fallback seems like
a good idea.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- Type 8, DAC = 1 FID = 16: This should only be decoded as "Persons on board" if the message length is either 72 or 136 bits long (if even then).
- Type 8, DAC = 1 FID = 16: If decoded, the json output should use the ais->type8.dac1fid16.persons field and not the type6 field
Was not able to locate entry that had persons actually populated in AISHub sample data file
added sample data that should not be decoded to sample.aivdm
had to regenerate large sample file as it had multiple entries for this
Identified by: Stefan Roels <sroels-gpsd-dev@42solutions.nl>
type 8 dac 1 fid 16 can have either data or persons
had to add logic to detect structured vs not structured for type 8 FID 16 messages
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
I ran into a problem with the type 8 unstructured data. When the number
of bits available is not dividable by 8, the last byte may contain
random bits. I would expect the bits beyond the valid range to be 0.
An example of this can be found in the following message:
!AIVDM,3,1,4,A,85PH6TAKfDOkp95`nCRt5w<:qFUiaihFhBc7s4AHGsQ,0*40
!AIVDM,3,2,4,A,DcMJM18k6<=m7rwVm3b5RRWEskwJWej8uP<0:W5K6PA,0*61
!AIVDM,3,3,4,A,gPs<dwulp,4*14
The current version decodes this to:
{"class":"AIS","device":"stdin","type":8,"repeat":0,"mmsi":369493649,"scaled":true,"dac":366,"fid":57,"data":"510:47f3e09168d938bc17f30ae56971a71c16c12ac7ec44585fb854add69d048cc630dd47ebf9b50ea1628a757bcff6a7b7223d80c00a9c56c6811be0eccb3ff74e"}
The 510 bits of data are 63 full bytes and then 6 remaining bits. The
last byte has value: 4e, which is 0100 1110. I would expect 4c, 0100 1100.
To fix this, the following patch could be applied. Please note that the
problem might also be present in other message types, but I have not
encountered any problems in other message types in the 1 million
messages I have in my test dataset. Message types that could be impacted
are: 6, 17, 25 and 26, as they all use the same memcpy to get
unstructured data.
Identified by: Stefan Roels <sroels-gpsd-dev@42solutions.nl>
Added new sample entry to sample.aivdm that exercises this case
Had to regenerate large ais test data log
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When there are unknown FID values in type 6 messages for DAC values that
have some known FIDs (DAC 200, 235 and 250), the data is not read (and
thus printed as all zeros). I have added some example AIVDM messages
which exhibit this problem, some that did not have the problem in the
existing version and a patch for the problem.
Added sample
!AIVDM,1,1,,A,601uEPprEH2@<P<j00,4*32
-{"class":"AIS","device":"stdin","type":6,"repeat":0,"mmsi":2053507,"scaled":true,"seqno":2,"dest_mmsi":244670500,"retransmit":false,"dac":200,"fid":3,"data":"16:0000"}
+{"class":"AIS","device":"stdin","type":6,"repeat":0,"mmsi":2053507,"scaled":true,"seqno":2,"dest_mmsi":244670500,"retransmit":false,"dac":200,"fid":3,"data":"16:3200"}
adjusted loop logic to allow it to continue to process other types
Identified by: Stefan Roels <sroels-gpsd-dev@42solutions.nl>
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
- Type 8, DAC = 1 FID = 11: airtemp, dewpoint and watertemp calculations go wrong due to unsigned ints.
Identified by: Stefan Roels <sroels-gpsd-dev@42solutions.nl>
unsigned/signed subtraction was causing math to go wonky.
new sample data added to sample.aivdm which exibits the conversion errors
Also identified watertemp that was failing from same data
Regenerated the corresponding check files from current code
|
| |
|
|
|
|
| |
Type 9: the ais->type9.speed field should be used instead of the type1 field
Identifed by: Stefan Roels <sroels-gpsd-dev@42solutions.nl>
|
| |
|
|
|
| |
Required one checkfile rebuild to integrate the sentence that triggered
the error. All regression tests pass.
|
| | |
|
| | |
|
| |
|
|
| |
Also, add a regression test for this.
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
All regression tests pass.
|
| | |
|
| | |
|
| |
|
|
|
| |
Some modifications were lost during last merge, fix that.
Regression pass except for ait250.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A test campaign was run with live data from aishub.net (ca. 25 millions
sentence, 1.2 GB of logs), the goal was to check the decoding and the JSON
parsing and dumping code.
On all the AIS data, the following tests were done:
- AIVDM decoding
- JSON scaled dump
- JSON validity check on scaled dump.
- JSON unscaled dump
- JSON validity check on unscaled dump
- JSON unscaled idempotency check
- JSON unscaled/scaled idempotency check
The last check consisted of:
- parsing the unscaled dump
- dumping it back in scaled mode
- compare it with the original scaled dump.
This test campaign has unveiled plenty of small bugs all around the AIS code.
This patch fixes all of them and adds new sentences in the regression tests.
|
| |
|
|
| |
splint and check pass.
|
| | |
|
| |
|
|
| |
scons check pass
|
| |
|
|
| |
scons check pass
|
| |
|
|
|
|
| |
Nug was "AIS message type 8 Functional ID (fid) field at wrong offset."
All regression tests pass. Code splints and cppchecks clean.
|
| | |
|
| |
|
|
|
|
|
|
| |
Patch adds unit tests for all the collision cases.
All regression tests pass.
Signed-off-by: Eric S. Raymond <esr@thyrsus.com>
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
The problem with CSV is that string fields (such as ship name in type 5) can
contain commas. I could have backslash-escaped them, but I think it's better
to make old scripts fail in a way that's likely to be noticed than perpetuate
a situation in which unescaped commas could cause output to be unpacked
wrongly. I changed the Python decoder as well.
Required rebuilding one regression test. All regression tests pass.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Wire protocol and API minor versions get bumped. All changes are as
documented in AIVDM.txt, which now describes known message 6 and 8
subtypes.
Involved rebuilding a couple of AIS regression tests. All regression
tests pass.
Also includes various typo fixes for AIVDM.txt discovered by Baylink while
we were reviewing these changes.
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
Also fix type 16 interpretation in the Python decoder.
|
| | |
|
|
|
Not everything in the AIVDM test load is from Kurt Schwehr any more.
|
