Reverse engineering CRC8 algorithm

Question

I am trying to reverse-engineer a protocol which includes a final byte that is (possibly) a CRC-8 with some unknown parameters.

I have collected some data, however, the downside is the fixed length

<------------------ data ------------------>  CRC Byte
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  B1
20 20 20 20 20 20 20 20 20 20 20 20 20 20 20  6B
C8 6E 4D 00 00 00 00 00 00 00 00 00 00 00 00  55
38 3C 80 40 40 40 A0 95 91 33 23 0B A1 C6 19  44
01 40 BA 1E 5D 4C 0A 04 16 60 E5 4A C0 78 35  DF

I have already tried reveng with no luck. I can provide more messages and their respective CRCs.

EDIT 1:

Data with small differences. I have collected some data (pastebin) with small differences, for example, the following:

38 3C 80 40 20 70 A5 15 A5 33 64 F5 A0 2A 59 BD
38 3C 80 40 20 70 A5 15 A5 33 64 F5 A0 26 59 B4
                                        ^
38 3C 80 40 20 40 A0 95 91 33 23 0B A0 CA 1D 09
38 3C 80 40 20 40 A0 95 91 33 23 0B A0 C5 1D 42
                                        ^

EDIT 2:

I have found some messages with one bit of difference:

38 3C 80 40 40 70 A5 15 A5 33 64 F5 A1 31 5D B1
38 3C 80 40 40 70 A5 15 A5 33 64 F5 A1 34 5D 77
38 3C 80 40 40 70 A5 15 A5 33 64 F5 A1 35 5D B6
38 3C 80 40 40 70 A5 15 A5 33 64 F5 A1 38 5D 7E
38 3C 80 40 40 70 A5 15 A5 33 64 F5 A1 39 5D BF
                                        ^
38 3C 80 40 40 40 A0 95 91 33 23 E7 A1 38 1E EC
38 3C 80 40 40 40 A0 95 91 33 23 E7 A1 39 1E 2D
                                        ^
38 3C 80 40 40 40 A0 95 91 33 23 0B A1 CA 1D 2F
38 3C 80 40 40 40 A0 95 91 33 23 0B A1 CE 1D 28
                                        ^

EDIT 3:

I have found some patterns following this tutorial using the following messages (pastebin):

383c80404070a515a53364f5a1315db1
383c80404070a515a53364f5a1345d77
383c80404070a515a53364f5a1355db6
383c80404070a515a53364f5a1385d7e
383c80404070a515a53364f5a1395dbf
383c80404070a515a53364f5a13a5dfd
383c80404070a515a53364f5a13b5d3c
                           ^

After XORing I can get the following difference messages:

It seems that after shifting the difference message 1 bit to the left, the resulting checksum also is being shifted 1 bit to the left. And if the leading bit of the checksum (the one about to be shifted out) is 1, then the resulting checksum will be equal to the previous shifted one place and XORed with 1 (as if 1 was the polynomial of the crc8).

However, this does not hold further for these messages:

383c80404070a515a53364f5a1245d73
383c80404070a515a53364f5a1345d77

After XORing:

Some other examples:

383c80404040a095913323e0a0d51d0d
383c80404040a095913323e0a1d51d8b
--------------------------------
                         1000086
383c80402070a515a5332422a02d7568
383c80402070a515a5332422a42d756a

                     4000002

EDIT 4:

Take the following example:

383c80404040a0959133230ba1ce1d28
383c80404040a0959133230ba1d41da1
--------------------------------
                          1A0089

1A0089 can be obtained by XORing 100004 ^ 8000E ^ 20083. I am not sure if it has something to do with the checksum.

EDIT 5:

I have made a table of single bit difference messages:

00 00 00 01 -> 98
00 00 00 02 -> 31 ?
00 00 00 04 -> 62
00 00 00 08 -> C4
00 00 00 10 -> 91
00 00 00 20 -> 23
00 00 00 40 -> 46 ?
00 00 00 80 -> 8C ?
00 00 01 00 -> C1
00 00 02 00 -> 83
00 00 04 00 -> 07
00 00 08 00 -> 0E
00 00 10 00 -> 04
00 00 20 00 -> 08 ?
00 00 40 00 -> 10 ?
00 00 80 00 -> 20 ?
00 01 00 00 -> 86
00 02 00 00 -> 
00 04 00 00 -> 02
00 08 00 00 ->
00 10 00 00 ->
00 20 00 00 ->
00 40 00 00 -> 67
00 80 00 00 ->

I have marked with ? the entries I predicted myself following the steps described in edit 3. Other entries were obtained XORing either original or difference messages (as described in edit 4).

EDIT 6:

I have collected more data (pastebin).

Can you give more details what the protocol is used for? What makes you believe that this is checksum/CRC and not part of data? — dieter reichl, Oct 16 '22 at 08:47
@dieterreichl I have no details about the protocol, but I think it is some sort of low level protocol where the data is divided in blocks of 16 bytes. I am sure the last byte is a checksum because it is deterministic with respect of the input data. Same data outputs same checksum. I have even found collisions. — Afck, Oct 18 '22 at 11:27
More data would be beneficial, yes. When possible show data with differences in single bytes only like 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 B1, 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 ab, 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 cd — dieter reichl, Oct 23 '22 at 11:58
@dieterreichl I have posted a pastebin with similar data. I don't know if there are many cases with single byte differences but I have found one as shown in the question. — Afck, Oct 25 '22 at 16:51
@dieterreichl if you discover something interesting, let me know, please! — Afck, Oct 25 '22 at 16:52
I have many of the CRC algorithms coded in Asm86, from decades ago. I will dig them out tomorrow and pass the data through them to see if it is CRC8. — Rohit Gupta, Nov 03 '22 at 12:04
@RohitGupta thank you, if you find something useful, let me know! — Afck, Nov 05 '22 at 12:14
Those messages make me extremely sure that it is not CRC, because the checksums are too similar to the others: 38 3C 80 40 40 70 A5 15 A5 33 64 F5 A1 31 5D B1 // 38 3C 80 40 40 70 A5 15 A5 33 64 F5 A1 34 5D 77 // 38 3C 80 40 40 70 A5 15 A5 33 64 F5 A1 35 5D B6 // 38 3C 80 40 40 70 A5 15 A5 33 64 F5 A1 38 5D 7E // 38 3C 80 40 40 70 A5 15 A5 33 64 F5 A1 39 5D BF // I have no solution at the moment, but can you post more messages to have a full row 30, 32, 32, 33 ...? Maybe this gives a hint — dieter reichl, Dec 11 '22 at 16:50
@dieterreichl I cannot get a specific message as I want. All the messages are very similar, I don't think I can provide like a full row you are asking — Afck, Dec 11 '22 at 22:57
@dieterreichl Maybe it is not a CRC but it is definitely a checksum using simple binary operations (XORs, shifts...) — Afck, Dec 11 '22 at 22:58
agree to latest comment, that was the reason for above question. The more data you have, the more you can check for rules in the data. If you only have have less data, the more you need to be clever or guess the right operation. As I had similar case, I automatized the collection, and after having almost the full dataset, then I started to check for rules. I'm not good in guessing, and I didn't found a software that could do that for me. — dieter reichl, Dec 16 '22 at 17:26
@dieterreichl I added more data. But I dont think I will be able to have the full dataset. Hope this helps to find some rules. — Afck, Feb 16 '23 at 21:51
Could you tell please where the data comes from? Are those real raw data or have there some processing done? Can you tell how do you exactly receive them? I just see that they are not randomly distributed, and some more likely combinations that remembers me to somewhat I had from radio protocolls. — dieter reichl, Feb 24 '23 at 11:48
@dieterreichl Unfortunately, I don't have more information about it. — Afck, Feb 24 '23 at 14:00

Reverse engineering CRC8 algorithm

0 Answers0