How fail-safe is nr as stop bytes?
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ margin-bottom:0;
}
$begingroup$
In my UART communication I need to know the start byte and the stop byte of the message sent. The start byte is easy but the stop byte, not so much. I have implemented two stop bytes at the end of my message, that is n and r (10 and 13 decimal). UART only works on bytes 0-255 values so how fail-safe is this? I can imagine, though low probability, that my message might contain the values "10 and 13" after each other when they are not the stop bytes.
Is there a better way to implement this?
uart communication protocol
edited Apr 19 at 21:07
Nick Alexeev♦
32.7k1066167
asked Apr 18 at 10:33
Marius GulbrandsenMarius Gulbrandsen
169113
$endgroup$
|
show 11 more comments
$begingroup$
In my UART communication I need to know the start byte and the stop byte of the message sent. The start byte is easy but the stop byte, not so much. I have implemented two stop bytes at the end of my message, that is n and r (10 and 13 decimal). UART only works on bytes 0-255 values so how fail-safe is this? I can imagine, though low probability, that my message might contain the values "10 and 13" after each other when they are not the stop bytes.
Is there a better way to implement this?
uart communication protocol
edited Apr 19 at 21:07
Nick Alexeev♦
32.7k1066167
asked Apr 18 at 10:33
Marius GulbrandsenMarius Gulbrandsen
169113
$endgroup$
7
$begingroup$
To send arbitrary data you either have to go to using packets or byte stuffing. In your case the probability of the pattern appearing in a certain location is 1/65536. Which gets to 1 if you have a long enough random data stream.
$endgroup$
– Oldfart
Apr 18 at 10:36
4
$begingroup$
Can you provide context please. Stop bits are part of UART communication but stop bytes? This sounds like a pure software issue and depends what has been agreed by the sender and receiver.
$endgroup$
– Warren Hill
Apr 18 at 10:39
2
$begingroup$
@MariusGulbrandsen if your data is truly arbitrary and not strictly text (think ASCII) then null termination will not work; you will have to implement a packet.
$endgroup$
– RamblinRose
Apr 18 at 10:47
4
$begingroup$
BTW: That common practice is to put the carriage return before the line feed:"x0Dx0A".
$endgroup$
– Adrian McCarthy
Apr 18 at 21:16
3
$begingroup$
@AdrianMcCarthy I think the point of reversing it is to minimize the odds of it being a valid sequence. That said, two Windows line-endings in a row would give yournrnwhich contains thenrsequence in the middle...
$endgroup$
– Mike Caron
Apr 19 at 16:05
|
show 11 more comments
$begingroup$
In my UART communication I need to know the start byte and the stop byte of the message sent. The start byte is easy but the stop byte, not so much. I have implemented two stop bytes at the end of my message, that is n and r (10 and 13 decimal). UART only works on bytes 0-255 values so how fail-safe is this? I can imagine, though low probability, that my message might contain the values "10 and 13" after each other when they are not the stop bytes.
Is there a better way to implement this?
uart communication protocol
edited Apr 19 at 21:07
Nick Alexeev♦
32.7k1066167
asked Apr 18 at 10:33
Marius GulbrandsenMarius Gulbrandsen
169113
$endgroup$
In my UART communication I need to know the start byte and the stop byte of the message sent. The start byte is easy but the stop byte, not so much. I have implemented two stop bytes at the end of my message, that is n and r (10 and 13 decimal). UART only works on bytes 0-255 values so how fail-safe is this? I can imagine, though low probability, that my message might contain the values "10 and 13" after each other when they are not the stop bytes.
Is there a better way to implement this?
uart communication protocol
uart communication protocol
edited Apr 19 at 21:07
Nick Alexeev♦
32.7k1066167
asked Apr 18 at 10:33
Marius GulbrandsenMarius Gulbrandsen
169113
edited Apr 19 at 21:07
Nick Alexeev♦
32.7k1066167
asked Apr 18 at 10:33
Marius GulbrandsenMarius Gulbrandsen
169113
edited Apr 19 at 21:07
Nick Alexeev♦
32.7k1066167
edited Apr 19 at 21:07
Nick Alexeev♦
32.7k1066167
edited Apr 19 at 21:07
Nick Alexeev♦
32.7k1066167
32.7k1066167
asked Apr 18 at 10:33
Marius GulbrandsenMarius Gulbrandsen
169113
asked Apr 18 at 10:33
Marius GulbrandsenMarius Gulbrandsen
169113
asked Apr 18 at 10:33
Marius GulbrandsenMarius Gulbrandsen
169113
169113
7
$begingroup$
To send arbitrary data you either have to go to using packets or byte stuffing. In your case the probability of the pattern appearing in a certain location is 1/65536. Which gets to 1 if you have a long enough random data stream.
$endgroup$
– Oldfart
Apr 18 at 10:36
4
$begingroup$
Can you provide context please. Stop bits are part of UART communication but stop bytes? This sounds like a pure software issue and depends what has been agreed by the sender and receiver.
$endgroup$
– Warren Hill
Apr 18 at 10:39
2
$begingroup$
@MariusGulbrandsen if your data is truly arbitrary and not strictly text (think ASCII) then null termination will not work; you will have to implement a packet.
$endgroup$
– RamblinRose
Apr 18 at 10:47
4
$begingroup$
BTW: That common practice is to put the carriage return before the line feed:"x0Dx0A".
$endgroup$
– Adrian McCarthy
Apr 18 at 21:16
3
$begingroup$
@AdrianMcCarthy I think the point of reversing it is to minimize the odds of it being a valid sequence. That said, two Windows line-endings in a row would give yournrnwhich contains thenrsequence in the middle...
$endgroup$
– Mike Caron
Apr 19 at 16:05
|
show 11 more comments
7
$begingroup$
To send arbitrary data you either have to go to using packets or byte stuffing. In your case the probability of the pattern appearing in a certain location is 1/65536. Which gets to 1 if you have a long enough random data stream.
$endgroup$
– Oldfart
Apr 18 at 10:36
4
$begingroup$
Can you provide context please. Stop bits are part of UART communication but stop bytes? This sounds like a pure software issue and depends what has been agreed by the sender and receiver.
$endgroup$
– Warren Hill
Apr 18 at 10:39
2
$begingroup$
@MariusGulbrandsen if your data is truly arbitrary and not strictly text (think ASCII) then null termination will not work; you will have to implement a packet.
$endgroup$
– RamblinRose
Apr 18 at 10:47
4
$begingroup$
BTW: That common practice is to put the carriage return before the line feed:"x0Dx0A".
$endgroup$
– Adrian McCarthy
Apr 18 at 21:16
3
$begingroup$
@AdrianMcCarthy I think the point of reversing it is to minimize the odds of it being a valid sequence. That said, two Windows line-endings in a row would give yournrnwhich contains thenrsequence in the middle...
$endgroup$
– Mike Caron
Apr 19 at 16:05
7
7
$begingroup$
To send arbitrary data you either have to go to using packets or byte stuffing. In your case the probability of the pattern appearing in a certain location is 1/65536. Which gets to 1 if you have a long enough random data stream.
$endgroup$
– Oldfart
Apr 18 at 10:36
$begingroup$
To send arbitrary data you either have to go to using packets or byte stuffing. In your case the probability of the pattern appearing in a certain location is 1/65536. Which gets to 1 if you have a long enough random data stream.
$endgroup$
– Oldfart
Apr 18 at 10:36
4
4
$begingroup$
Can you provide context please. Stop bits are part of UART communication but stop bytes? This sounds like a pure software issue and depends what has been agreed by the sender and receiver.
$endgroup$
– Warren Hill
Apr 18 at 10:39
$begingroup$
Can you provide context please. Stop bits are part of UART communication but stop bytes? This sounds like a pure software issue and depends what has been agreed by the sender and receiver.
$endgroup$
– Warren Hill
Apr 18 at 10:39
2
2
$begingroup$
@MariusGulbrandsen if your data is truly arbitrary and not strictly text (think ASCII) then null termination will not work; you will have to implement a packet.
$endgroup$
– RamblinRose
Apr 18 at 10:47
$begingroup$
@MariusGulbrandsen if your data is truly arbitrary and not strictly text (think ASCII) then null termination will not work; you will have to implement a packet.
$endgroup$
– RamblinRose
Apr 18 at 10:47
4
4
$begingroup$
BTW: That common practice is to put the carriage return before the line feed:
"x0Dx0A".$endgroup$
– Adrian McCarthy
Apr 18 at 21:16
$begingroup$
BTW: That common practice is to put the carriage return before the line feed:
"x0Dx0A".$endgroup$
– Adrian McCarthy
Apr 18 at 21:16
3
3
$begingroup$
@AdrianMcCarthy I think the point of reversing it is to minimize the odds of it being a valid sequence. That said, two Windows line-endings in a row would give you
rnrn which contains the nr sequence in the middle...$endgroup$
– Mike Caron
Apr 19 at 16:05
$begingroup$
@AdrianMcCarthy I think the point of reversing it is to minimize the odds of it being a valid sequence. That said, two Windows line-endings in a row would give you
rnrn which contains the nr sequence in the middle...$endgroup$
– Mike Caron
Apr 19 at 16:05
|
show 11 more comments
5 Answers
5
active
oldest
votes
$begingroup$
There are different ways to prevent this:
- Make sure you never send a 10/13 combination in your regular messages (so only as stop bytes). E.g. to send 20 21 22 23 24 25:
20 21 22 23 24 25 10 13
- Escape 10 and 13 (or all non ASCII characters with an escape character e.g. . So to send 20 21 10 13 25 26 send: (see comment of/credits for: DanW)
20 21 1b 10 1b 13 25 26
- Define a packet when sending messages. E.g. if you want to send message 20 21 22 23 24 25 than instead add the number of bytes to sent, so the package is:
< nr_of_data_bytes > < data >
If your messages are max 256 bytes send:
06 20 21 22 23 24 25
So you know after receiving 6 data bytes that is the end; you don't have to send a 10 13 afterwards. And you can send 10 13 inside a message. If your messages can be longer, you can use 2 bytes for the data size.
Update 1: Another way of defining packets
Another alternative is to send commands which have a specific length and can have many variances, e.g.
10 20 30 (Command 10 which always has 2 data bytes)
11 30 40 50 (Command 11 which always has 3 data bytes)
12 06 10 11 12 13 14 15 (Command 12 + 1 byte for the number of data bytes that follow)
13 01 02 01 02 03 ... (Command 13 + 2 bytes (01 02 for 256 + 2 = 258 data bytes that follow)
14 80 90 10 13 (Command 14 that is followed by an ASCII string ending with 10 13)
Update 2: Bad connection/byte losses
All of the above only work when the UART line is sending bytes correctly. If you want to use more reliable ways of sending, there are also many possibilities. Below are a few:
- Sending a checksum within the package (check google for CRC: Cyclic Redundancy Check). If the CRC is ok, the receiver knows the message has been sent ok (with high probability).
- If you need a message to be resent, than an acknowledgement (ACK/reply) mechanism needs to be used (e.g. sender sends something, receiver receives corrupt data, sends a NACK (not acknowledged), sender can than send again.
- Timeout: In case the receiver does not get an ACK or NACK in time, a message needs to be resend.
Note that all above mechanism can be simple or as complicated as you want (or need) to be. In case of resending message, also a mechanism for identifying messages is needed (e.g. adding a sequence number into the package).
answered Apr 18 at 10:59
Michel KeijzersMichel Keijzers
7,20293173
$endgroup$
1
$begingroup$
"Make sure you never send a 10/13 combination in your regular messages (so only as stop bytes)." – you've not said how to send data which does include a 10/13 combination – you need to escape it. So "20 10 13 23 10 13" might be sent as "20 1b 10 1b 13 23" with 1b as your escape character.
$endgroup$
– Dan W
Apr 18 at 18:33
1
$begingroup$
Note that using a length field as proposed, you’ll get in trouble when your serial link is bad and loses a single byte. Everything will go out of sync.
$endgroup$
– Jonas Schäfer
Apr 18 at 19:55
$begingroup$
@DanW If you use the first one or 2 bytes as the number of data bytes, it does not matter if 10 or 13 are part of those data... So 20 10 13 23 10 13 can be send as 06 20 10 13 23 10 13 where 06 is the number of data bytes that follow.
$endgroup$
– Michel Keijzers
Apr 18 at 22:08
$begingroup$
@MichelKeijzers - yes, but that’s the second solution you mention. Your first solution is missing an explanation of escape sequences to prevent the stop bytes being transmitted.
$endgroup$
– Dan W
Apr 18 at 22:25
$begingroup$
Both approaches work, and are commonly used, but they have different advantages and disadvantages, which you could add if wanted, though it’s beyond what the OP asked for.
$endgroup$
– Dan W
Apr 18 at 22:27
|
show 1 more comment
$begingroup$
How fail-safe is nr as stop bytes?
If you send send arbitrary data -> probably not fail-safe enough.
A common solution is to use escaping:
Let's define that the characters 0x02 (STX - frame start) and 0x03 (ETX - frame end) need to be unique within the transmitted data stream. This way the start and the end of a message can be safely detected.
If one of these characters should be send within the message frame, it is replaced by prefixing an escape character (ESC = 0x1b) and adding 0x20 to the original character.
Original character replaced by
0x02 -> 0x1b 0x22
0x03 -> 0x1b 0x23
0x1b -> 0x1b 0x3b
The receiver reverses this process: Anytime he receives an escape character, this character is dropped and the next character is subtracted by 0x20.
This only adds some processing overhead but is 100% reliable (assuming no transmission errors occur, which you could/should verify by additionally implementing a checksum mechanism).
answered Apr 18 at 11:05
Rev1.0Rev1.0
7,72043369
$endgroup$
1
$begingroup$
Nice answer. The common escape character used for ASCII protocols was'x10'DLE (Data Link Escape). Some of the Wikipedia pages suggest that DLE was often used in the opposite way: to say that the next byte was a control character rather than a data byte. In my experience, that's generally the opposite meaning for an escape.
$endgroup$
– Adrian McCarthy
Apr 18 at 21:23
2
$begingroup$
One thing to watch our for here is that your worst case buffer size doubles. If memory is really tight that might not be the best solution.
$endgroup$
– TechnoSam
Apr 18 at 21:23
1
$begingroup$
@Rev What's the rationale for adding 0x20 to the original character? Wouldn't the escaping scheme work without that just as well?
$endgroup$
– Nick Alexeev♦
Apr 19 at 21:02
$begingroup$
@NickAlexeev: It is easier/faster to identify the actual frame boundaries if you remove any other occurrence of the reserved chars from the stream. That way, you can seperate frame reception and frame parsing (including the un-escaping). This may be especially relevant, if you have a very slow controller without FIFO and/or high data rates. So you can just copy the incoming bytes (between STX/ETX) into the frame buffer as they arrive, mark the frame as complete and do the processing with lower priority.
$endgroup$
– Rev1.0
Apr 21 at 18:08
$begingroup$
@TechnoSam: Good point.
$endgroup$
– Rev1.0
Apr 21 at 18:08
add a comment |
$begingroup$
You know, ASCII already has bytes for these functions.
- 0x01 : start of heading -- start byte
- 0x02 : start of text -- end headers, begin payload
- 0x03 : end of text -- end payload
- 0x04 : end of transmission -- stop byte
- 0x17 : end of transmission block -- message continues in next block
It also has codes for various uses inside the payload.
- 0x1b : escape (escape the next character -- use in payload to indicate next character is not one of the structure describing codes used in your protocol)
- 0x1c, 0x1d, 0x1e, 0x1f : file, group, record, and unit separator, respectively -- used as simultaneous stop and start byte for parts of hierarchical data
Your protocol should specify the finest granularity of ACK (0x06) and NAK (0x15), so that negative acknowledged data can be retransmitted. Down to this finest granularity, it is wise to have a length field immediately after any (unescaped) start indicator and (as explained in other answer(s)) it is wise to follow any (unescaped) stop indicator with a CRC.
answered Apr 18 at 15:55
Eric TowersEric Towers
20912
$endgroup$
$begingroup$
I will be sending arbitrary data, I guess it might have been confusing to use "nr" in my question when I'm not sending ASCII data. Even though, I like this answer, it's very informative on sending ASCII over UART
$endgroup$
– Marius Gulbrandsen
Apr 19 at 21:50
$begingroup$
@MariusGulbrandsen : As long as your protocol establishes where payload is and which codes must be escaped in each payload section, you can send anything, not just text-ish data.
$endgroup$
– Eric Towers
Apr 19 at 22:57
add a comment |
$begingroup$
UART is not fail-safe by its very nature - we are talking about 1960s technology here.
The root of the problem being that UART only syncs once per 10 bits, allowing a lot of gibberish to pass between those sync periods. Unlike for example CAN which samples every individual bit multiple times.
Any double bit error occurring inside the data will corrupt an UART frame and pass undetected. Bit errors in start/stop bits may or may not get detected in the form of overrun errors.
Therefore, no matter if you use raw data or packets, there is always a probability that bit flips caused by EMI result in unexpected data.
There exist numerous ways of "traditional UART quackery" to improve the situation ever so slightly. You can add sync bytes, sync bits, parity, double stop bits. You could add checksums that count the sum of all bytes (and then invert it - because why not) or you could count the number of binary ones as a checksum. All of this is widely used, wildly unscientific and with a high probability of missing errors. But this was what people did from 1960s to 1990s and lots of weird things like these lives on today.
The most professional way to deal with safe transmission over UART is to have a 16 bit CRC checksum at the end of the packet. Everything else isn't very safe and has a high probability of missing errors.
Then on the hardware level you can use differential RS-422/RS-485 to drastically improve ruggedness of the transmission. This is a must for safe transmission over longer distances. TTL level UART should only be used for on-board communication. RS-232 should not be used for any other purpose but backwards compatibility with old stuff.
Overall, the closer to the hardware your error detection mechanism is, the more effective it is. In terms of effectiveness, differential signals add the most, followed by checking for framing/overrun etc errors. CRC16 adds some, and then "traditional UART quackery" adds a little bit.
answered Apr 18 at 11:27
LundinLundin
4,5291131
$endgroup$
7
$begingroup$
This advice is fairly tangential - you haven't actually addressed the question asked. In particular, your proposed solutions may solve other problems, but they do not solve the basic problem of the question on this page, which is confusion between framing byes and payload byes. At most, your proposal would reject valid data embedding a framing byte due to CRC or similar failure, with no way to communicate such.
$endgroup$
– Chris Stratton
Apr 18 at 14:28
3
$begingroup$
In fact, this answer makes it worse. The original had just data bytes and stop bytes. This adds a third category, CRC bytes. And as presented here, those can take on any value, including {10,13}.
$endgroup$
– MSalters
Apr 18 at 15:53
1
$begingroup$
@MSalters: The CRC can be ASCII encoded hex to prevent this issue. Another trick that I've seen on RS485 is to set bit 7 on the start / address byte.
$endgroup$
– Transistor
Apr 18 at 16:03
$begingroup$
Re "CAN which samples every individual bit multiple times.": The actual sampling of the bit value is only once per bit. What are you referring to here? Some kind of error checking, like by the sender? Clock synchronisation?
$endgroup$
– Peter Mortensen
Apr 19 at 12:33
$begingroup$
The inverting of the checksum was done so that summing the entire block of data would result in a zero, which is a bit easier to code and a bit faster to execute. Also, CRC is much better than you make it out to be, look it up in the Wikipedia.
$endgroup$
– toolforger
Apr 19 at 22:04
|
show 6 more comments
$begingroup$
... I can imagine, though low probability, that my message might contain the values "10 and 13" after each other when they are not the stop bytes.
A situation when a portion of data is equal to terminating sequence should be considered when designing the format of a serial data packet. Another thing to consider is that any character can get corrupted or lost during transmission. A start character, a stop character, a data payload byte, a checksum or CRC byte, a forward error correction byte aren't immune to corruption. The framing mechanism has to be able to detect when a packet has corrupt data.
There a several ways to approach all this.
I'm making the working assumption that packets are framed only with the serial bytes. Handshake lines aren't used for framing. Time delays aren't used for framing.
Send packet length
Send the length of the packet in the beginning, instead of [or in addition to] the terminating character at the end.
pros: Payload is sent in a efficient binary format.
cons: Need to know the packet length at the start of the transmission.
Escape the special characters
Escape the special characters when sending the payload data. This is already explained in a an earlier answer.
pros: Sender doesn't need to know the length of the packet at the beginning of the transmission.
cons: Slightly less efficient, depending on how many payload bytes need to be escaped.
Payload data encoded such that it can't contain start and stop characters
The payload of the packet is encoded such that it can't contain the start or stop characters. Usually, this is done by sending numbers as their ASCII or Hex-ASCII representation.
pros: Human-readable with common terminal programs. No need for code to handle escaping. No need to know the length of the packet at the start of the transmission
cons: Lower efficiency. For one byte of payload data, several bytes are sent.
answered Apr 19 at 21:07
Nick Alexeev♦Nick Alexeev
32.7k1066167
$endgroup$
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("schematics", function () {
StackExchange.schematics.init();
});
}, "cicuitlab");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "135"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2felectronics.stackexchange.com%2fquestions%2f433202%2fhow-fail-safe-is-n-r-as-stop-bytes%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
5 Answers
5
active
oldest
votes
5 Answers
5
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
There are different ways to prevent this:
- Make sure you never send a 10/13 combination in your regular messages (so only as stop bytes). E.g. to send 20 21 22 23 24 25:
20 21 22 23 24 25 10 13
- Escape 10 and 13 (or all non ASCII characters with an escape character e.g. . So to send 20 21 10 13 25 26 send: (see comment of/credits for: DanW)
20 21 1b 10 1b 13 25 26
- Define a packet when sending messages. E.g. if you want to send message 20 21 22 23 24 25 than instead add the number of bytes to sent, so the package is:
< nr_of_data_bytes > < data >
If your messages are max 256 bytes send:
06 20 21 22 23 24 25
So you know after receiving 6 data bytes that is the end; you don't have to send a 10 13 afterwards. And you can send 10 13 inside a message. If your messages can be longer, you can use 2 bytes for the data size.
Update 1: Another way of defining packets
Another alternative is to send commands which have a specific length and can have many variances, e.g.
10 20 30 (Command 10 which always has 2 data bytes)
11 30 40 50 (Command 11 which always has 3 data bytes)
12 06 10 11 12 13 14 15 (Command 12 + 1 byte for the number of data bytes that follow)
13 01 02 01 02 03 ... (Command 13 + 2 bytes (01 02 for 256 + 2 = 258 data bytes that follow)
14 80 90 10 13 (Command 14 that is followed by an ASCII string ending with 10 13)
Update 2: Bad connection/byte losses
All of the above only work when the UART line is sending bytes correctly. If you want to use more reliable ways of sending, there are also many possibilities. Below are a few:
- Sending a checksum within the package (check google for CRC: Cyclic Redundancy Check). If the CRC is ok, the receiver knows the message has been sent ok (with high probability).
- If you need a message to be resent, than an acknowledgement (ACK/reply) mechanism needs to be used (e.g. sender sends something, receiver receives corrupt data, sends a NACK (not acknowledged), sender can than send again.
- Timeout: In case the receiver does not get an ACK or NACK in time, a message needs to be resend.
Note that all above mechanism can be simple or as complicated as you want (or need) to be. In case of resending message, also a mechanism for identifying messages is needed (e.g. adding a sequence number into the package).
answered Apr 18 at 10:59
Michel KeijzersMichel Keijzers
7,20293173
$endgroup$
1
$begingroup$
"Make sure you never send a 10/13 combination in your regular messages (so only as stop bytes)." – you've not said how to send data which does include a 10/13 combination – you need to escape it. So "20 10 13 23 10 13" might be sent as "20 1b 10 1b 13 23" with 1b as your escape character.
$endgroup$
– Dan W
Apr 18 at 18:33
1
$begingroup$
Note that using a length field as proposed, you’ll get in trouble when your serial link is bad and loses a single byte. Everything will go out of sync.
$endgroup$
– Jonas Schäfer
Apr 18 at 19:55
$begingroup$
@DanW If you use the first one or 2 bytes as the number of data bytes, it does not matter if 10 or 13 are part of those data... So 20 10 13 23 10 13 can be send as 06 20 10 13 23 10 13 where 06 is the number of data bytes that follow.
$endgroup$
– Michel Keijzers
Apr 18 at 22:08
$begingroup$
@MichelKeijzers - yes, but that’s the second solution you mention. Your first solution is missing an explanation of escape sequences to prevent the stop bytes being transmitted.
$endgroup$
– Dan W
Apr 18 at 22:25
$begingroup$
Both approaches work, and are commonly used, but they have different advantages and disadvantages, which you could add if wanted, though it’s beyond what the OP asked for.
$endgroup$
– Dan W
Apr 18 at 22:27
|
show 1 more comment
$begingroup$
There are different ways to prevent this:
- Make sure you never send a 10/13 combination in your regular messages (so only as stop bytes). E.g. to send 20 21 22 23 24 25:
20 21 22 23 24 25 10 13
- Escape 10 and 13 (or all non ASCII characters with an escape character e.g. . So to send 20 21 10 13 25 26 send: (see comment of/credits for: DanW)
20 21 1b 10 1b 13 25 26
- Define a packet when sending messages. E.g. if you want to send message 20 21 22 23 24 25 than instead add the number of bytes to sent, so the package is:
< nr_of_data_bytes > < data >
If your messages are max 256 bytes send:
06 20 21 22 23 24 25
So you know after receiving 6 data bytes that is the end; you don't have to send a 10 13 afterwards. And you can send 10 13 inside a message. If your messages can be longer, you can use 2 bytes for the data size.
Update 1: Another way of defining packets
Another alternative is to send commands which have a specific length and can have many variances, e.g.
10 20 30 (Command 10 which always has 2 data bytes)
11 30 40 50 (Command 11 which always has 3 data bytes)
12 06 10 11 12 13 14 15 (Command 12 + 1 byte for the number of data bytes that follow)
13 01 02 01 02 03 ... (Command 13 + 2 bytes (01 02 for 256 + 2 = 258 data bytes that follow)
14 80 90 10 13 (Command 14 that is followed by an ASCII string ending with 10 13)
Update 2: Bad connection/byte losses
All of the above only work when the UART line is sending bytes correctly. If you want to use more reliable ways of sending, there are also many possibilities. Below are a few:
- Sending a checksum within the package (check google for CRC: Cyclic Redundancy Check). If the CRC is ok, the receiver knows the message has been sent ok (with high probability).
- If you need a message to be resent, than an acknowledgement (ACK/reply) mechanism needs to be used (e.g. sender sends something, receiver receives corrupt data, sends a NACK (not acknowledged), sender can than send again.
- Timeout: In case the receiver does not get an ACK or NACK in time, a message needs to be resend.
Note that all above mechanism can be simple or as complicated as you want (or need) to be. In case of resending message, also a mechanism for identifying messages is needed (e.g. adding a sequence number into the package).
answered Apr 18 at 10:59
Michel KeijzersMichel Keijzers
7,20293173
$endgroup$
1
$begingroup$
"Make sure you never send a 10/13 combination in your regular messages (so only as stop bytes)." – you've not said how to send data which does include a 10/13 combination – you need to escape it. So "20 10 13 23 10 13" might be sent as "20 1b 10 1b 13 23" with 1b as your escape character.
$endgroup$
– Dan W
Apr 18 at 18:33
1
$begingroup$
Note that using a length field as proposed, you’ll get in trouble when your serial link is bad and loses a single byte. Everything will go out of sync.
$endgroup$
– Jonas Schäfer
Apr 18 at 19:55
$begingroup$
@DanW If you use the first one or 2 bytes as the number of data bytes, it does not matter if 10 or 13 are part of those data... So 20 10 13 23 10 13 can be send as 06 20 10 13 23 10 13 where 06 is the number of data bytes that follow.
$endgroup$
– Michel Keijzers
Apr 18 at 22:08
$begingroup$
@MichelKeijzers - yes, but that’s the second solution you mention. Your first solution is missing an explanation of escape sequences to prevent the stop bytes being transmitted.
$endgroup$
– Dan W
Apr 18 at 22:25
$begingroup$
Both approaches work, and are commonly used, but they have different advantages and disadvantages, which you could add if wanted, though it’s beyond what the OP asked for.
$endgroup$
– Dan W
Apr 18 at 22:27
|
show 1 more comment
$begingroup$
There are different ways to prevent this:
- Make sure you never send a 10/13 combination in your regular messages (so only as stop bytes). E.g. to send 20 21 22 23 24 25:
20 21 22 23 24 25 10 13
- Escape 10 and 13 (or all non ASCII characters with an escape character e.g. . So to send 20 21 10 13 25 26 send: (see comment of/credits for: DanW)
20 21 1b 10 1b 13 25 26
- Define a packet when sending messages. E.g. if you want to send message 20 21 22 23 24 25 than instead add the number of bytes to sent, so the package is:
< nr_of_data_bytes > < data >
If your messages are max 256 bytes send:
06 20 21 22 23 24 25
So you know after receiving 6 data bytes that is the end; you don't have to send a 10 13 afterwards. And you can send 10 13 inside a message. If your messages can be longer, you can use 2 bytes for the data size.
Update 1: Another way of defining packets
Another alternative is to send commands which have a specific length and can have many variances, e.g.
10 20 30 (Command 10 which always has 2 data bytes)
11 30 40 50 (Command 11 which always has 3 data bytes)
12 06 10 11 12 13 14 15 (Command 12 + 1 byte for the number of data bytes that follow)
13 01 02 01 02 03 ... (Command 13 + 2 bytes (01 02 for 256 + 2 = 258 data bytes that follow)
14 80 90 10 13 (Command 14 that is followed by an ASCII string ending with 10 13)
Update 2: Bad connection/byte losses
All of the above only work when the UART line is sending bytes correctly. If you want to use more reliable ways of sending, there are also many possibilities. Below are a few:
- Sending a checksum within the package (check google for CRC: Cyclic Redundancy Check). If the CRC is ok, the receiver knows the message has been sent ok (with high probability).
- If you need a message to be resent, than an acknowledgement (ACK/reply) mechanism needs to be used (e.g. sender sends something, receiver receives corrupt data, sends a NACK (not acknowledged), sender can than send again.
- Timeout: In case the receiver does not get an ACK or NACK in time, a message needs to be resend.
Note that all above mechanism can be simple or as complicated as you want (or need) to be. In case of resending message, also a mechanism for identifying messages is needed (e.g. adding a sequence number into the package).
answered Apr 18 at 10:59
Michel KeijzersMichel Keijzers
7,20293173
$endgroup$
There are different ways to prevent this:
- Make sure you never send a 10/13 combination in your regular messages (so only as stop bytes). E.g. to send 20 21 22 23 24 25:
20 21 22 23 24 25 10 13
- Escape 10 and 13 (or all non ASCII characters with an escape character e.g. . So to send 20 21 10 13 25 26 send: (see comment of/credits for: DanW)
20 21 1b 10 1b 13 25 26
- Define a packet when sending messages. E.g. if you want to send message 20 21 22 23 24 25 than instead add the number of bytes to sent, so the package is:
< nr_of_data_bytes > < data >
If your messages are max 256 bytes send:
06 20 21 22 23 24 25
So you know after receiving 6 data bytes that is the end; you don't have to send a 10 13 afterwards. And you can send 10 13 inside a message. If your messages can be longer, you can use 2 bytes for the data size.
Update 1: Another way of defining packets
Another alternative is to send commands which have a specific length and can have many variances, e.g.
10 20 30 (Command 10 which always has 2 data bytes)
11 30 40 50 (Command 11 which always has 3 data bytes)
12 06 10 11 12 13 14 15 (Command 12 + 1 byte for the number of data bytes that follow)
13 01 02 01 02 03 ... (Command 13 + 2 bytes (01 02 for 256 + 2 = 258 data bytes that follow)
14 80 90 10 13 (Command 14 that is followed by an ASCII string ending with 10 13)
Update 2: Bad connection/byte losses
All of the above only work when the UART line is sending bytes correctly. If you want to use more reliable ways of sending, there are also many possibilities. Below are a few:
- Sending a checksum within the package (check google for CRC: Cyclic Redundancy Check). If the CRC is ok, the receiver knows the message has been sent ok (with high probability).
- If you need a message to be resent, than an acknowledgement (ACK/reply) mechanism needs to be used (e.g. sender sends something, receiver receives corrupt data, sends a NACK (not acknowledged), sender can than send again.
- Timeout: In case the receiver does not get an ACK or NACK in time, a message needs to be resend.
Note that all above mechanism can be simple or as complicated as you want (or need) to be. In case of resending message, also a mechanism for identifying messages is needed (e.g. adding a sequence number into the package).
answered Apr 18 at 10:59
Michel KeijzersMichel Keijzers
7,20293173
edited Apr 18 at 22:42
answered Apr 18 at 10:59
Michel KeijzersMichel Keijzers
7,20293173
answered Apr 18 at 10:59
Michel KeijzersMichel Keijzers
7,20293173
answered Apr 18 at 10:59
Michel KeijzersMichel Keijzers
7,20293173
7,20293173
1
$begingroup$
"Make sure you never send a 10/13 combination in your regular messages (so only as stop bytes)." – you've not said how to send data which does include a 10/13 combination – you need to escape it. So "20 10 13 23 10 13" might be sent as "20 1b 10 1b 13 23" with 1b as your escape character.
$endgroup$
– Dan W
Apr 18 at 18:33
1
$begingroup$
Note that using a length field as proposed, you’ll get in trouble when your serial link is bad and loses a single byte. Everything will go out of sync.
$endgroup$
– Jonas Schäfer
Apr 18 at 19:55
$begingroup$
@DanW If you use the first one or 2 bytes as the number of data bytes, it does not matter if 10 or 13 are part of those data... So 20 10 13 23 10 13 can be send as 06 20 10 13 23 10 13 where 06 is the number of data bytes that follow.
$endgroup$
– Michel Keijzers
Apr 18 at 22:08
$begingroup$
@MichelKeijzers - yes, but that’s the second solution you mention. Your first solution is missing an explanation of escape sequences to prevent the stop bytes being transmitted.
$endgroup$
– Dan W
Apr 18 at 22:25
$begingroup$
Both approaches work, and are commonly used, but they have different advantages and disadvantages, which you could add if wanted, though it’s beyond what the OP asked for.
$endgroup$
– Dan W
Apr 18 at 22:27
|
show 1 more comment
1
$begingroup$
"Make sure you never send a 10/13 combination in your regular messages (so only as stop bytes)." – you've not said how to send data which does include a 10/13 combination – you need to escape it. So "20 10 13 23 10 13" might be sent as "20 1b 10 1b 13 23" with 1b as your escape character.
$endgroup$
– Dan W
Apr 18 at 18:33
1
$begingroup$
Note that using a length field as proposed, you’ll get in trouble when your serial link is bad and loses a single byte. Everything will go out of sync.
$endgroup$
– Jonas Schäfer
Apr 18 at 19:55
$begingroup$
@DanW If you use the first one or 2 bytes as the number of data bytes, it does not matter if 10 or 13 are part of those data... So 20 10 13 23 10 13 can be send as 06 20 10 13 23 10 13 where 06 is the number of data bytes that follow.
$endgroup$
– Michel Keijzers
Apr 18 at 22:08
$begingroup$
@MichelKeijzers - yes, but that’s the second solution you mention. Your first solution is missing an explanation of escape sequences to prevent the stop bytes being transmitted.
$endgroup$
– Dan W
Apr 18 at 22:25
$begingroup$
Both approaches work, and are commonly used, but they have different advantages and disadvantages, which you could add if wanted, though it’s beyond what the OP asked for.
$endgroup$
– Dan W
Apr 18 at 22:27
1
1
$begingroup$
"Make sure you never send a 10/13 combination in your regular messages (so only as stop bytes)." – you've not said how to send data which does include a 10/13 combination – you need to escape it. So "20 10 13 23 10 13" might be sent as "20 1b 10 1b 13 23" with 1b as your escape character.
$endgroup$
– Dan W
Apr 18 at 18:33
$begingroup$
"Make sure you never send a 10/13 combination in your regular messages (so only as stop bytes)." – you've not said how to send data which does include a 10/13 combination – you need to escape it. So "20 10 13 23 10 13" might be sent as "20 1b 10 1b 13 23" with 1b as your escape character.
$endgroup$
– Dan W
Apr 18 at 18:33
1
1
$begingroup$
Note that using a length field as proposed, you’ll get in trouble when your serial link is bad and loses a single byte. Everything will go out of sync.
$endgroup$
– Jonas Schäfer
Apr 18 at 19:55
$begingroup$
Note that using a length field as proposed, you’ll get in trouble when your serial link is bad and loses a single byte. Everything will go out of sync.
$endgroup$
– Jonas Schäfer
Apr 18 at 19:55
$begingroup$
@DanW If you use the first one or 2 bytes as the number of data bytes, it does not matter if 10 or 13 are part of those data... So 20 10 13 23 10 13 can be send as 06 20 10 13 23 10 13 where 06 is the number of data bytes that follow.
$endgroup$
– Michel Keijzers
Apr 18 at 22:08
$begingroup$
@DanW If you use the first one or 2 bytes as the number of data bytes, it does not matter if 10 or 13 are part of those data... So 20 10 13 23 10 13 can be send as 06 20 10 13 23 10 13 where 06 is the number of data bytes that follow.
$endgroup$
– Michel Keijzers
Apr 18 at 22:08
$begingroup$
@MichelKeijzers - yes, but that’s the second solution you mention. Your first solution is missing an explanation of escape sequences to prevent the stop bytes being transmitted.
$endgroup$
– Dan W
Apr 18 at 22:25
$begingroup$
@MichelKeijzers - yes, but that’s the second solution you mention. Your first solution is missing an explanation of escape sequences to prevent the stop bytes being transmitted.
$endgroup$
– Dan W
Apr 18 at 22:25
$begingroup$
Both approaches work, and are commonly used, but they have different advantages and disadvantages, which you could add if wanted, though it’s beyond what the OP asked for.
$endgroup$
– Dan W
Apr 18 at 22:27
$begingroup$
Both approaches work, and are commonly used, but they have different advantages and disadvantages, which you could add if wanted, though it’s beyond what the OP asked for.
$endgroup$
– Dan W
Apr 18 at 22:27
|
show 1 more comment
$begingroup$
How fail-safe is nr as stop bytes?
If you send send arbitrary data -> probably not fail-safe enough.
A common solution is to use escaping:
Let's define that the characters 0x02 (STX - frame start) and 0x03 (ETX - frame end) need to be unique within the transmitted data stream. This way the start and the end of a message can be safely detected.
If one of these characters should be send within the message frame, it is replaced by prefixing an escape character (ESC = 0x1b) and adding 0x20 to the original character.
Original character replaced by
0x02 -> 0x1b 0x22
0x03 -> 0x1b 0x23
0x1b -> 0x1b 0x3b
The receiver reverses this process: Anytime he receives an escape character, this character is dropped and the next character is subtracted by 0x20.
This only adds some processing overhead but is 100% reliable (assuming no transmission errors occur, which you could/should verify by additionally implementing a checksum mechanism).
answered Apr 18 at 11:05
Rev1.0Rev1.0
7,72043369
$endgroup$
1
$begingroup$
Nice answer. The common escape character used for ASCII protocols was'x10'DLE (Data Link Escape). Some of the Wikipedia pages suggest that DLE was often used in the opposite way: to say that the next byte was a control character rather than a data byte. In my experience, that's generally the opposite meaning for an escape.
$endgroup$
– Adrian McCarthy
Apr 18 at 21:23
2
$begingroup$
One thing to watch our for here is that your worst case buffer size doubles. If memory is really tight that might not be the best solution.
$endgroup$
– TechnoSam
Apr 18 at 21:23
1
$begingroup$
@Rev What's the rationale for adding 0x20 to the original character? Wouldn't the escaping scheme work without that just as well?
$endgroup$
– Nick Alexeev♦
Apr 19 at 21:02
$begingroup$
@NickAlexeev: It is easier/faster to identify the actual frame boundaries if you remove any other occurrence of the reserved chars from the stream. That way, you can seperate frame reception and frame parsing (including the un-escaping). This may be especially relevant, if you have a very slow controller without FIFO and/or high data rates. So you can just copy the incoming bytes (between STX/ETX) into the frame buffer as they arrive, mark the frame as complete and do the processing with lower priority.
$endgroup$
– Rev1.0
Apr 21 at 18:08
$begingroup$
@TechnoSam: Good point.
$endgroup$
– Rev1.0
Apr 21 at 18:08
add a comment |
$begingroup$
How fail-safe is nr as stop bytes?
If you send send arbitrary data -> probably not fail-safe enough.
A common solution is to use escaping:
Let's define that the characters 0x02 (STX - frame start) and 0x03 (ETX - frame end) need to be unique within the transmitted data stream. This way the start and the end of a message can be safely detected.
If one of these characters should be send within the message frame, it is replaced by prefixing an escape character (ESC = 0x1b) and adding 0x20 to the original character.
Original character replaced by
0x02 -> 0x1b 0x22
0x03 -> 0x1b 0x23
0x1b -> 0x1b 0x3b
The receiver reverses this process: Anytime he receives an escape character, this character is dropped and the next character is subtracted by 0x20.
This only adds some processing overhead but is 100% reliable (assuming no transmission errors occur, which you could/should verify by additionally implementing a checksum mechanism).
answered Apr 18 at 11:05
Rev1.0Rev1.0
7,72043369
$endgroup$
1
$begingroup$
Nice answer. The common escape character used for ASCII protocols was'x10'DLE (Data Link Escape). Some of the Wikipedia pages suggest that DLE was often used in the opposite way: to say that the next byte was a control character rather than a data byte. In my experience, that's generally the opposite meaning for an escape.
$endgroup$
– Adrian McCarthy
Apr 18 at 21:23
2
$begingroup$
One thing to watch our for here is that your worst case buffer size doubles. If memory is really tight that might not be the best solution.
$endgroup$
– TechnoSam
Apr 18 at 21:23
1
$begingroup$
@Rev What's the rationale for adding 0x20 to the original character? Wouldn't the escaping scheme work without that just as well?
$endgroup$
– Nick Alexeev♦
Apr 19 at 21:02
$begingroup$
@NickAlexeev: It is easier/faster to identify the actual frame boundaries if you remove any other occurrence of the reserved chars from the stream. That way, you can seperate frame reception and frame parsing (including the un-escaping). This may be especially relevant, if you have a very slow controller without FIFO and/or high data rates. So you can just copy the incoming bytes (between STX/ETX) into the frame buffer as they arrive, mark the frame as complete and do the processing with lower priority.
$endgroup$
– Rev1.0
Apr 21 at 18:08
$begingroup$
@TechnoSam: Good point.
$endgroup$
– Rev1.0
Apr 21 at 18:08
add a comment |
$begingroup$
How fail-safe is nr as stop bytes?
If you send send arbitrary data -> probably not fail-safe enough.
A common solution is to use escaping:
Let's define that the characters 0x02 (STX - frame start) and 0x03 (ETX - frame end) need to be unique within the transmitted data stream. This way the start and the end of a message can be safely detected.
If one of these characters should be send within the message frame, it is replaced by prefixing an escape character (ESC = 0x1b) and adding 0x20 to the original character.
Original character replaced by
0x02 -> 0x1b 0x22
0x03 -> 0x1b 0x23
0x1b -> 0x1b 0x3b
The receiver reverses this process: Anytime he receives an escape character, this character is dropped and the next character is subtracted by 0x20.
This only adds some processing overhead but is 100% reliable (assuming no transmission errors occur, which you could/should verify by additionally implementing a checksum mechanism).
answered Apr 18 at 11:05
Rev1.0Rev1.0
7,72043369
$endgroup$
How fail-safe is nr as stop bytes?
If you send send arbitrary data -> probably not fail-safe enough.
A common solution is to use escaping:
Let's define that the characters 0x02 (STX - frame start) and 0x03 (ETX - frame end) need to be unique within the transmitted data stream. This way the start and the end of a message can be safely detected.
If one of these characters should be send within the message frame, it is replaced by prefixing an escape character (ESC = 0x1b) and adding 0x20 to the original character.
Original character replaced by
0x02 -> 0x1b 0x22
0x03 -> 0x1b 0x23
0x1b -> 0x1b 0x3b
The receiver reverses this process: Anytime he receives an escape character, this character is dropped and the next character is subtracted by 0x20.
This only adds some processing overhead but is 100% reliable (assuming no transmission errors occur, which you could/should verify by additionally implementing a checksum mechanism).
answered Apr 18 at 11:05
Rev1.0Rev1.0
7,72043369
answered Apr 18 at 11:05
Rev1.0Rev1.0
7,72043369
answered Apr 18 at 11:05
Rev1.0Rev1.0
7,72043369
answered Apr 18 at 11:05
Rev1.0Rev1.0
7,72043369
7,72043369
1
$begingroup$
Nice answer. The common escape character used for ASCII protocols was'x10'DLE (Data Link Escape). Some of the Wikipedia pages suggest that DLE was often used in the opposite way: to say that the next byte was a control character rather than a data byte. In my experience, that's generally the opposite meaning for an escape.
$endgroup$
– Adrian McCarthy
Apr 18 at 21:23
2
$begingroup$
One thing to watch our for here is that your worst case buffer size doubles. If memory is really tight that might not be the best solution.
$endgroup$
– TechnoSam
Apr 18 at 21:23
1
$begingroup$
@Rev What's the rationale for adding 0x20 to the original character? Wouldn't the escaping scheme work without that just as well?
$endgroup$
– Nick Alexeev♦
Apr 19 at 21:02
$begingroup$
@NickAlexeev: It is easier/faster to identify the actual frame boundaries if you remove any other occurrence of the reserved chars from the stream. That way, you can seperate frame reception and frame parsing (including the un-escaping). This may be especially relevant, if you have a very slow controller without FIFO and/or high data rates. So you can just copy the incoming bytes (between STX/ETX) into the frame buffer as they arrive, mark the frame as complete and do the processing with lower priority.
$endgroup$
– Rev1.0
Apr 21 at 18:08
$begingroup$
@TechnoSam: Good point.
$endgroup$
– Rev1.0
Apr 21 at 18:08
add a comment |
1
$begingroup$
Nice answer. The common escape character used for ASCII protocols was'x10'DLE (Data Link Escape). Some of the Wikipedia pages suggest that DLE was often used in the opposite way: to say that the next byte was a control character rather than a data byte. In my experience, that's generally the opposite meaning for an escape.
$endgroup$
– Adrian McCarthy
Apr 18 at 21:23
2
$begingroup$
One thing to watch our for here is that your worst case buffer size doubles. If memory is really tight that might not be the best solution.
$endgroup$
– TechnoSam
Apr 18 at 21:23
1
$begingroup$
@Rev What's the rationale for adding 0x20 to the original character? Wouldn't the escaping scheme work without that just as well?
$endgroup$
– Nick Alexeev♦
Apr 19 at 21:02
$begingroup$
@NickAlexeev: It is easier/faster to identify the actual frame boundaries if you remove any other occurrence of the reserved chars from the stream. That way, you can seperate frame reception and frame parsing (including the un-escaping). This may be especially relevant, if you have a very slow controller without FIFO and/or high data rates. So you can just copy the incoming bytes (between STX/ETX) into the frame buffer as they arrive, mark the frame as complete and do the processing with lower priority.
$endgroup$
– Rev1.0
Apr 21 at 18:08
$begingroup$
@TechnoSam: Good point.
$endgroup$
– Rev1.0
Apr 21 at 18:08
1
1
$begingroup$
Nice answer. The common escape character used for ASCII protocols was
'x10' DLE (Data Link Escape). Some of the Wikipedia pages suggest that DLE was often used in the opposite way: to say that the next byte was a control character rather than a data byte. In my experience, that's generally the opposite meaning for an escape.$endgroup$
– Adrian McCarthy
Apr 18 at 21:23
$begingroup$
Nice answer. The common escape character used for ASCII protocols was
'x10' DLE (Data Link Escape). Some of the Wikipedia pages suggest that DLE was often used in the opposite way: to say that the next byte was a control character rather than a data byte. In my experience, that's generally the opposite meaning for an escape.$endgroup$
– Adrian McCarthy
Apr 18 at 21:23
2
2
$begingroup$
One thing to watch our for here is that your worst case buffer size doubles. If memory is really tight that might not be the best solution.
$endgroup$
– TechnoSam
Apr 18 at 21:23
$begingroup$
One thing to watch our for here is that your worst case buffer size doubles. If memory is really tight that might not be the best solution.
$endgroup$
– TechnoSam
Apr 18 at 21:23
1
1
$begingroup$
@Rev What's the rationale for adding 0x20 to the original character? Wouldn't the escaping scheme work without that just as well?
$endgroup$
– Nick Alexeev♦
Apr 19 at 21:02
$begingroup$
@Rev What's the rationale for adding 0x20 to the original character? Wouldn't the escaping scheme work without that just as well?
$endgroup$
– Nick Alexeev♦
Apr 19 at 21:02
$begingroup$
@NickAlexeev: It is easier/faster to identify the actual frame boundaries if you remove any other occurrence of the reserved chars from the stream. That way, you can seperate frame reception and frame parsing (including the un-escaping). This may be especially relevant, if you have a very slow controller without FIFO and/or high data rates. So you can just copy the incoming bytes (between STX/ETX) into the frame buffer as they arrive, mark the frame as complete and do the processing with lower priority.
$endgroup$
– Rev1.0
Apr 21 at 18:08
$begingroup$
@NickAlexeev: It is easier/faster to identify the actual frame boundaries if you remove any other occurrence of the reserved chars from the stream. That way, you can seperate frame reception and frame parsing (including the un-escaping). This may be especially relevant, if you have a very slow controller without FIFO and/or high data rates. So you can just copy the incoming bytes (between STX/ETX) into the frame buffer as they arrive, mark the frame as complete and do the processing with lower priority.
$endgroup$
– Rev1.0
Apr 21 at 18:08
$begingroup$
@TechnoSam: Good point.
$endgroup$
– Rev1.0
Apr 21 at 18:08
$begingroup$
@TechnoSam: Good point.
$endgroup$
– Rev1.0
Apr 21 at 18:08
add a comment |
$begingroup$
You know, ASCII already has bytes for these functions.
- 0x01 : start of heading -- start byte
- 0x02 : start of text -- end headers, begin payload
- 0x03 : end of text -- end payload
- 0x04 : end of transmission -- stop byte
- 0x17 : end of transmission block -- message continues in next block
It also has codes for various uses inside the payload.
- 0x1b : escape (escape the next character -- use in payload to indicate next character is not one of the structure describing codes used in your protocol)
- 0x1c, 0x1d, 0x1e, 0x1f : file, group, record, and unit separator, respectively -- used as simultaneous stop and start byte for parts of hierarchical data
Your protocol should specify the finest granularity of ACK (0x06) and NAK (0x15), so that negative acknowledged data can be retransmitted. Down to this finest granularity, it is wise to have a length field immediately after any (unescaped) start indicator and (as explained in other answer(s)) it is wise to follow any (unescaped) stop indicator with a CRC.
answered Apr 18 at 15:55
Eric TowersEric Towers
20912
$endgroup$
$begingroup$
I will be sending arbitrary data, I guess it might have been confusing to use "nr" in my question when I'm not sending ASCII data. Even though, I like this answer, it's very informative on sending ASCII over UART
$endgroup$
– Marius Gulbrandsen
Apr 19 at 21:50
$begingroup$
@MariusGulbrandsen : As long as your protocol establishes where payload is and which codes must be escaped in each payload section, you can send anything, not just text-ish data.
$endgroup$
– Eric Towers
Apr 19 at 22:57
add a comment |
$begingroup$
You know, ASCII already has bytes for these functions.
- 0x01 : start of heading -- start byte
- 0x02 : start of text -- end headers, begin payload
- 0x03 : end of text -- end payload
- 0x04 : end of transmission -- stop byte
- 0x17 : end of transmission block -- message continues in next block
It also has codes for various uses inside the payload.
- 0x1b : escape (escape the next character -- use in payload to indicate next character is not one of the structure describing codes used in your protocol)
- 0x1c, 0x1d, 0x1e, 0x1f : file, group, record, and unit separator, respectively -- used as simultaneous stop and start byte for parts of hierarchical data
Your protocol should specify the finest granularity of ACK (0x06) and NAK (0x15), so that negative acknowledged data can be retransmitted. Down to this finest granularity, it is wise to have a length field immediately after any (unescaped) start indicator and (as explained in other answer(s)) it is wise to follow any (unescaped) stop indicator with a CRC.
answered Apr 18 at 15:55
Eric TowersEric Towers
20912
$endgroup$
$begingroup$
I will be sending arbitrary data, I guess it might have been confusing to use "nr" in my question when I'm not sending ASCII data. Even though, I like this answer, it's very informative on sending ASCII over UART
$endgroup$
– Marius Gulbrandsen
Apr 19 at 21:50
$begingroup$
@MariusGulbrandsen : As long as your protocol establishes where payload is and which codes must be escaped in each payload section, you can send anything, not just text-ish data.
$endgroup$
– Eric Towers
Apr 19 at 22:57
add a comment |
$begingroup$
You know, ASCII already has bytes for these functions.
- 0x01 : start of heading -- start byte
- 0x02 : start of text -- end headers, begin payload
- 0x03 : end of text -- end payload
- 0x04 : end of transmission -- stop byte
- 0x17 : end of transmission block -- message continues in next block
It also has codes for various uses inside the payload.
- 0x1b : escape (escape the next character -- use in payload to indicate next character is not one of the structure describing codes used in your protocol)
- 0x1c, 0x1d, 0x1e, 0x1f : file, group, record, and unit separator, respectively -- used as simultaneous stop and start byte for parts of hierarchical data
Your protocol should specify the finest granularity of ACK (0x06) and NAK (0x15), so that negative acknowledged data can be retransmitted. Down to this finest granularity, it is wise to have a length field immediately after any (unescaped) start indicator and (as explained in other answer(s)) it is wise to follow any (unescaped) stop indicator with a CRC.
answered Apr 18 at 15:55
Eric TowersEric Towers
20912
$endgroup$
You know, ASCII already has bytes for these functions.
- 0x01 : start of heading -- start byte
- 0x02 : start of text -- end headers, begin payload
- 0x03 : end of text -- end payload
- 0x04 : end of transmission -- stop byte
- 0x17 : end of transmission block -- message continues in next block
It also has codes for various uses inside the payload.
- 0x1b : escape (escape the next character -- use in payload to indicate next character is not one of the structure describing codes used in your protocol)
- 0x1c, 0x1d, 0x1e, 0x1f : file, group, record, and unit separator, respectively -- used as simultaneous stop and start byte for parts of hierarchical data
Your protocol should specify the finest granularity of ACK (0x06) and NAK (0x15), so that negative acknowledged data can be retransmitted. Down to this finest granularity, it is wise to have a length field immediately after any (unescaped) start indicator and (as explained in other answer(s)) it is wise to follow any (unescaped) stop indicator with a CRC.
answered Apr 18 at 15:55
Eric TowersEric Towers
20912
answered Apr 18 at 15:55
Eric TowersEric Towers
20912
answered Apr 18 at 15:55
Eric TowersEric Towers
20912
answered Apr 18 at 15:55
Eric TowersEric Towers
20912
20912
$begingroup$
I will be sending arbitrary data, I guess it might have been confusing to use "nr" in my question when I'm not sending ASCII data. Even though, I like this answer, it's very informative on sending ASCII over UART
$endgroup$
– Marius Gulbrandsen
Apr 19 at 21:50
$begingroup$
@MariusGulbrandsen : As long as your protocol establishes where payload is and which codes must be escaped in each payload section, you can send anything, not just text-ish data.
$endgroup$
– Eric Towers
Apr 19 at 22:57
add a comment |
$begingroup$
I will be sending arbitrary data, I guess it might have been confusing to use "nr" in my question when I'm not sending ASCII data. Even though, I like this answer, it's very informative on sending ASCII over UART
$endgroup$
– Marius Gulbrandsen
Apr 19 at 21:50
$begingroup$
@MariusGulbrandsen : As long as your protocol establishes where payload is and which codes must be escaped in each payload section, you can send anything, not just text-ish data.
$endgroup$
– Eric Towers
Apr 19 at 22:57
$begingroup$
I will be sending arbitrary data, I guess it might have been confusing to use "nr" in my question when I'm not sending ASCII data. Even though, I like this answer, it's very informative on sending ASCII over UART
$endgroup$
– Marius Gulbrandsen
Apr 19 at 21:50
$begingroup$
I will be sending arbitrary data, I guess it might have been confusing to use "nr" in my question when I'm not sending ASCII data. Even though, I like this answer, it's very informative on sending ASCII over UART
$endgroup$
– Marius Gulbrandsen
Apr 19 at 21:50
$begingroup$
@MariusGulbrandsen : As long as your protocol establishes where payload is and which codes must be escaped in each payload section, you can send anything, not just text-ish data.
$endgroup$
– Eric Towers
Apr 19 at 22:57
$begingroup$
@MariusGulbrandsen : As long as your protocol establishes where payload is and which codes must be escaped in each payload section, you can send anything, not just text-ish data.
$endgroup$
– Eric Towers
Apr 19 at 22:57
add a comment |
$begingroup$
UART is not fail-safe by its very nature - we are talking about 1960s technology here.
The root of the problem being that UART only syncs once per 10 bits, allowing a lot of gibberish to pass between those sync periods. Unlike for example CAN which samples every individual bit multiple times.
Any double bit error occurring inside the data will corrupt an UART frame and pass undetected. Bit errors in start/stop bits may or may not get detected in the form of overrun errors.
Therefore, no matter if you use raw data or packets, there is always a probability that bit flips caused by EMI result in unexpected data.
There exist numerous ways of "traditional UART quackery" to improve the situation ever so slightly. You can add sync bytes, sync bits, parity, double stop bits. You could add checksums that count the sum of all bytes (and then invert it - because why not) or you could count the number of binary ones as a checksum. All of this is widely used, wildly unscientific and with a high probability of missing errors. But this was what people did from 1960s to 1990s and lots of weird things like these lives on today.
The most professional way to deal with safe transmission over UART is to have a 16 bit CRC checksum at the end of the packet. Everything else isn't very safe and has a high probability of missing errors.
Then on the hardware level you can use differential RS-422/RS-485 to drastically improve ruggedness of the transmission. This is a must for safe transmission over longer distances. TTL level UART should only be used for on-board communication. RS-232 should not be used for any other purpose but backwards compatibility with old stuff.
Overall, the closer to the hardware your error detection mechanism is, the more effective it is. In terms of effectiveness, differential signals add the most, followed by checking for framing/overrun etc errors. CRC16 adds some, and then "traditional UART quackery" adds a little bit.
answered Apr 18 at 11:27
LundinLundin
4,5291131
$endgroup$
7
$begingroup$
This advice is fairly tangential - you haven't actually addressed the question asked. In particular, your proposed solutions may solve other problems, but they do not solve the basic problem of the question on this page, which is confusion between framing byes and payload byes. At most, your proposal would reject valid data embedding a framing byte due to CRC or similar failure, with no way to communicate such.
$endgroup$
– Chris Stratton
Apr 18 at 14:28
3
$begingroup$
In fact, this answer makes it worse. The original had just data bytes and stop bytes. This adds a third category, CRC bytes. And as presented here, those can take on any value, including {10,13}.
$endgroup$
– MSalters
Apr 18 at 15:53
1
$begingroup$
@MSalters: The CRC can be ASCII encoded hex to prevent this issue. Another trick that I've seen on RS485 is to set bit 7 on the start / address byte.
$endgroup$
– Transistor
Apr 18 at 16:03
$begingroup$
Re "CAN which samples every individual bit multiple times.": The actual sampling of the bit value is only once per bit. What are you referring to here? Some kind of error checking, like by the sender? Clock synchronisation?
$endgroup$
– Peter Mortensen
Apr 19 at 12:33
$begingroup$
The inverting of the checksum was done so that summing the entire block of data would result in a zero, which is a bit easier to code and a bit faster to execute. Also, CRC is much better than you make it out to be, look it up in the Wikipedia.
$endgroup$
– toolforger
Apr 19 at 22:04
|
show 6 more comments
$begingroup$
UART is not fail-safe by its very nature - we are talking about 1960s technology here.
The root of the problem being that UART only syncs once per 10 bits, allowing a lot of gibberish to pass between those sync periods. Unlike for example CAN which samples every individual bit multiple times.
Any double bit error occurring inside the data will corrupt an UART frame and pass undetected. Bit errors in start/stop bits may or may not get detected in the form of overrun errors.
Therefore, no matter if you use raw data or packets, there is always a probability that bit flips caused by EMI result in unexpected data.
There exist numerous ways of "traditional UART quackery" to improve the situation ever so slightly. You can add sync bytes, sync bits, parity, double stop bits. You could add checksums that count the sum of all bytes (and then invert it - because why not) or you could count the number of binary ones as a checksum. All of this is widely used, wildly unscientific and with a high probability of missing errors. But this was what people did from 1960s to 1990s and lots of weird things like these lives on today.
The most professional way to deal with safe transmission over UART is to have a 16 bit CRC checksum at the end of the packet. Everything else isn't very safe and has a high probability of missing errors.
Then on the hardware level you can use differential RS-422/RS-485 to drastically improve ruggedness of the transmission. This is a must for safe transmission over longer distances. TTL level UART should only be used for on-board communication. RS-232 should not be used for any other purpose but backwards compatibility with old stuff.
Overall, the closer to the hardware your error detection mechanism is, the more effective it is. In terms of effectiveness, differential signals add the most, followed by checking for framing/overrun etc errors. CRC16 adds some, and then "traditional UART quackery" adds a little bit.
answered Apr 18 at 11:27
LundinLundin
4,5291131
$endgroup$
7
$begingroup$
This advice is fairly tangential - you haven't actually addressed the question asked. In particular, your proposed solutions may solve other problems, but they do not solve the basic problem of the question on this page, which is confusion between framing byes and payload byes. At most, your proposal would reject valid data embedding a framing byte due to CRC or similar failure, with no way to communicate such.
$endgroup$
– Chris Stratton
Apr 18 at 14:28
3
$begingroup$
In fact, this answer makes it worse. The original had just data bytes and stop bytes. This adds a third category, CRC bytes. And as presented here, those can take on any value, including {10,13}.
$endgroup$
– MSalters
Apr 18 at 15:53
1
$begingroup$
@MSalters: The CRC can be ASCII encoded hex to prevent this issue. Another trick that I've seen on RS485 is to set bit 7 on the start / address byte.
$endgroup$
– Transistor
Apr 18 at 16:03
$begingroup$
Re "CAN which samples every individual bit multiple times.": The actual sampling of the bit value is only once per bit. What are you referring to here? Some kind of error checking, like by the sender? Clock synchronisation?
$endgroup$
– Peter Mortensen
Apr 19 at 12:33
$begingroup$
The inverting of the checksum was done so that summing the entire block of data would result in a zero, which is a bit easier to code and a bit faster to execute. Also, CRC is much better than you make it out to be, look it up in the Wikipedia.
$endgroup$
– toolforger
Apr 19 at 22:04
|
show 6 more comments
$begingroup$
UART is not fail-safe by its very nature - we are talking about 1960s technology here.
The root of the problem being that UART only syncs once per 10 bits, allowing a lot of gibberish to pass between those sync periods. Unlike for example CAN which samples every individual bit multiple times.
Any double bit error occurring inside the data will corrupt an UART frame and pass undetected. Bit errors in start/stop bits may or may not get detected in the form of overrun errors.
Therefore, no matter if you use raw data or packets, there is always a probability that bit flips caused by EMI result in unexpected data.
There exist numerous ways of "traditional UART quackery" to improve the situation ever so slightly. You can add sync bytes, sync bits, parity, double stop bits. You could add checksums that count the sum of all bytes (and then invert it - because why not) or you could count the number of binary ones as a checksum. All of this is widely used, wildly unscientific and with a high probability of missing errors. But this was what people did from 1960s to 1990s and lots of weird things like these lives on today.
The most professional way to deal with safe transmission over UART is to have a 16 bit CRC checksum at the end of the packet. Everything else isn't very safe and has a high probability of missing errors.
Then on the hardware level you can use differential RS-422/RS-485 to drastically improve ruggedness of the transmission. This is a must for safe transmission over longer distances. TTL level UART should only be used for on-board communication. RS-232 should not be used for any other purpose but backwards compatibility with old stuff.
Overall, the closer to the hardware your error detection mechanism is, the more effective it is. In terms of effectiveness, differential signals add the most, followed by checking for framing/overrun etc errors. CRC16 adds some, and then "traditional UART quackery" adds a little bit.
answered Apr 18 at 11:27
LundinLundin
4,5291131
$endgroup$
UART is not fail-safe by its very nature - we are talking about 1960s technology here.
The root of the problem being that UART only syncs once per 10 bits, allowing a lot of gibberish to pass between those sync periods. Unlike for example CAN which samples every individual bit multiple times.
Any double bit error occurring inside the data will corrupt an UART frame and pass undetected. Bit errors in start/stop bits may or may not get detected in the form of overrun errors.
Therefore, no matter if you use raw data or packets, there is always a probability that bit flips caused by EMI result in unexpected data.
There exist numerous ways of "traditional UART quackery" to improve the situation ever so slightly. You can add sync bytes, sync bits, parity, double stop bits. You could add checksums that count the sum of all bytes (and then invert it - because why not) or you could count the number of binary ones as a checksum. All of this is widely used, wildly unscientific and with a high probability of missing errors. But this was what people did from 1960s to 1990s and lots of weird things like these lives on today.
The most professional way to deal with safe transmission over UART is to have a 16 bit CRC checksum at the end of the packet. Everything else isn't very safe and has a high probability of missing errors.
Then on the hardware level you can use differential RS-422/RS-485 to drastically improve ruggedness of the transmission. This is a must for safe transmission over longer distances. TTL level UART should only be used for on-board communication. RS-232 should not be used for any other purpose but backwards compatibility with old stuff.
Overall, the closer to the hardware your error detection mechanism is, the more effective it is. In terms of effectiveness, differential signals add the most, followed by checking for framing/overrun etc errors. CRC16 adds some, and then "traditional UART quackery" adds a little bit.
answered Apr 18 at 11:27
LundinLundin
4,5291131
edited Apr 18 at 11:38
answered Apr 18 at 11:27
LundinLundin
4,5291131
answered Apr 18 at 11:27
LundinLundin
4,5291131
answered Apr 18 at 11:27
LundinLundin
4,5291131
4,5291131
7
$begingroup$
This advice is fairly tangential - you haven't actually addressed the question asked. In particular, your proposed solutions may solve other problems, but they do not solve the basic problem of the question on this page, which is confusion between framing byes and payload byes. At most, your proposal would reject valid data embedding a framing byte due to CRC or similar failure, with no way to communicate such.
$endgroup$
– Chris Stratton
Apr 18 at 14:28
3
$begingroup$
In fact, this answer makes it worse. The original had just data bytes and stop bytes. This adds a third category, CRC bytes. And as presented here, those can take on any value, including {10,13}.
$endgroup$
– MSalters
Apr 18 at 15:53
1
$begingroup$
@MSalters: The CRC can be ASCII encoded hex to prevent this issue. Another trick that I've seen on RS485 is to set bit 7 on the start / address byte.
$endgroup$
– Transistor
Apr 18 at 16:03
$begingroup$
Re "CAN which samples every individual bit multiple times.": The actual sampling of the bit value is only once per bit. What are you referring to here? Some kind of error checking, like by the sender? Clock synchronisation?
$endgroup$
– Peter Mortensen
Apr 19 at 12:33
$begingroup$
The inverting of the checksum was done so that summing the entire block of data would result in a zero, which is a bit easier to code and a bit faster to execute. Also, CRC is much better than you make it out to be, look it up in the Wikipedia.
$endgroup$
– toolforger
Apr 19 at 22:04
|
show 6 more comments
7
$begingroup$
This advice is fairly tangential - you haven't actually addressed the question asked. In particular, your proposed solutions may solve other problems, but they do not solve the basic problem of the question on this page, which is confusion between framing byes and payload byes. At most, your proposal would reject valid data embedding a framing byte due to CRC or similar failure, with no way to communicate such.
$endgroup$
– Chris Stratton
Apr 18 at 14:28
3
$begingroup$
In fact, this answer makes it worse. The original had just data bytes and stop bytes. This adds a third category, CRC bytes. And as presented here, those can take on any value, including {10,13}.
$endgroup$
– MSalters
Apr 18 at 15:53
1
$begingroup$
@MSalters: The CRC can be ASCII encoded hex to prevent this issue. Another trick that I've seen on RS485 is to set bit 7 on the start / address byte.
$endgroup$
– Transistor
Apr 18 at 16:03
$begingroup$
Re "CAN which samples every individual bit multiple times.": The actual sampling of the bit value is only once per bit. What are you referring to here? Some kind of error checking, like by the sender? Clock synchronisation?
$endgroup$
– Peter Mortensen
Apr 19 at 12:33
$begingroup$
The inverting of the checksum was done so that summing the entire block of data would result in a zero, which is a bit easier to code and a bit faster to execute. Also, CRC is much better than you make it out to be, look it up in the Wikipedia.
$endgroup$
– toolforger
Apr 19 at 22:04
7
7
$begingroup$
This advice is fairly tangential - you haven't actually addressed the question asked. In particular, your proposed solutions may solve other problems, but they do not solve the basic problem of the question on this page, which is confusion between framing byes and payload byes. At most, your proposal would reject valid data embedding a framing byte due to CRC or similar failure, with no way to communicate such.
$endgroup$
– Chris Stratton
Apr 18 at 14:28
$begingroup$
This advice is fairly tangential - you haven't actually addressed the question asked. In particular, your proposed solutions may solve other problems, but they do not solve the basic problem of the question on this page, which is confusion between framing byes and payload byes. At most, your proposal would reject valid data embedding a framing byte due to CRC or similar failure, with no way to communicate such.
$endgroup$
– Chris Stratton
Apr 18 at 14:28
3
3
$begingroup$
In fact, this answer makes it worse. The original had just data bytes and stop bytes. This adds a third category, CRC bytes. And as presented here, those can take on any value, including {10,13}.
$endgroup$
– MSalters
Apr 18 at 15:53
$begingroup$
In fact, this answer makes it worse. The original had just data bytes and stop bytes. This adds a third category, CRC bytes. And as presented here, those can take on any value, including {10,13}.
$endgroup$
– MSalters
Apr 18 at 15:53
1
1
$begingroup$
@MSalters: The CRC can be ASCII encoded hex to prevent this issue. Another trick that I've seen on RS485 is to set bit 7 on the start / address byte.
$endgroup$
– Transistor
Apr 18 at 16:03
$begingroup$
@MSalters: The CRC can be ASCII encoded hex to prevent this issue. Another trick that I've seen on RS485 is to set bit 7 on the start / address byte.
$endgroup$
– Transistor
Apr 18 at 16:03
$begingroup$
Re "CAN which samples every individual bit multiple times.": The actual sampling of the bit value is only once per bit. What are you referring to here? Some kind of error checking, like by the sender? Clock synchronisation?
$endgroup$
– Peter Mortensen
Apr 19 at 12:33
$begingroup$
Re "CAN which samples every individual bit multiple times.": The actual sampling of the bit value is only once per bit. What are you referring to here? Some kind of error checking, like by the sender? Clock synchronisation?
$endgroup$
– Peter Mortensen
Apr 19 at 12:33
$begingroup$
The inverting of the checksum was done so that summing the entire block of data would result in a zero, which is a bit easier to code and a bit faster to execute. Also, CRC is much better than you make it out to be, look it up in the Wikipedia.
$endgroup$
– toolforger
Apr 19 at 22:04
$begingroup$
The inverting of the checksum was done so that summing the entire block of data would result in a zero, which is a bit easier to code and a bit faster to execute. Also, CRC is much better than you make it out to be, look it up in the Wikipedia.
$endgroup$
– toolforger
Apr 19 at 22:04
|
show 6 more comments
$begingroup$
... I can imagine, though low probability, that my message might contain the values "10 and 13" after each other when they are not the stop bytes.
A situation when a portion of data is equal to terminating sequence should be considered when designing the format of a serial data packet. Another thing to consider is that any character can get corrupted or lost during transmission. A start character, a stop character, a data payload byte, a checksum or CRC byte, a forward error correction byte aren't immune to corruption. The framing mechanism has to be able to detect when a packet has corrupt data.
There a several ways to approach all this.
I'm making the working assumption that packets are framed only with the serial bytes. Handshake lines aren't used for framing. Time delays aren't used for framing.
Send packet length
Send the length of the packet in the beginning, instead of [or in addition to] the terminating character at the end.
pros: Payload is sent in a efficient binary format.
cons: Need to know the packet length at the start of the transmission.
Escape the special characters
Escape the special characters when sending the payload data. This is already explained in a an earlier answer.
pros: Sender doesn't need to know the length of the packet at the beginning of the transmission.
cons: Slightly less efficient, depending on how many payload bytes need to be escaped.
Payload data encoded such that it can't contain start and stop characters
The payload of the packet is encoded such that it can't contain the start or stop characters. Usually, this is done by sending numbers as their ASCII or Hex-ASCII representation.
pros: Human-readable with common terminal programs. No need for code to handle escaping. No need to know the length of the packet at the start of the transmission
cons: Lower efficiency. For one byte of payload data, several bytes are sent.
answered Apr 19 at 21:07
Nick Alexeev♦Nick Alexeev
32.7k1066167
$endgroup$
add a comment |
$begingroup$
... I can imagine, though low probability, that my message might contain the values "10 and 13" after each other when they are not the stop bytes.
A situation when a portion of data is equal to terminating sequence should be considered when designing the format of a serial data packet. Another thing to consider is that any character can get corrupted or lost during transmission. A start character, a stop character, a data payload byte, a checksum or CRC byte, a forward error correction byte aren't immune to corruption. The framing mechanism has to be able to detect when a packet has corrupt data.
There a several ways to approach all this.
I'm making the working assumption that packets are framed only with the serial bytes. Handshake lines aren't used for framing. Time delays aren't used for framing.
Send packet length
Send the length of the packet in the beginning, instead of [or in addition to] the terminating character at the end.
pros: Payload is sent in a efficient binary format.
cons: Need to know the packet length at the start of the transmission.
Escape the special characters
Escape the special characters when sending the payload data. This is already explained in a an earlier answer.
pros: Sender doesn't need to know the length of the packet at the beginning of the transmission.
cons: Slightly less efficient, depending on how many payload bytes need to be escaped.
Payload data encoded such that it can't contain start and stop characters
The payload of the packet is encoded such that it can't contain the start or stop characters. Usually, this is done by sending numbers as their ASCII or Hex-ASCII representation.
pros: Human-readable with common terminal programs. No need for code to handle escaping. No need to know the length of the packet at the start of the transmission
cons: Lower efficiency. For one byte of payload data, several bytes are sent.
answered Apr 19 at 21:07
Nick Alexeev♦Nick Alexeev
32.7k1066167
$endgroup$
add a comment |
$begingroup$
... I can imagine, though low probability, that my message might contain the values "10 and 13" after each other when they are not the stop bytes.
A situation when a portion of data is equal to terminating sequence should be considered when designing the format of a serial data packet. Another thing to consider is that any character can get corrupted or lost during transmission. A start character, a stop character, a data payload byte, a checksum or CRC byte, a forward error correction byte aren't immune to corruption. The framing mechanism has to be able to detect when a packet has corrupt data.
There a several ways to approach all this.
I'm making the working assumption that packets are framed only with the serial bytes. Handshake lines aren't used for framing. Time delays aren't used for framing.
Send packet length
Send the length of the packet in the beginning, instead of [or in addition to] the terminating character at the end.
pros: Payload is sent in a efficient binary format.
cons: Need to know the packet length at the start of the transmission.
Escape the special characters
Escape the special characters when sending the payload data. This is already explained in a an earlier answer.
pros: Sender doesn't need to know the length of the packet at the beginning of the transmission.
cons: Slightly less efficient, depending on how many payload bytes need to be escaped.
Payload data encoded such that it can't contain start and stop characters
The payload of the packet is encoded such that it can't contain the start or stop characters. Usually, this is done by sending numbers as their ASCII or Hex-ASCII representation.
pros: Human-readable with common terminal programs. No need for code to handle escaping. No need to know the length of the packet at the start of the transmission
cons: Lower efficiency. For one byte of payload data, several bytes are sent.
answered Apr 19 at 21:07
Nick Alexeev♦Nick Alexeev
32.7k1066167
$endgroup$
... I can imagine, though low probability, that my message might contain the values "10 and 13" after each other when they are not the stop bytes.
A situation when a portion of data is equal to terminating sequence should be considered when designing the format of a serial data packet. Another thing to consider is that any character can get corrupted or lost during transmission. A start character, a stop character, a data payload byte, a checksum or CRC byte, a forward error correction byte aren't immune to corruption. The framing mechanism has to be able to detect when a packet has corrupt data.
There a several ways to approach all this.
I'm making the working assumption that packets are framed only with the serial bytes. Handshake lines aren't used for framing. Time delays aren't used for framing.
Send packet length
Send the length of the packet in the beginning, instead of [or in addition to] the terminating character at the end.
pros: Payload is sent in a efficient binary format.
cons: Need to know the packet length at the start of the transmission.
Escape the special characters
Escape the special characters when sending the payload data. This is already explained in a an earlier answer.
pros: Sender doesn't need to know the length of the packet at the beginning of the transmission.
cons: Slightly less efficient, depending on how many payload bytes need to be escaped.
Payload data encoded such that it can't contain start and stop characters
The payload of the packet is encoded such that it can't contain the start or stop characters. Usually, this is done by sending numbers as their ASCII or Hex-ASCII representation.
pros: Human-readable with common terminal programs. No need for code to handle escaping. No need to know the length of the packet at the start of the transmission
cons: Lower efficiency. For one byte of payload data, several bytes are sent.
answered Apr 19 at 21:07
Nick Alexeev♦Nick Alexeev
32.7k1066167
edited Apr 19 at 21:28
answered Apr 19 at 21:07
Nick Alexeev♦Nick Alexeev
32.7k1066167
answered Apr 19 at 21:07
Nick Alexeev♦Nick Alexeev
32.7k1066167
answered Apr 19 at 21:07
Nick Alexeev♦Nick Alexeev
32.7k1066167
32.7k1066167
add a comment |
add a comment |
Thanks for contributing an answer to Electrical Engineering Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2felectronics.stackexchange.com%2fquestions%2f433202%2fhow-fail-safe-is-n-r-as-stop-bytes%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
7
$begingroup$
To send arbitrary data you either have to go to using packets or byte stuffing. In your case the probability of the pattern appearing in a certain location is 1/65536. Which gets to 1 if you have a long enough random data stream.
$endgroup$
– Oldfart
Apr 18 at 10:36
4
$begingroup$
Can you provide context please. Stop bits are part of UART communication but stop bytes? This sounds like a pure software issue and depends what has been agreed by the sender and receiver.
$endgroup$
– Warren Hill
Apr 18 at 10:39
2
$begingroup$
@MariusGulbrandsen if your data is truly arbitrary and not strictly text (think ASCII) then null termination will not work; you will have to implement a packet.
$endgroup$
– RamblinRose
Apr 18 at 10:47
4
$begingroup$
BTW: That common practice is to put the carriage return before the line feed:
"x0Dx0A".$endgroup$
– Adrian McCarthy
Apr 18 at 21:16
3
$begingroup$
@AdrianMcCarthy I think the point of reversing it is to minimize the odds of it being a valid sequence. That said, two Windows line-endings in a row would give you
rnrnwhich contains thenrsequence in the middle...$endgroup$
– Mike Caron
Apr 19 at 16:05