Question:
As concluded in Singapore.
When receiving a packet with an unexpected key phase, an implementation should compute the new keys: This creates a timing side-channel, since an attacker can modify the KEYPHASE bit, thereby making a peer compute the next key generation. The packet will be discarded, since the header is protected by the AEAD, but the fact that the key is computed on receipt of that packet is a side-channel that might be used to recover the header protection key. The way to avoid this side-channel is to pre-compute the N+1 key when rolling over keys from key phase N-1 to N. An implementation would then use the N+1 keys when receiving a packet with an unexpected KEYPHASE, in exactly the same way it would do decryption on a packet sent at key phase N. This however contradicts the text that claims that an implementation may choose to only keep two read keys at any given moment: Directly after a key update, an implementation would have to store (at least) 3 key generations: N-1 for a duration of 3 PTO, in order to be able to decrypt delayed / reordered packets, N, as well as the precomputed N+1 to avoid the timing side-channel described above. I don't think that storing 3 keys is a problem (I think NAME disagrees, I'll let him explain his reasoning himself), but the spec shouldn't pretend that it's possible to create an implementation that both isn't vulnerable to the timing side-channel and at the same time gracefully handles moderate amounts of reordering around a key update.
I think you are correct in pointing out that there is a timing side-channel, and that we need to fix the issue. OTOH, I disagree that requiring all endpoints to be capable of retaining 3 keys is not a good way of fixing the issue due to the following reasons. Firstly, I do not think we can justify why an endpoint needs to be capable of retaining that much keys, even though Key Update is expected to happen very rarely. The purpose of Key Update is to replace a key after extensive use, we would never be using a key that becomes weak just after couple of PTOs. Considering that, using 2 keys should be sufficient; 3 keys is likely a sign of a needlessly complex design. Let's compare the proposal of the 3-key design with a 2-key design (that allows the receiver to delay the installation of the next key for 3 PTO after since Key Update, which is ). In the 3-key design, the next key is installed when a Key Update happens. The old key is disposed 3 PTO after a Key Update, and the slot becomes NULL. Note also that an endpoint needs to retain a mapping between packet number and key phase to determine whether an incoming packet is to be decrypted using the oldest key or the newest key (those two key share the same KEY_PHASE bit on the wire). In the 2-key design, installation of the next key and the disposal of the old key happens at the same moment (i.e. 3 PTO after a Key Update). An endpoint would always retain 2 keys once the handshake is confirmed. There is no need for retaining a mapping between packet numbers and key phases. The oddity of the key phase bit directly maps to the decryption key. To me it seems that the latter is simpler. As stated above, there is no need for doing Key Update every 3 PTO, and therefore, assuming that we need to make a change due to the attack vector, I think we should simply allow endpoints delay the installation of the next key rather than changing the increasing the minimum state that an endpoint needs to be capable of maintaining.
I agree, there's not really a need to do key updates that frequently. However, I like the fact that we're currently defining a clear rule: An endpoint is allowed to update to N+1 as soon as it has received an acknowledgement for a packet sent at key phase N. You should still keep track of the packet number at which the key update happened, in order to check that the peer is not reusing the old key to encrypt packets after it already switched to new keys. Strictly speaking, this is not required for a working implementation, but it's a really easy check to catch a misbehaving peer (or detect an active attack). As I've mentioned in the discussion in , the reason I'm skeptical about delaying key updates by 3 PTO (and effectively dropping all packets with a higher key phase until this period is over) is that we're replacing a very clear definition by an approximate rule: the PTO is calculated locally, and depending on the network conditions, the two endpoints might have different opinions about the exact value of the PTO. An endpoint that wants to ensure that no packets are dropped due to the unavailability of keys would have no choice but to wait >> 3 PTO. While this should be well within the safety limits of the cipher suites we're using, not having a clear criterion seems to be a sign of suboptimal protocol design.
Perhaps we should state that an endpoint should precompute the key early but not update keys too frequently to prevent timing attacks and overhead. That is not necessarily a bad thing, but it is unfortunate that there is no clear definition. We could also say the idle timeout period. As long as we compute keys using SHA256 it might not be a concern, but computing keys too frequently could exhaust safety parameters on other potentially faster algorithms.
To clarify, the key phase bit is inside header protection, so the attacker in question is the peer?
NAME No, this is an on-path attack. Header protection is not authenticated, so an attacker can just flip bits in the header. Decrypting the packet will fail then, since the header is part of the authenticated data used in the AEAD. We have a timing side-channel as soon as we treat packets differently based on values in the unprotected header, before authenticating the packet. This is analogous to the .
Thanks for the extra explanation, I'm not sure that's a very good side-channel, but I agree it is one.
Does anyone have any ideas what kind of actual attack could take advantage of this? The attacker can only cause you to install your keys early once per key phase change. Assuming you hardly every change key phase, what then? I agree it's a way the unauthenticated attacker can change your state, but I don't think it's a big enough problem we need to worry about fixing.
That's a good question. I agree that this attack vector doesn't look very promising at first sight, but then again I'm always surprised what cryptanalysts can extract from a single bit. That depends. If you create the new keys, try decrypt the packet, and then discard the keys (if decryption fails), the attacker has an unbounded number of tries. Obviously, this would also be a DoS vector, since computing new keys is more than order of magnitude more expensive than an AEAD operation (at least in my implementation). If you cache the new keys once they are computed, then the attacker indeed only has a single shot per key phase. However, then you might as precompute the new keys when installing the current key phase. The problem with that is that the spec currently suggests that you only ever need to keep state for 2 key generations at any given moment. This is incompatible with caching the keys for the next key phase, as I described above.
NAME But the current spec. also recommends to wait for 3 PTO before initiating the next Key Update. That would not change, and as we agree, nothing in practice requires us to do Key Updates every 3 PTO. I agree that an endpoint might still want to track the packet number at which key update happened. But even in such case, the simplicity of the 2-key design still exists, in sense that you'd only use it for detecting a suspicious peer. It's much straightforward and easier to test, because the rare case (of actually using 3 keys at the same time) never happens. To clarify, I am not saying that 3-key design is incorrect. My argument is that lack of clear signal is a non-issue (because there is no need to do a Key Update every 3 PTO or more frequently), and that therefore that argument cannot be used for prohibiting people to choose the 2-key design (which is simpler).
Isn't the middle ground here to pre-compute the N+1 key when the N-1 key is dropped, with the drop happening at the first of: 3xPTO after the N key was first successfully used First arrival of a packet which purports to be N+1 The attacker's ability at that point would be to cause any delayed N-1 packets to be dropped, but if we're presuming the ability to bit-flip, they can cause those packets to be invalid and dropped anyway.
NAME That certainly reduces the information being potentially exposed to the attacker to one bit per every key update. But no less than that for example when an attacker injects two packets slightly more frequent than every 3 PTO, with the only difference between the two packets being the value of the KEY_PHASE bit. In such an attack scenario, the attacker can tell which of the two packets caused the calculation of the updated key for every key update. Admittedly, the leak is tiny; but I am not sure if it is tiny enough that we can ignore.
To recap my comments at the mic: it's not clear to me what the attacker is learning here. Can someone expand on this?
Discussed in Montreal: NAME to work with NAME and NAME to work out the details
I'd like to second NAME comment at the mic. I don’t think this is an issue for which we need text. Without loss of generality, let’s assume that in the worst case an attacker learns the entire unprotected header bytes for every packet. (That is, that header protection does nothing overwhelmingly helpful.) Revisiting the header protection algorithm, it basically works as follows: where H’ and H are the protected and unprotected header bytes, respectively, and sample is known to the adversary. If we assume mask is the output of a PRP, and therefore both IND-CPA secure and KPA secure, an adversary learns nothing from known plaintext and ciphertext pairs. (The typical game-based IND-CPA security definition says that an adversary with access to an oracle for encrypting any message Mi of its choice has negligible probability in selecting b, upon giving the challenger M0 and M1 and receiving an encryption of Mb. Recovering the header protection key would break this assumption, as it would let the adversary encrypt M0 and M1 -- without the oracle -- and trivially match against M_b.) All that said, I think adding text describing that endpoints should precompute keys is fine. After all, it'd be nice if header protection worked as intended.
NAME and NAME to resolve their issue before we'll take it forward
Discussed in Singapore; proposal is to change SHOULD to MAY regarding pregeneration of next phase of keys.

perform this process as part of packet processing, but this creates a timing signal that can be used by an attacker to learn when key updates happen and thus the value of the Key Phase bit in certain <del> packets.  Endpoints SHOULD instead defer the creation of the next set of receive packet protection keys until some time after a key update </del> <ins> packets.  Endpoints MAY instead defer the creation of the next set of receive packet protection keys until some time after a key update </ins> completes, up to three times the PTO; see old-keys-recv. Once generated, the next set of packet protection keys SHOULD be


Question:
The Retry packet can be discarded.
The Retry packet currently allows sending an empty token. The Retry token is defined as an opaque blob where "indicates that x is variable-length", which also includes zero-length values. On the wire, an Initial packet with a zero-length token is identical to an Initial without a token. However, there's a difference on the client side, since a client will perform at most a single Retry.
Tagging this editorial. This was clearly intended to be considered non-zero length.

Clients MUST discard Retry packets that contain an Original Destination Connection ID field that does not match the Destination Connection ID from its Initial packet.  This prevents an off-path <del> attacker from injecting a Retry packet. </del> <ins> attacker from injecting a Retry packet.  A client MUST discard a Retry packet with a zero-length Retry Token field. </ins> The client responds to a Retry packet with an Initial packet that includes the provided Retry Token to continue connection


Question:
Addresses feedback from NAME and NAME to add a reference to the recovery doc on handling bursts resulting from flow control credit; that advice on when to send *_DATA frames (at least 2 RTTs before sender is blocked) is too specific. I've also done some editorializing to fix flow.
Is this the right place to discuss pacing? In my mind, we’re talking about different layers here. Bursty sending could happen due to many reasons, only one of them is due to increased flow control credit.
Yeah I was wondering that too. We do talk about application limited in the recovery doc, so maybe that's the place to clarify this. I'll see (after the weekend) if it makes sense to add anything there.
I think we should discuss what to do in one place, but given the sort of traffic bursts that can arise if you happen to get this wrong, I am still keen to see a couple of sentences here that explain the pathology when a sender has cwnd and no credit, and then suddenly get lots of credit. We see the enormous burst, and it is REALLY not nice if you don't have a pacer to fix it (or figure out the receiver should have sent the credit earlier).
NAME in the recovery draft covers the general case of an under-utilized cwnd, which applies here. The same bursting can happen with a bursty app as well, but we've said it all in one place -- in that section of the recovery draft. I think that section could use some editing work (which I'll take on separately), but we've already said there that it applies to flow control. I think that ought to cover the pacing concern.

4.2. <del> This document leaves when and how many bytes to advertise in a MAX_STREAM_DATA or MAX_DATA frame to implementations, but offers a few considerations.  These frames contribute to connection overhead. Therefore frequently sending frames with small changes is undesirable.  At the same time, larger increments to limits are necessary to avoid blocking if updates are less frequent, requiring larger resource commitments at the receiver.  Thus there is a trade- off between resource commitment and overhead when determining how large a limit is advertised. </del> <ins> Implementations decide when and how much credit to advertise in MAX_STREAM_DATA and MAX_DATA frames, but this section offers a few considerations. To avoid blocking a sender, a receiver can send a MAX_STREAM_DATA or MAX_DATA frame multiple times within a round trip or send it early enough to allow for recovery from loss of the frame. Control frames contribute to connection overhead.  Therefore, frequently sending MAX_STREAM_DATA and MAX_DATA frames with small changes is undesirable.  On the other hand, if updates are less frequent, larger increments to limits are necessary to avoid blocking a sender, requiring larger resource commitments at the receiver. There is a trade-off between resource commitment and overhead when determining how large a limit is advertised. </ins> A receiver can use an autotuning mechanism to tune the frequency and amount of advertised additional credit based on a round-trip time estimate and the rate at which the receiving application consumes <del> data, similar to common TCP implementations.  As an optimization, sending frames related to flow control only when there are other frames to send or when a peer is blocked ensures that flow control doesn't cause extra packets to be sent. If a sender runs out of flow control credit, it will be unable to send new data and is considered blocked.  It is generally considered best to not let the sender become blocked.  To avoid blocking a sender, and to reasonably account for the possibility of loss, a receiver should send a MAX_DATA or MAX_STREAM_DATA frame at least two round trips before it expects the sender to get blocked. A receiver MUST NOT wait for a STREAM_DATA_BLOCKED or DATA_BLOCKED frame before sending MAX_STREAM_DATA or MAX_DATA, since doing so will mean that a sender will be blocked for at least an entire round trip, and potentially for longer if the peer chooses to not send STREAM_DATA_BLOCKED or DATA_BLOCKED frames. </del> <ins> data, similar to common TCP implementations.  As an optimization, an endpoint could send frames related to flow control only when there are other frames to send or when a peer is blocked, ensuring that flow control does not cause extra packets to be sent. A blocked sender is not required to send STREAM_DATA_BLOCKED or DATA_BLOCKED frames.  Therefore, a receiver MUST NOT wait for a STREAM_DATA_BLOCKED or DATA_BLOCKED frame before sending a MAX_STREAM_DATA or MAX_DATA frame; doing so could result in the sender being blocked for the rest of the connection.  Even if the sender sends these frames, waiting for them will result in the sender being blocked for at least an entire round trip. When a sender receives credit after being blocked, it might be able to send a large amount of data in response, resulting in short-term congestion; see Section 6.9 in QUIC-RECOVERY for a discussion of how a sender can avoid this congestion. </ins> 4.3.


Question:
Fixes issue
Sect. 7 in transport refers to packer number spaces without any reference to what that might mean. These are only introduced later in sect. 12.3. Also the text talks about crypto sequence numbers and the text is not very clear about these being different from packet numbers if you haven't been through TLS text.
This is still an issue. I think adding references to where packet number spaces are mentioned (List of Common Terms and Section 7) would be sufficient.
Sounds great, would you be willing to write a PR NAME
Fixed in .

acknowledgment (see sending-acknowledgements). A packet that does not increase the largest received packet number <del> for its packet number space by exactly one.  A packet can arrive out of order if it is delayed or if earlier packets are lost or delayed. </del> <ins> for its packet number space (packet-numbers) by exactly one.  A packet can arrive out of order if it is delayed or if earlier packets are lost or delayed. </ins> An entity that can participate in a QUIC connection by generating, receiving, and processing QUIC packets.  There are only two types


Question:
Fixes issue
Sect. 7 in transport refers to packer number spaces without any reference to what that might mean. These are only introduced later in sect. 12.3. Also the text talks about crypto sequence numbers and the text is not very clear about these being different from packet numbers if you haven't been through TLS text.
This is still an issue. I think adding references to where packet number spaces are mentioned (List of Common Terms and Section 7) would be sufficient.
Sounds great, would you be willing to write a PR NAME
Fixed in .

An endpoint can verify support for Explicit Congestion Notification (ECN) in the first packets it sends, as described in ecn-validation. <del> The CRYPTO frame can be sent in different packet number spaces.  The sequence numbers used by CRYPTO frames to ensure ordered delivery of cryptographic handshake data start from zero in each packet number space. </del> <ins> The CRYPTO frame can be sent in different packet number spaces (packet-numbers).  The sequence numbers used by CRYPTO frames to ensure ordered delivery of cryptographic handshake data start from zero in each packet number space. </ins> Endpoints MUST explicitly negotiate an application protocol.  This avoids situations where there is a disagreement about the protocol


Question:
Fixes issue
Thanks for this. It's a lot clearer this way.
restricts frame type varint encoding: The relevant sentence is: This means that the restriction may not apply to frames defined by QUIC extensions (such as the frame in the Delayed Acks_ extension). Must each extension spell out its own frame type encoding rules? It would be easier to change the transport draft to extend the minimal encoding restriction to not-as-yet-specified frames as well.
I read the first sentence as applying to all QUIC frame types, core or extensions. The examples confuse the matter by explicitly mentioning the core frames. Section 19.21 contains some guidance for extension frames, adding a pointer back to the encoding rules from would be a good editorial change.
The sentence I highlighted talks about frames in this document and then these frames, effectively overriding what the first sentence may imply.
yes I think we agree :)
I think we all agree that it can be made more explicit. The first sentence says that the rule for all frame types is that they be minimally encoded. Applying that principle to all frame types defined in this document means that they get one-byte encodings. We should clarify that it's an application of the rule, not a limitation of it.
Why did we have this rule in the first place? It just seems to complicate things in unexpected ways.
NAME See . IIRC the rationale for using minimal varint encoding is that stacks that support only the standardized frames can live with a simpler dispatch scheme that takes one byte as an input (regardless of the dispatch method being a table or a switch clause or whatever). While I do not have a strong opinion regarding that requirement, I'd prefer not revisiting the discussion.
PR has been merged -- closing bug.

The Frame Type field uses a variable length integer encoding (see integer-encoding) with one exception.  To ensure simple and efficient implementations of frame parsing, a frame type MUST use the shortest <del> possible encoding.  Though a two-, four- or eight-byte encoding of the frame types defined in this document is possible, the Frame Type field for these frames is encoded on a single byte.  For instance, though 0x4001 is a legitimate two-byte encoding for a variable-length integer with a value of 1, PING frames are always encoded as a single byte with the value 0x01.  An endpoint MAY treat the receipt of a frame type that uses a longer encoding than necessary as a connection error of type PROTOCOL_VIOLATION. </del> <ins> possible encoding.  For frame types defined in this document, this means a single-byte encoding, even though it is possible to encode these values as a two-, four- or eight-byte variable length integer. For instance, though 0x4001 is a legitimate two-byte encoding for a variable-length integer with a value of 1, PING frames are always encoded as a single byte with the value 0x01.  This rule applies to all current and future QUIC frame types.  An endpoint MAY treat the receipt of a frame type that uses a longer encoding than necessary as a connection error of type PROTOCOL_VIOLATION. </ins> 13.


Question: These are all MUST in recovery and the editors discussed this and agreed that these should be MUST too.

server, can cause a situation in which the server cannot send when the client has no data to send and the anti-amplification limit is reached.  In order to avoid this causing a handshake deadlock, <del> clients SHOULD send a packet upon a probe timeout, as described in </del> <ins> clients MUST send a packet upon a probe timeout, as described in </ins> QUIC-RECOVERY.  If the client has no data to retransmit and does not <del> have Handshake keys, it SHOULD send an Initial packet in a UDP datagram of at least 1200 bytes.  If the client has Handshake keys, it SHOULD send a Handshake packet. </del> <ins> have Handshake keys, it MUST send an Initial packet in a UDP datagram of at least 1200 bytes.  If the client has Handshake keys, it SHOULD send a Handshake packet. </ins> A server might wish to validate the client address before starting the cryptographic handshake.  QUIC uses a token in the Initial packet


Question:
Clarifies why a client might send a Handshake packet, as well as improving the surrounding anti-deadlock text. Changes 2 SHOULDs to MUSTs to line up with recovery.
We don't provide examples of the weird cases where you need to send Handshake packets to prevent amplification deadlock. The strangest one is where the client has all Initial data (and therefore has Handshake keys), but never receives a Handshake packet from the server. In this case, the client needs to send a Handshake packet that won't contain ACK or CRYPTO frames. We should explain this in a note. See .
I think the logic in the recovery draft misses some of these cases entirely. The rule in draft 23 is: but in your example case there is not necessarily any unacknowledged data to transmit in a probe, and a is forbidden at Handshake level, leaving no reasonable options to compose an ACK-eliciting packet.
a few suggestions

the server is not overly constrained by the amplification restriction. <del> Packet loss, in particular loss of a Handshake packet from the server, can cause a situation in which the server cannot send when the client has no data to send and the anti-amplification limit is reached.  In order to avoid this causing a handshake deadlock, clients MUST send a packet upon a probe timeout, as described in QUIC-RECOVERY.  If the client has no data to retransmit and does not have Handshake keys, it MUST send an Initial packet in a UDP datagram of at least 1200 bytes.  If the client has Handshake keys, it SHOULD send a Handshake packet. </del> <ins> Loss of an Initial or Handshake packet from the server can cause a deadlock if the client does not send additional Initial or Handshake packets.  A deadlock could occur when the server reaches its anti- amplification limit and the client has received acknowledgements for all the data it has sent.  In this case, when the client has no reason to send additional packets, the server will be unable to send more data because it has not validated the client's address.  To prevent this deadlock, clients MUST send a packet on a probe timeout (PTO, see Section 5.3 of QUIC-RECOVERY).  Specifically, the client MUST send an Initial packet in a UDP datagram of at least 1200 bytes if it does not have Handshake keys, and otherwise send a Handshake packet. </ins> A server might wish to validate the client address before starting the cryptographic handshake.  QUIC uses a token in the Initial packet


Question:
Currently, the "Ignoring Loss of Undecryptable Packets" section indicates that some loss can be ignored, but now that we have multiple packet number spaces, I think we can constrain the set of ignorable losses to those below the smallest acknowledged packet in a packet number space. Ignoring the loss of a subsequently sent packet was not the intent.
I might have to put in a lint for HTAB...

6.6. During the handshake, some packet protection keys might not be <del> available when a packet arrives.  In particular, Handshake and 0-RTT packets cannot be processed until the Initial packets arrive, and 1-RTT packets cannot be processed until the handshake completes. Endpoints MAY ignore the loss of Handshake, 0-RTT, and 1-RTT packets that might arrive before the peer has packet protection keys to process those packets. </del> <ins> available when a packet arrives and the receiver can choose to drop the packet.  In particular, Handshake and 0-RTT packets cannot be processed until the Initial packets arrive and 1-RTT packets cannot be processed until the handshake completes.  Endpoints MAY ignore the loss of Handshake, 0-RTT, and 1-RTT packets that might have arrived before the peer had packet protection keys to process those packets. Endpoints MUST NOT ignore the loss of packets that were sent after the earliest acknowledged packet in a given packet number space. </ins> 6.7.


Question:
It might pay to reconcile: {{vars-of-interest}}) to be larger than the congestion window, unless the packet is sent on a PTO timer expiration; see {{pto}}. And now exceeds the recently reduced congestion window. Now, you might implement the latter using PTO machinery, but that text doesn't really suggest that in any way.
Where does this second sentence come from? I'm pretty sure I haven't implemented anything like this in quic-go. Should I?
That sentence is the the definition of the . We haven't implemented that either, and I was wondering if it truly helps.
For TCP this second case if important because as long as you don't retransmit the lost packet you will not get any ACK that acknowledges new data. I guess we don't need that for QUIC.
The second sentence was added later in , and the MUST wasn't updated, so I'll fix that. Chromium currently implements PRR(RFC6937), which is a more complex variant of this which is designed to keep the ACK clock going. I thought we discussed adding an informative reference to PRR when we added this text, and I'm not sure why we didn't.
seems to be the wrong issue you are pointed to...
Good catch, it's
I've suggested some symbolic references.

layer.  By providing reliability at the stream level and congestion control across the entire connection, it has the capability to improve the performance of HTTP compared to a TCP mapping.  QUIC also <del> incorporates TLS 1.3 at the transport layer, offering comparable security to running TLS over TCP, with the improved connection setup latency of TCP Fast Open RFC7413. </del> <ins> incorporates TLS 1.3 TLS13 at the transport layer, offering comparable security to running TLS over TCP, with the improved connection setup latency of TCP Fast Open TFO. </ins> This document defines a mapping of HTTP semantics over the QUIC transport protocol, drawing heavily on the design of HTTP/2.  While


Question:
Having non-overlapping conditions makes sense; adjusted wording.
The recovery draft : This is consistent with the pseudocode, in particular . When the server receives the client's final handshake flight, it discards handshake keys and hence cannot acknowledge it. When the client receives the resulting , the handshake is confirmed, but no Handshake or 1-RTT acknowledgements have necessarily been received. This results in a PTO being set even though the client's address has certainly been verified and there are no ack-eliciting packets in flight. As written, the pseudocode would in fact set a PTO for the Initial space at a garbage time. I think this could be corrected by appending "and has not discarded handshake keys" or "and has not confirmed the handshake" to the language, and adding the analogous clause to the conditions checked in the final statement of .
I thought that was implied, but implications are bad, so I agree it's best to explicitly state it. I'm happy to review a PR if you have time to write one, or I'll fix this fairly soon ideally.
When a reverse proxy forwards a HTTP response to the client, sometimes it sees the upstream server terminate the response early. For example, a HTTP/1.1 connection being closed prior to transmitting number of bytes specified by the field. Or a chunked response being closed abruptly. The only way of delegating such a condition to the client in HTTP/2 was to send a RSTSTREAM frame with . However, use of the error code is awkward in this case, considering the fact that the error did not happen within the communication between the endpoints using the particular HTTP/2 connection. I think it makes sense to have an error code that better designates the case. One way to address the issue in H3 would be to change to , and let the server use the error code (as an argument of the RESET_STREAM frame) to indicate that the response has been "cancelled". Originally proposed in URL
What if you do not want to admit to being a proxy? (who does?)
HTTPMESSAGECANCELLED would simply mean that the transfer of the response is cancelled, regardless of the server being an origin or a reverse proxy.
I can get behind this. As long as we're not looking for a byte-by-byte reproduction of the bytes the intermediary received (as was once requested and rejected, I think at the Seattle interim), then it makes sense to signal that the message was killed.
This is basically an expansion of the CANCELLED/REJECTED codes to allow a server to do either. WFM.
NAME Yes. And thank you for the PR.

unblock the server until it is certain that the server has finished its address validation (see Section 8 of QUIC-TRANSPORT).  That is, the client MUST set the probe timer if the client has not received an <del> acknowledgement for one of its Handshake or 1-RTT packets. </del> <ins> acknowledgement for one of its Handshake or 1-RTT packets, and has not received a HANDSHAKE_DONE frame. </ins> Prior to handshake completion, when few to none RTT samples have been generated, it is possible that the probe timer expiration is due to


Question:
Adds editorial text about why a receiver could decide to send ACKs less frequently than every two packets. This keeps coming up(ie: ), so I thought text was necessary.
As currently specified, the default QUIC ACK policy models TCP’s ACK policy, but consumes significantly more return path capacity. This can impact resource usage of the return channel, can constrain forward throughput, and can impact the performance of other sessions sharing an uplink (draft-fairhurst-quic-ack-scaling). This was not discussed in Zurich - where discussion moved instead to the separate topic of draft-iyengar-quic-delayed-ack. In many current network segments (e.g., cable networks, mobile cellular, WiFi and other radio tech), the effects of asymetry are presently mitigated by network devices RFC3449 that rely upon observing TCP headers. This results in ossification problems, but does practically have beneit. It reduces the TCP ACK rate over the constrained link, it reduces consumption of "radio" transmission opportunities, and reduces traffic volume from ACKs. These mitigations are not possible, nor desirable for QUIC. However, the default performance of QUIC is therefore much worse than for TCP. In QUIC, the default ACK Ratio has to set by the endpoints implementing the protocol because the sender and receiver have no information about the "cost" of sending an ACK using a link along an asymmetric path. These endpoints can also choose a non-default value to enable a new CC, scale to higher rates, etc (e.g. draft-iyengar-quic-delayed-ack). We have shown there can be significant benefit from an ACK Ratio of 1:10 and this is expected to alos have benefit for higher levels of asymmetry. The proposed change from 1:2 to 1:10 is predicated on QUIC's design decisions: starting a connection with an Initial Window of 10 is acceptable, QUIC uses PTO as the loss detection method (a predicate to issue ), and QUIC has some form of pacing at the sender, at least to mitigate IW bursts.
Even though we've avoided straying this far from the TCP spec, I think allowing this would be good for the internet. It should probably be documented for TCP, too. How does ack stretching of this nature work for BBR bandwidth estimation? It's harder to use the ack pacing, no?
I don't believe QUIC ACKs are significantly larger. In fact, I believe the ACK frame itself is typically smaller than TCP ACKs, but the savings there are used on QUIC's 16-byte auth hash and likely a non-zero-length CID. Do you have a size analysis written up somewhere? I think this is the core issue. It's not that QUIC ACKs are really consuming significantly more capacity in terms of bytes on the wire, it's that middleboxes can't decide to thin the ACKs, because they don't know which packets are ACK-only packets. Did you implement the optimization already described in the transport draft?: "As an optimization, a receiver MAY process multiple packets before sending any ACK frames in response. In this case the receiver can determine whether an immediate or delayed acknowledgement should be generated after processing incoming packets." NAME did and I believe found it hugely reduced the number of ACKs, sometimes too much. As I've said before, I would really prefer not specify an approach in QUIC transport that is a heuristic like does. In particular, on lower bandwidth networks, 10 packets may be the entire congestion window and is too large a value. On higher bandwidth networks, every 20 packets has shown to be slightly better than 10, at least when using BBR. I wrote to add some editorial text about why every 2 packets is a SHOULD, not a MUST. Chrome did not default enable the ACK strategy recommends until BBR was launched, because there was a performance regression with Cubic. I'd rather not assume everyone is using BBR or a BBR-like congestion controller. NAME BBR does fine with stretch ACKs. Cubic and Reno suffer in some cases, however.
QUIC ACKs are larger! The smallest IPv4 QUIC ACK packet is 51 bytes. The ACK frame is 4 bytes, but is sent using a QUIC packet. For IPv4 this is: 20 (IP) + 8 (UDP) +3 (1+1+1 QUIC header ... assuming a 1 B CID) + 4 (minimum ACK frame size)+ 16 (crypto). Other types of frames may also be sent in the same packet, increasing this size. That size difference matters. The point is that QUIC ACKs are consuming more capacity in terms of bytes on the wire, and often also important: in more transmission opportunities at the physical layer. That makes QUIC perform more poorly than it needs to across some paths. The text "As an optimization, a receiver MAY process multiple packets before sending any ACK frames in response. In this case the receiver can determine whether an immediate or delayed acknowledgement should be generated after processing incoming packets." Is permissive to do something different - alas the effect of ordering of the processing depends on rate, timing, etc - The present text is not a recommendation to do something appropriate, which I think it needs to. If this were to describe that a receiver SHOULD reduce the ACK to data packet ratio to 1:10, then that would help. On low bandwidth links, then I agree that waiting for 10 packets would not be suitable, but then wouldn't the rule 4 ACKs/RTT still cover this case? (I have no problems believing every an ACK 20 packets can be slightly better than 10 in some cases, but then we have chosen IW at 10 and that relates to bustiness and responsiveness of the basic spec, so I think 1:10 makes more sense as the recommendation. If the CC and path information suggest 1:20 - perhaps as the rate increases, then a sender can "tune" as per the transport parameter proposal. That I STILL assert that this is a different issue from the expected default.)
Would it help to give a recommendation like, if the congestion window is large, the ACK radio should be reduced? Not sure if we can easily give a threshold for the window but to align with the 4 ACKs per RTT rule we could say 40 or to be more conservation maybe 100?
If the receiver knew the cwnd... we could add that. The current proposal we made is here (along with some of the arguments about why QUIC and TCP are slightly different): draft-fairhurst-quic-ack-scaling
So the min TCP ack is 40 bytes vs QUIC at 50(0 byte CIDs are common, so I'll assume that), 25% larger in the minimum case. However, with a few SACK blocks, TCP ACKs are likely to be larger than QUIC ACKs, since SACK consumes 10 bytes for the first block and 8 for each additional block. By the time the second block is added, there's a good chance QUIC ACKs are smaller and it's almost certainly true with 3 SACK blocks(which I think is a typical number). So I'll go back to my previous statement that the real issue is that the network can't thin QUIC ACKs effectively.
May be "yes", may be "no" on a larger size after loss - since QUIC ACK ranges are often larger - but that depends on loss pattern, etc. In our case, we saw more bytes. But "yes" on me saying there is a need for QUIC to make good default decisions about the ACK rate, because QUIC becomes responsible for transport working across heterogeneous paths and we need to get good defaults rather than rely on "ACK-Thining" and other network modification.
Anybody has a gut feeling how very low latency congestion control fares with sparse ACKs?. I have experienced that DCTCP is more picky with ACK aggregation than e.g Reno. I suspect that ACK-thinning can have the give the issue as well. A too sparse ACK spacing can also give problems. Another aspect is how e.g. TCP Prague with paced chirps work with sparse ACKs ?. To me the RTT/4 rule makes sense but I feel that the best alternative depends a lot on both the congestion control and the source material. Large file transfers are more trivial than sources that are occasionally idle.
I think this isn't the correct question. This is about the default policy. The default policy is about what a receiver does by "default" - the current text uses a value of ACK every other packet, 1:2. TCP specified this when transmission rates were low, and TCP depended on individual ACKs to clock data (no pacing, IW=2), detect loss (no PTO), increase cwnd (per ACK packet), etc. In response, TCP causes ACK congestion and strange interactions with radio access. Performance over WiFi, DOCSIS etc suffers unless the network tries to address this. I don't see why QUIC, which doesn't rely on any of these, needs to set a DEFAULT of 1:2. This is literally a few lines of code difference, and we could use 1:10. These CC's you mention may have different needs and that needs to be handled by telling the remote that your CC has these specific needs - which I think is what Jana's extension is about.
I am kind of the donkey in the middle... ACK thinning in networks should be discouraged. Standardization of ACK thinning mechanisms was proposed in 3GPP RAN2, this request was turned down mainly with reference to that it can harm transport protocol performance and does not work safely with QUIC. Nothing stops UE (terminal and modem) vendors from implement proprietary ACK-thinning but I certainly hope that 5G UE vendors don't attempt to thin QUIC-ACKs as this is a slippery slope. High ACK rates is a 5G problem mainly for high band NR (=5G at e.g 28GHz). It is mainly a coverage issue as UEs have a limited output power due to practical and regulatory requirements. At the same time one need to address that end users don't only download movies today and even VoD streaming implies uplink traffic in the form of chunk requests. And then we have an increasing amount of people doing uplink streaming or upload an infinite amount of pictures of food plates and cats :-) Operators and 5G vendors have to deal with this and solve the problem with deployment of 5G access in various frequency bands to get the best downlink and uplink performance. Reduced ACK rates is of course welcome but I don't currently see a problem that the initial ACK ratio is 1:2 , to increase to 1:? as the congestion window grows, bounded by e.g an RTT/4 rule. But here I need to admit that I at least I have not studied this in detail. Ergo.. I am not convinced that it is necessary to hardwire the QUIC spec to 1:10 , but I also cannot say that it should not be done.
This is a small enough change. However, there is still a bar for those, and the current level of discussion makes me believe we should close this with no action?
Just so that it's easy to find, there is data and a continued discussion at URL
Folks who care about a change here need to prioritize progressing this issue. We're close to being able to last-call the specs, and would like this resolved in time.
We’ve completed a round of updated analysis of the proposed change to the QUIC spec, using quicly (changing to a 1:10 policy), and chromium (which currently does 1:10 by default). Our hope is that this change to the spec, to actually make standard end-to-end QUIC work better than TCP over asymmetric paths! This is a short slide deck about the performance of the proposed change: URL There have been a number of other change made by the QUIC ID editors relating to ACK processing, so we wanted to be sure these results were up to date. (These results do not consider additional ACKs after re-ordering - reflecting the latest text removing that.) We also really hope this will motivate a change the practice of forcing of QUIC fallback to TCP (with PEP/Thining) over highly asymmetric paths. People may also be interested in work for L4S to evaluate the impact of ACK Thining for TCP-Prague. Please do provide feedback, or any questions on this - as Lars notes, we think the time is running out for considering new changes, even relatively simple ones like this. Gorry P.S. First email: URL
I believe that it is beneficial to use a 1:10 policy, I understand that this means that a 1:2 policy that still be used in the initial phase (flow start) or does the timer mechanisms kick in to avoid that ACKs are transmitted too sparsely in this case ?.
Yes, our proposal counted ACK-eliciting packets at the receiver (because the receiver does not in general know the congestion state, so you need a heuristic), and we finally chose the same method in currect QUIC Chromium - i.e. for the first 100 packets we do not use ACK 1:10, changing only after 100 packets.
OK, it sounds like a good idea to use 1:10 after 100 packets or so. Just one potentially stupid question, Is there any special treatment necessary after an RTO ?
[This seems to be a forwarded comment from NAME Hi, Gorry Thank you for sharing your experiments. I'm delighted to see results showing that use of ack10 gives us comparable throughput to ack2. Recently, I have been looking into how ack-frequency affects the computational efficiency of the server [1]. Using the (almost) same setup, I have today run a benchmark comparing three ACK strategies: ack2, ack10, ack 1/8cwnd. ack2: 334Mbps ack10: 414Mbps ack 1/8cwnd: 487Mbps As can be seen, ack10 is better than ack2, but lags behind ack1/8cwnd. That's because there are still frequent ACKs. They take considerable amount of CPU cycles. If we want to see QUIC being as lightweight as TLS over TCP, it is critically important to reduce ACK frequency as much as possible. This is because the cost of processing ACK in QUIC is much more expensive than TCP, due QUIC being a userspace transport. Based on this, if we are to change what we have in the base drafts, I'd much prefer adopting what we have in the proposed delayed ack extension [2], rather than just changing the default ack policy from ack2 to ack10. By adopting what we have in the delayed ack extension, we can expect better performance, and the endpoints would have the freedom to change the behavior when the default ack policy is deemed insufficient (regardless of the default being ack2 or ack10). [1] URL [2] URL 2020428() 20:37 Gorry Fairhurst : issue. We're close to being able to last-call the specs, and would like this resolved in time. QUIC spec, using quicly (changing to a 1:10 policy), and chromium (which currently does 1:10 by default). Our hope is that this change to the spec, to actually make standard end-to-end QUIC work better than TCP over asymmetric paths! relating to ACK processing, so we wanted to be sure these results were up to date. (These results do not consider additional ACKs after re-ordering - reflecting the latest text removing that.) of QUIC fallback to TCP (with PEP/Thining) over highly asymmetric paths. People may also be interested in work for L4S to evaluate the impact of ACK Thining for TCP-Prague. think the time is running out for considering new changes, even relatively simple ones like this. URL
These results/article on computational efficiency are really interesting. Please see below. This is useful, we see here there's a clear benefit here in using a larger ACK Ratio also in terms of server load. We can see already the benefit at 1:10 and GSO helps as expected. We think that this is also really true. QUIC ACKs are also larger than TCP ACKs (see URL). I don't think the two approaches conflict. The difference between the two approaches is that the small change to the base specification to use an ACK Ratio 1:10 ensures the default return path is at least as good as TCP for common traffic (even with common TCP ACK Thinning). Without this change, the end user could experience significant differences in performance between TCP and between different QUIC versions over asymmetric paths. Whereas to me, the transport parameter extension looks like an exciting way to enable many more things, but there are also many ways this could usefully evolve based on CC, offload, pacing etec and I think this will take much more to work-out what is needed, safe and best ... then to reach consensus. We still also need to specify a default ACK Ratio in this case as well. Best wishes, Gorry
I'm greatly in favour of draft-iyengar-quic-delayed-ack-00 (BTW, where would you prefer me to give review comments about that?) I am much less keen on any default delayed ACK policy, whether 1:2 as now in quic-transport, or the 2-10/100 numbers proposed in draft-fairhurst-quic-ack-scaling-02 My concerns would melt away if the default policy was explicitly /implementation-dependent/ and the draft instead gave /guidance/ to implementers. The draft could still give the same 2-10/100 policy, but as /guidance/ for the current public Internet. The draft should also state the conditions under which this guidance was determined (e.g. Internet paths with BDP between x and y, or whatever). What's the difference? Stating a default policy implies this is a matter of interoperability. It's not. If we change to giving guidance, I'm sure most implementers will just use the guidance. However, implementers will be free to evolve their default as they gain experience. This reduces the possibility that other behaviours will evolve around any default like 2-10/100. For instance, network monitoring using it as a signature. Or congestion controls optimizing around it. In turn, this will reduce the chance of ossification around the first numbers we thought of. Senders can always use the ACK frequency frame to ask a receiver to change from its default. Then receivers will gradually learn what senders prefer them to do, and change their default accordingly. IMO, a continually improving default is important for short flows, so senders can avoid requesting a different ACK frequency right from the start.
Need to admit that I have not had too much time to follow the QUIC work lately. I too believe that draft-iyengar-quic-delayed-ack-00 is a good long term solution that allows endpoint to set a desired ACK frequency that is not more than necessary. As experiments show there are good incentives to keep the ACK frequency as low as possible in terms of client/server load and asymmetric links limitations. An ACK-FREQUENCY frame will allow to find the proper balance between the requirements given by the congestion control, the traffic type and the above given limitations. I am not too concerned if it takes an extra development/standards cycle to have that in place in the standard and in running code as long as the QUIC community shows other SDOs that the challenge with QUIC ACKs is being addressed.
NAME NAME could you please trim the quoted text when you reply by email?
(I trimmed the comments, but forwarding inline replies to email messages doesn't work well.)
Given that this is an area where there is still obviously experimentation and refinement, NAME comment seems on point; baking a specific algorithm as requirements into the specifications doesn't seem like a good idea (although putting constraints on endpoints might make sense). Can we focus on general guidance rather than documenting a specific algorithm? What would that PR look like?
+1 to NAME suggestion. If we want to make changes for v1, they need to happen now, and they must be in a form that can let us get consensus.
To be clear, when I said that having a default policy is not a matter of interoperability, that's only strictly true if QUIC supports something like draft-iyengar-quic-delayed-ack for the sender to ask the receiver to use a certain policy. Bob
I've proposed more context and text around our current guidance in the draft, see .
I believe that the text around asymmetric links look fine, not really necessary to write up much more. What can be good to know that for the 5G case with limited UL (uplink = modem->base station transmission) , the pain that a flow with an unnecessary high ACK rate causes is on the flow itself or other flows sharing the same access.
There are some different points of pain here, the default and the need to adapt to the endpoint: An ACK Ratio (AR 1:1): is the finest granuality of feedback. A QUIC AR of 1:1 would consume return path capacity proportional to about 6% of the forward traffic (assuming no loss, etc). This is more than TCP (about 3%). An ACK Ratio (AR 1:2): For a return path bottleneck, QUIC with an AR of 1:2 will still have more impact on sharing flows than TCP would with the same AR 1:2 (because QUIC ACKs are larger, and because networks do not currently thin QUIC ACKs). So QUIC needs to choose the default - is it 1:2, 1:4, 1:8, 1:10? A QUIC AR 1:10 results in an ACK traffic volume of about 1% of the forward traffic. To me, that's a small proportion, which is why I argued that 1:10 is a good recommendation. The default is important when: the endpoints have no knowledge of the asymetry or the impact they have on a link's layer 2 loading, or the endpoints need to reduce the collateral damage to other IP flows sharing the return path capacity. Negotiation: The AR can be driven by the application or endpoint - e.g., a higher rate connection may well have a benefit from an AR of 1:10, 1:20, 1:100, etc - but such ARs are seldom chosen to reduce the impact on sharing a return bottleneck, more likely they are chosen to benefit the performance of an application or endpoint. That consideration is different, and likely depends on the application or ednpoint - which to me is where the flexibility offered by a transport parameter is so important. draft-iyengar-quic-delayed-ack will still need guidance on the "default". I argued that the default policy impacts the congestion behaviour when there is appreciable asymmetry this causes at layer 2 or the collateral damage (congestion impact on sharing flows). Sure, the choice of 1% is just a number I said was reasonable, but I've seldom encountered L2 technologies where less than 1% of forward traffic would be significantly better.
NAME : I understand your position. While I agree that ack thinning happens with TCP, it is not what I expect to find on a common network path. And as far as I know, the endpoints still implement the RFC 5681 recommendation of acking every other packet. What you are arguing is that with ack thinners in the network, TCP's behavior on those network paths is different. However, I do not believe we understand the performance effects of these middleboxes, that is, how they might reduce connection throughput when the asymmetry is not a problem. Importantly, I don't think we should be specifying as default what some middleboxes might be doing without fully understanding the consequences of doing this. Please bear in mind that Chrome's ack policy of 1:10 works with BBR, not with Cubic or Reno. I do not believe there is a magic number at the moment, and as NAME noted, even 1:10 is not a small enough ratio under some conditions. Given this, and given that we know 1:2 is used by TCP endpoints, it makes the most sense to specify that as the suggested guidance. The new text proposed in describes the tradeoff involved, and the rationale for using 1:2. I am strongly against doing anything substantially different than what we have in the draft now without overwhelming information supporting a different position.
Agree with NAME 1:2 seems a reasonable default, and there seems to be considerable interest in draft-iyengar-quic-delayed-ack, which has a better proposal for stacks that care.
I think the new ACK text is improving the specification, but I'd like to be sure we have thought about this decision. Let me try to carefully respond on-list here, to see where we agree/disagree: I disagree, I think we shouldn't be perpetuating a myth that a sender receives a TCP ACK for every other packet. Even in the 90's implementations TCP stacks often generated stretch-ACKs for 3 segments. Since then, limited byte counting was introduced and significantly improved the sender's response to stretch ACKs, and that has been good for everyone. Senders still rely on byte-counting in TCP to handle Stretch-ACKs, and this need is not decreasing: new network cards reduce per-packet receive processing using Large Receive Offload (LRO) or Generic Receiver Offload (GRO). I am saying many TCP senders actually do today see stretch-ACKs. There are papers that have shown significant presence of stretch ACKs, e.g., (H. Ding and M. Rabinovich. TCP stretch acknowledgements and timestamps: findings and implications for passive RTT measurement. ACM SIGCOMM CCR, 45(3):20–27, 2015.). A TCP Stretch-ACK of 3 or 4 was common in their datasets (about 10% of cases). In some networks, the proportion of Stretch-ACKs will be much much higher, since ACK Thining is now widely deployed in WiFi drivers as well as cable, satellite and other access technologies - often reducing ACK size/rate by a factor of two. I am intrigued and would like to know more, in case I missed something? Typical algorithms I am aware of track a queue at the "bottleneck" and then make a decision based on upon the contents of the queue. If there is no asymmtery, there isn't normally a queue and the algorithm won't hunt to discard an "ACK". But, maybe, there is some truth in being wary: Encrypted flows can also build return path queues (as does QUIC traffic). Without an ability to interpret the transport data (which nobody, including me, wants for QUIC), a bottleneck router is forced to either use another rule to drop packets from the queue (such as the smallest packets) or to allow a queue to grow, limiting forward direction throughput. I would say neither of these outcomes are great because the router is trying to fix the transport's asymetry. However, if QUIC causes this problem, I suggest some method will proliferate as QUIC traffic increases. I didn't find something in QUIC spec. that was a major concern. It looked to me like QUIC had learned from TCP how to handle stretch-ACKs. After that, we changed quicly to use 1:10 with Reno, and things worked well. Didn't NAME also use Reno? Sure, the deisgn of TCP recognised that AR 1:1 generated too much traffic, AR 1:2 for TCP resulted in an Forward:Return traffic ratio of ~1.5%, and QUIC increases this ~3%. However, as network link speeds increased, this proved too low for many network links and so, TCP ACK thinning came into being. This often reduces the TCP AR to around 1:4 ( and will do this without bias, whatever the outcome is.
NAME Just to clarify, in our experiment, we reduced the ACK frequency from the default (2) to after leaving slow start, then updated it every 4 PTO. I think that would work fine on a high-bandwidth, stable network. But I am not sure if it would work as well as ack2 on all network conditions. Anyways, I'd explain our experiment as an optimization of ack 2, as it was reduction of ack rate after leaving slow start. I think that that was a sensible choice, because IIUC having frequent acks during slow start is essential. Assuming that we'd prefer using ack 2 during slow start, the question is if we need to hard-code in the base draft how to optimize after leaving slow start. While I agree that continuing to use ack 2 after slow start is concerning for certain networks, I am reluctant to spending time on looking for such a fixed set of values across diverse networks (including much slower links or lossy ones). Rather, my preference goes to shipping the base-draft as-is, then moving on to finalize the ack-frequency frame. I think we can assume most if not all of the popular clients to support the ack-frequency frame. Then, it becomes less of a question what the default in the base-draft is.
Thanks kazuho, 1/8 or 1/4 cwnd seems reasonable, as does using Ack 2 for the first part of slow start ... that closed to what we proposed. And yes there will anyway be cases where any default does not work. If QUIC decides to retain Ack 2, can we at least capture that Ack 2 with QUIC doesn't have the same return path performance as Ack 2 with TCP - so that people know that less frequent Acts are something to think about from the start? I'm doing a talk in PANRG this week, and have decided to talk-through this topic (I was already going to talk about things related to return paths). That'll give people a view of some data - and a little thought about the implications this has for QUIC, and whether there is an incentive for network device to thin QUIC to restore performance.
NAME : I'd be in support of saying that ack thinning happens for TCP and cite those studies, and that in the absence of that, there might be throughput consequences for QUIC. I think we've said exactly that in other words in , but if we want to say more there, that's fine with me. That would make this an editorial issue, but I'm entirely happy to do more words there. I am against changing the recommendation here for another arbitrary recommendation. I think the right approach is to allow endpoints to specify this during a connection, which is where the extension is helpful. As NAME notes, our hope is that the extension gets widely deployed. If you want to make the argument for the extension to become part of the base draft, that's a different conversation, but that's not the one we're having here.
Attempting to summarise the current status of this issue as a chair: there appears to have been a fair amount of discussion about making a design change, with representatives both in favour and against. It appears there is no omnipotent default policy, choosing an optimal one depends on many factors. The editors have landed an editorial change that attempts to better describe the rationale behind the QUICS's current design and highlights some of the factors. There may be further editorial improvements but they no not need to block the design issue. The proposal therefore is to close this issue with no design change. We had prepared some slides for the Vancouver IETD to discuss QUIC's ACKs, and how to make QUIC work better than TCP in common asymmetric paths. This is a short slide deck that summarises what we have learned about ACKs in QUIC: URL It compares the ACK traffic of TCP with that of QUIC from the point of view of a path where QUIC's return traffic is somehow constrained or could impact other user traffic. We think that ACKs are a concern for various shared capacity return link (e.g., radio-based methods in WiFi or mobile cellular were ACKs consume spectrum/radio resource or battery; shared capacity satellite broadband return link or the uplink of a cable system). For TCP, such systems commonly involve a PEP or an ACK Filtering mechanism (which clearly isn't wanted or even possible for QUIC). The baseline QUIC performance (as per current spec) suffers over these paths. So our first question is: Do other people think it is important that the default specified for QUIC works well with asymmetric paths ? Gorry

An ACK frame SHOULD be generated for at least every second ack- eliciting packet.  This recommendation is in keeping with standard <del> practice for TCP RFC5681. </del> <ins> practice for TCP RFC5681.  A receiver could decide to send an ACK frame less frequently if it has information about how frequently the sender's congestion controller needs feedback, or if the receiver is CPU or bandwidth constrained. </ins> In order to assist loss detection at the sender, an endpoint SHOULD send an ACK frame immediately on receiving an ack-eliciting packet


Question:
If you drop the first paragraph, this is just an editorial fix as far as I'm concerned. That is, not sending a frame that was previously sent is not a change. And - assuming that you dump it - "immediately" has no defined meaning anyway.
NAME I'm happy to drop the first paragraph, and I'm glad to hear that it's editorial. Two people on our team took the existing text to mean that we had to send RCID after receiving NCID even though we'd previously retired that sequence number. Update coming soon.
NAME Feel free to change to "editorial"
This is a small point, but it's pointlessly creating pain in our implementation and possibly some others. In Sec 19.15 (NEWCONNECTIONID frame): IMO this is poorly formulated. If the receiver has already sent an RCID frame for the sequence number and that RCID has been acked, it may have thrown away state for that sequence number and shouldn't have to resend a frame the peer has received. In particular, a buggy or malicious peer might send a very old sequence number, or the NCID receiver might have preemptively retired a sequence number it hadn't seen when it got the retire-prior-to field. Simply appending 'if has not already done so' to the text above would do the trick I think. I'm happy to file a PR unless most people think this is a terrible idea.
As a related point, it would be useful to say that an NCID receiver SHOULD NOT immediately retire old CIDs solely as the result of the NCID frame, except when instructed by Retire-Prior-to. This can lead to infinite loops if the peer tries to keep a specific number of CIDs available.
I think this would be a nice clarification. I think we already prohibit such behavior through the use of activeconnectionidlimit TP. Section 5.1.1 states: endpoints store received connection IDs for future use and advertise the number of connection IDs they are willing to store with the activeconnectionidlimit transport parameter. To paraphrase, by using the TP, an endpoint willing to retain N active connection IDs prohibits the peer from sending more than N.
I was probably unclear. For a while, our implementation was doing something dumb: each time it got a new CID, it started using it and retired all previous ones regardless of Retire Prior To. This created infinite loops where the client was trying to make sure that we always had two CIDs. We should probably warn against that. Please see if the text in the PR is clear enough.
Taking a look at the PR, I'm finding the proposed text to be a bit unclear, too. Based on the discussion on the list, this seems to be a misreading of the text itself. From the list: I don't believe that this is actually what the text says. The text says: This to me reads as "If you said everything below CID 10 is done, and then you send me CID 8, I must immediately send a RCID frame for 8 since I can't use it". I don't believe that there is any CID-specific state required at all for this. You do need to store the current highest value that you've ever seen for RPT, which is a single integer, I'll call it . Beyond that, when you get a NCID frame, you check the sequence number. If it's less than , you should immediately enqueue (which I think might be a reasonable wording change to make, since immediately seems to have caused concern when the intent was that you put it on the tracks leaving the station, not that you force the train out the door before it's allowed) a RCID frame for that sequence number. Note that a RCID frame contains a single value, the sequence number of the connection ID being retired, which incidentally is contained by the NCID frame you're responding to. We can argue as to whether or not you should bother to send RCID again when the same frame has been acked, but (a) storing that is actually more state and (b) we're working pretty hard to avoid tracking ACKs of packets. I might be missing something here -- can you help me understand the concern about storing state in order to meet the requirements of this text?
Yes, I concede that the original email was garbled. I read the original text the same way you do. The issue is that if I decide to retire immediately without actually having the sequence number, that should be sufficient. Making me send it again after the NCID is gratuitous, and in at least one implementation creates state solely to track this corner case.
Ah cool, that's excellent then! :) What's the state that gets created to track that case? If you retired them all upon getting RPT without checking to see if you had them yet, don't you still have to keep track of the RPT value or which ones you've already retired? I'm probably missing some other piece of state in this case...
I think what an endpoint is required to track are: active connection IDs sequence number of connection IDs that are being retired (but are yet to be acknowledged) When receiving a NCID frame with RPT, an endpoint checks if there are any active CIDs below the RPT, and retires them. I do not think an endpoint has to track RPT, though admittedly, I haven't implement them (yet).
NAME what you suggest would certainly work. However, a malicious peer could send an infinite number of NCIDs as long as it kept advancing rpt and didn't ack RCIDs, spiraling the state upwards. To avoid this, we keep a window of unacked but retired sequence numbers and construct RCID from that. That's why having an NCID come in well below the window results in additional state for us.
NAME Exactly. We have an issue for that problem: . Agreeing on what to do to resolve the unbounded state problem might help resolving this issue too.
Taking this as editorial based on the proposal (and promised tweaks).
Looks fine to me, though I do think it's better with some kind of language to send it sooner rather than later.

An endpoint that receives a NEW_CONNECTION_ID frame with a sequence number smaller than the Retire Prior To field of a previously <del> received NEW_CONNECTION_ID frame MUST immediately send a corresponding RETIRE_CONNECTION_ID frame that retires the newly received connection ID. </del> <ins> received NEW_CONNECTION_ID frame MUST send a corresponding RETIRE_CONNECTION_ID frame that retires the newly received connection ID, unless it has already done so for that sequence number. </ins> 19.16.


Question:
This is option 4 from the discussion in . That is, making preferredaddress equivalent to NEWCONNECTION_ID with a sequence number of 1 in every way that matters.
Offshoot from review. Servers behind a load balancer, and the load balancer isn't CID-aware. Each server gives out an IP:port per client in the parameter. If the client moves to the SPA, then future migration is fine -- it will continue to talk to the same server using the preferred address. If the client doesn't move, those future migrations will break. There's currently no way to set to "only if you don't use my preferred address". Similarly, if the primary connection is to an anycast address, if the client fails to move to the preferred address, subsequent migration will likely break. Should we blanket recommend that clients not actively migrate if they ignore / are unsuccessful in using the SPA? Or should there be a way for the server to indicate that?
Good question. This brings up a related question for me about disableactivemigration. Given a server can decide not to give out any additional connection IDs, doesn't that give it a mechanism for disabling active migration? Ironically, given recent decisions on connection CIDs and SPA, if an SPA is supplied, then the client always has a second connection ID it can use on any path, not just the SPA. So my suggestion of not giving out extra CIDs fails for the original path, but not the SPA.
What we’ve said for this so far is that SPA is limited to offering a minimum of a single additional CID that needs to work in both places, the original server and the preferred server. As Ian says, the original server should not issue the client any more CIDs (if you have some that are deployment specific) until after they’ve successfully migrated to the new endpoint, at which point the preferred server can handle that issuance. And as he notes, you still have to deal with the fact that there are two CIDs that the client could use, from any incoming address. It seems as though the full way to solve this is to define potentially disjoint sets of TPs that different servers could set to match their particular requirements (around more than just migration). In that world, if you had an initial server that couldn’t handle migration and a preferred server that did handle migration, you could set different TPs to indicate that. It goes well beyond that though, any other TP might have a different values for the original server and then have those be different with the preferred server. But that does feel an awful lot like something that goes with full server migration. In the limited form that we have now, if you want to be able to push someone off to a different server, there are some limitations that you need to deal with, like being able to set up enough of a shared configuration for the two that the same TPs work with both. In practice, I think this means we end up with the following situation: The client either moves to the requested address or it doesn’t. If it does, then you’re in good shape, in theory you gave it TPs that matched what the server wanted and everybody is happy. If it doesn’t, then you’re unhappy, since in theory you had a reason you asked it to move in the first place. You might choose to close it down more aggressively than you would on the preferred server to save resources, or give it smaller limits for flow control, etc. When you’re in the “client didn’t move” state: If the client then tries to use the original 2nd CID that you gave, from the same IP address/port, then a non-CID aware load balancer should be fine. If the client then tries to use the original 2nd CID that you gave, from a different IP address/port, then a non-CID aware load balancer will not be happy. Of course, the problematic case is even narrower, since if the client is probing the new path they’ll see it as though there isn’t any connectivity, just like they will if there’s anything else on the new path that prevents their connection from working — this is always a risk when migrating, so they should be unsurprised and stick with their current path. If they’re forced to move for whatever reason, then it will be just as though they didn’t have connectivity over the new path (which they don’t) and whether or not you disable migration you’ll see the same outcome — the connection is closed. Interestingly, I’m not actually seeing a difference from the client’s perspective in terms of outcome whether you let them try with that CID and fail or if you told them to not try.
In very much shorter terms: isn’t this a situation that all clients already need to be able to handle for other reasons?
NAME I do not like the first option, because IMO banning migration on the original address is going to be a premature optimization based on what we have now, has negative effect on what we would have in the future. My assumption is that the majority of the server deployments would be CID-aware, especially in the long term, especially in terms of traffic volume. Note that the volume of traffic is more important than the number of server deployments, as migration is about (improving) end-user experience. IMO there are two ways forward: (a) State that disableactivemigration TP only applies to the original address. Per-definition, server-preferred address is an address that is capable of receiving packets from any address. There is no need to prohibit clients doing migration when connected to SPA. (b) Do nothing. Clients are RECOMMENDED to implement SPA, and this issue is about a small fraction of deployments that rely on non-CID-aware load balancers. We can ignore the problem.
IIRC, we had that conversation when deciding whether to add in the first place. The short version is that taking this approach prevents the client from rotating CIDs on the primary path; if we want the client to be able to rotate CIDs but not migrate, they need to be distinct signals. NAME in a way you're correct that clients will have to deal with the fallout regardless of what the server says. However, the actual use of is likely to trigger a little more than just "don't do that" -- a client might PING more actively to prevent NAT rebindings, for example. I think it's this hinting aspect that's actually more valuable, since as you note, clients need to be prepared for attempted migrations to fail even with servers that tacitly support them. I don't think this is necessarily true, though I agree that we're talking about smaller and smaller fractions of server space.
NAME While I agree that it might not necessarily be true at the moment, I might argue that it is going to be true. The basis of the resolution for and (both on consensus call) is that servers using preferred address would not be using the server's address for distinguishing between connections. We also know that a server cannot use client's address (unless there is out-of-band knowledge), because clients are not required to use the same 2-tuple when migrating to a different address (and also because there are NATs). The outcome is that the server has to rely on CID when using SPA. And therefore I might insist that servers using SPA would always be capable of handling further migration.
NAME So your suggestion is that disableactivemigration applies only to the original address and the SPA must always support active migration, because it has to use CID routing? I don't think that'd be a problem for our use cases, but I'm not sure about others.
NAME Exactly. I think that that would be the correct thing to do, if we are to make changes. I agree that existing clients might be applying after migrating to SPA, and that they do not want change. But they can remain as they are, because the change in spec does not lead to interoperability issues. To paraphrase, I think making the aforementioned change is strictly better than doing nothing, because it would be an improvement to some, while not hurting anybody.
A truth table might help here. ... --- no preferred address preferred address Right now, our definition of disableactivemigration is a simple "don't migrate". But the argument here says that the server that provides a preferred address is probably capable of handling migrations on the basis that it has to support one. We might insist on the source address not change, but the fact is that some NATs (those with address-dependent mappings) will change the source address. Changing to a definition where disableactivemigration is instead "don't move to a new address unless you use a preferred address" is a little more complicated, but doable. Is it worth the extra complexity this late in the process? Is that complexity justified when it would be trivially possible to define another extension that said "I support migration on my preferred address"?
I'd like to minimize changes, but I believe the status quo is that disableactivemigration applies to both the original address and SPA. As such, all suggestions above seem like a change. There is a mechanism to disable migration on the SPA, which is don't give out any more CIDs once they migrate. I don't expect any hosts to support migration on the original path and not the SPA, so I don't think that's a use case worth supporting(or encouraging).
My proposal is that we defer the ability to signal "I support migration, but only on my preferred address" to an extension.
NAME And leave the current behavior of disableactivemigration applies to both the original address and SPA? I'm a bit concerned that makes SPA somewhat useless. Is this issue a candidate for parking and we can come back to and see if we can live with the status quo?
Which, again, not only disables migration, it disables the ability to rotate CIDs. That's not something we want servers to be doing. NAME argument is convincing that it's improbable for something which supports SPA to not be able to uniquely identify connections (whether by CID or because the SPA endpoint is unique). Therefore, I'd support scoping disableactivemigration to the handshake address, and defer the ability to disable it on the SPA to an extension. Loosening this definition doesn't make any current clients break, so I suspect it's bearable despite the schedule.
If we're going to make a change, I support NAME and NAME direction of scoping disableactivemigration to the handshake address. And I agree that I don't expect this breaks many existing clients.
I personally find the proposed change confusing. I'm not sure I find the use-case of "my original address doesn't support migration but my preferred address does" compelling. Since this can be easily solved by an extension with no downside over it being in the core docs, I don't see a reason to change the spec at this point in the process.
As this has sat here a while, I'll reiterate my proposal again: we defer the ability to signal "I support migration, but only on my preferred address" to an extension. Which means that disableactivemigration applies to both the original address and SPA, so answer the question (sorry Ian, I missed that). I realize that you could change the meaning of disableactivemigration to apply to the address the server uses in the handshake, but there are interesting corner cases and I don't think we need to concern ourselves with those. For instance, if you connect over V4 and get disableactivemigration and serverpreferredaddress. If that SPA includes a V6 address you can't use, along with the same V4 address, what then? Does the server really support migration? I know that the current situation is unappealing, and I think that disableactivemigration is not good for the protocol. But it exists for very good reasons. Trying to limit the scope is admirable, but as we haven't seen a concrete proposal, I think we need to move on.
NAME If this example is concerning, I'd argue that is an issue that already exists, orthogonal to what we are discussing here. I am still not convinced that limiting the scope of disableactivemigration to just the original address is going to cause any new issue. That said, if nobody needs this change, I'm more than happy to move on with status-quo.
I've thought about this, and I don't believe we need this change, but I also think the change NAME suggested makes the current behavior better overall. A server that supports migration on the SPA(ie: unicast) and not on the handshake address(ie: anycast) will end up not disabling migration and hoping that most clients migrate to the SPA or don't support migration of any sort. That seems like a plausible hope, though I'm a bit leery of hope based protocol design.
Thinking about this more, a statement like: "Migration is never guaranteed to succeed. For servers that advertise a preferred address, it's expected migration is more likely to be successful when using the preferred address than the handshake address."
tl;dr: I'm arguing for "do nothing". This conversation has made it clear to me that an address, and not a server, is the thing that supports client migration. Which makes sense -- migration support at the server is entirely a question of routability, and it makes sense that it would be tied to the address in question and not the endpoint. So if we were to do anything, I think I would propose: scoping disableactivemigration to the handshake address, and adding two disableactivemigration flags to the preferredaddress TP, for the two addresses in the TP. That said, this is a clean enough extension. A new extension can define a new TP that provides this granular interpretation of the three server addresses. A server that wishes to support this can send both the SPA and this newSPA TPs, with the SPA TP (with the current scoping of disableactivemigration applying to all server addresses) providing cover for legacy clients. Also I expect that clients are likely to switch to using the SPA if provided. A server that has different routability properties for the two addresses can wait for some deployment and make decisions on whether to risk allowing migration based on data of what it sees as client behavior with SPA. I would argue for doing nothing now. Experience will teach us the right way forward. If necessary, this extension will be easy enough to do, and importantly, the current interpretation of disableactive_migration gives us the right kind of cover in the meantime.
I would rather not make the change in . It assumes that migration is more likely to succeed when attempted to the preferred address as opposed to the handshake address. While this assumption might hold in some server deployments, it is not a fact. I'd rather we didn't bake this assumption into the core spec. I'd prefer to see this feature as an extension.
NAME Would you mind elaborating why a server, accepting connections on a preferred address, might not be able to disambiguate connections using CIDs? As stated previously, when a client migrates to the preferred address, the client's address port tuple can change due to the existence of NATs. A server using preferred address need to take that into consideration. That means that a server has to disambiguate connections based on the CID, which in turn means that the server would be possible to support migration.
Hmmm thanks for that note NAME Looking at things under that light, I agree that the assumption I described above is already part of the protocol. Please disregard my comment above URL .
We don't prohibit the use of a zero-length connection ID in the preferredaddress transport parameter. So a server can use a connection ID during connection establishment, then remove privacy protections by moving to a unique address (see ). Of course, this is largely moot, as the privacy protection offered through the use of connection IDs toward a unique address is nil. In either case, we should clarify whether preferredaddress is exactly like NEWCONNECTIONID, or whether it has its own rules. Right now, it looks like it has its own rules and that is likely bad.
Zero-length connection IDs don't have a stateless reset token associated with them, but the preferred_address TP requires you to set a token. That's an awkward corner case.
That is just one of the strange things, yeah. We should clarify that.
So I see a number of ways forward here: Do nothing. Leave this ambiguous and potentially special. Make the fields in preferredaddress exactly equivalent to a NEWCONNECTIONID with a sequence number of 1. But allow the connection ID to be zero-length. A zero-length connection ID provides a new stateless reset token that is bound to the new address. (This is a weird option that would go against previous decisions to eliminate special handling for this connection ID, but I include it for the sake of completeness.) As 2, except that the new stateless reset token has connection-wide scope such that either of the provided tokens can be used. This is more consistent with established principles. Prohibit the use of zero-length connection IDs in preferredaddress. That is, you can't use preferredaddress without a connection ID. That is, preferredaddress effectively is a new server address plus a NEWCONNECTIONID frame. Something else. From my perspective, I rank these as 4 > 3 >> 1 > 2. But that is largely based on my bias against zero-length connection IDs (as much as it seems like that might be good for privacy, I think that privacy is better served by the use of connection IDs). If people intend to use preferred_address to migrate to a connection-specific address, then I'm OK with 3; we already have to cover the privacy hole that arises from that.
My preference goes to option 4 too. I do not think servers would use both 0-byte CID and preferredaddress TP at the same time. The only case a server using 0-byte CID can also use preferredaddress is: when the server is never accepting more than one connection at a time, or when the server knows that the client's address tuple does not change when the client migrates to a new address IMO that's a niche. And when these cases can be met, I do not see why the client cannot know the preferred address of the server beforehand.
I forgot to add: There is (maybe) a case for using a connection ID during setup and a zero-length connection ID after migrating to a preferred address. That isn't entirely crazy, but I personally would be comfortable losing that option. But I don't operate a server, so I want to give those who do a chance to let me know that this is preemptive.
NAME My personal take would be that people would not do that. When a client migrates to the preferred address, there's fair chance that the client's address tuple will change. That means that the server should be capable of receiving short header packets with 0-byte CID from any address, and determine that the packet belongs to the connection that is migrating to the preferred address. That's possible theoretically, but in practice ... I agree that a server might want to switch to zero-byte CID after migration to SPA completes. But to do that, we need a NEWCONNECTIONID that can carry a zero-byte CID, which is forbidden at the moment. I do not think we'd want to change the spec now to allow that.
That's an excellent point, and a good reason not to allow changing to zero-length connection IDs in combination with preferred_address. The nasty part of that is that in most cases that design would work because clients won't use the "new" connection ID until after the handshake is complete. Of course, it would fail spectacularly if a client did decide to switch earlier - and the client might attract the blame, where it is clearly the fault of the server.
I'll note that prohibiting a 0 length CID doesn't actually ensure the server's IP isn't unique, though it does remove an incentive to make it unique. I'd like to leave the option to switch to a zero-length CID later in the connection, and I think I'd be fine with 4 if we allowed that. An easy example is a large upload. I mostly don't care about the CID, but if I'm uploading a 10GB file, maybe saving some bytes matters. Also, if you're constantly filling the pipe, NAT rebinds are extremely unlikely. Another example is an environment like a datacenter where there are no NATs and routes are very stable, unless a peer intentionally changes it, so the 5-tuple is sufficient. In that case, the ideal operating condition is to typically use a 0 byte CID and switch to a non-0 length anytime a peer changes IP or port. Changing port has some clear benefits in some cases, so it's in no way a threoretical use case. NAME may have thoughts on this range of use case as well.
In our implementation, we plan to do all routing based purely on the CID. IMO, the couple of byte overhead in a 1500 MTU (at least; in datacenter) packet isn't going to make much difference, so it's just simpler to have a common CID routing scheme. That being said, I personally am not against allowing for others to do something different. I've never understood why we make any restrictions on changing between 0-length and non-0-length CIDs at all. IMO, the complexity is that it can change length at all, not that it's zero or not.
NAME we already prohibit changing from non-zero-length to zero-length using NEWCONNECTIONID. I don't want to relitigate that. I want to concentrate on this specific issue.
If we're not going to re-litigate that, then I'd suggest we continue allowing a 0-length CID for the preferred address case.
NAME While I would not argue strongly against continue allowing that, I would appreciate it if you could clarify why allowing a zero-byte CID would be useful. As pointed out in URL, when you provide a zero-byte CID in SPA, you need to be prepared for receiving a short header packet using zero-length CID from any IP address, and be capable of associating that to an existing connection. This is because the client (or the network) might change the client's address tuple when it sends packets to the preferred address. I would anticipate that you wouldn't want to do that in your file upload case (or in any deployment that might accept multiple connections concurrently).
NAME If I know 5-tuples are stable, then using a zero-length CID is always fine, just like it's typically fine in the server to client direction. But in that case, hopefully I can avoid needing to use SPA, making the decision on this issue moot. After thinking about this more, we're probably better off with option 4 for the sake of consistency and if we want to allow 0-length CIDs in more cases, I can write an extension for how that would work, since I don't think it's only a matter of relaxing some constraints.
Thanks all. I think that we reached the same conclusion in the end, via different paths. The proposed fix here is firmly design chairs. A bugfix only, but still worth careful review.
Apologies for getting to this late, but I'm not sure I agree that we should disallow this. Here's a use-case that I think makes sense: the client wants to upload a very large file to the server over HTTP/3 and it has a small MTU, making it particularly sensitive to per-packet overheads. Let's also assume that we are in the future and the server supports IPv6 and has its own /64 (shocking!). In this scenario, a good way to reduce per-packet overhead is to operate the server on a given address, and then switch each client to their own unique server IP using preferred address, and then use zero-length CIDs. I've always found this use-case a good use of preferred address, and I would rather we didn't ban it without a good reason (and making the design cleaner doesn't necessarily count as a good reason at this stage in the process).
That is an example we discussed. As Kazuho noted, that doesn't work as we have already agreed that any connection ID (in this case, a zero-length one) can be used prior to handshake completion. That's why we ended up on 4. As Ian noted, this isn't ideal, but this is an area that is ripe for extensions that might allow this sort of thing.
I don't understand what you meant by the following: Can you clarify? I was under the impression that the client switches to the preferred address after the handshake is done.
Yes, you can't migrate (act on the address information) before the handshake is confirmed. However, the connection ID is not special; it can be treated just like any other connection ID you learn, so you can act on the connection ID information before you confirm the handshake. So that means that a server that opts to include a zero-length connection ID in the preferred_address might receive a packet with the zero-length connection ID at the old address. That might not work out.
What's the point of including a connection ID (and stateless reset token) in the transport parameter if that CID isn't tied to the preferred address in any way? At that point might as well remove that CID entirely and rely on NEWCONNECTIONID frames. My understanding was that the reason for there being a CID in the TP was specifically to make that CID special. Would it make sense to just say that this CID MUST NOT be used when sending to the handshake server IP?
the reason for having a connection ID in the transport parameter is only so that the client has a connection ID to use for the new path
NAME I agree with your intuition, but I think we discussed this in Zurich and ended up concluding the preferredaddress CIDs weren't special. This was in issue and also came up in (which I realize hasn't gone out for a consensus call yet, that PR seems stalled?). In the upload example, the server needs to know that the client is going to send a lot of data, which it might know via SNI, or it might not. I tend to think the more annoying restriction is that you can't change from a non-0 length CID to a 0-length one and vice versa. By supplying a 0-length CID later in the connection along with other CIDs, the client could choose whether it wanted to use a 0-length CID at any time. Thinking out loud, an easier solution may be a TP like 'acceptemptyconnectionid' that indicates at anytime, the peer can switch to a 0-length CID if it wants. It could either think the 5-tuple is stable or its happy re-establishing the path with a non 0-length CID after a NAT rebind. In this world, the Connection ID is more of a path re-establishment token.
Thanks for the details NAME and NAME I get a sense I might be trying to re-litigate past consensus without providing new information, so I'll back down. I do think that the ghost of hypothetical linkability has caused us to make some very odd design choices, but this isn't new to this issue/PR. In the end, this is the kind of feature that can be enabled by extensions so I can live with us further restricting the base spec for consistency.

connection ID when the client initiates migration to the preferred address. <ins> The Connection ID and Stateless Reset Token fields of a preferred address are identical in syntax and semantics to the corresponding fields of a NEW_CONNECTION_ID frame (frame-new-connection-id).  A server that chooses a zero-length connection ID MUST NOT provide a preferred address.  Similarly, a server MUST NOT include a zero- length connection ID in this transport parameter.  A client MUST treat violation of these requirements as a connection error of type TRANSPORT_PARAMETER_ERROR. </ins> The active connection ID limit is an integer value specifying the maximum number of connection IDs from the peer that an endpoint is willing to store.  This value includes the connection ID received


Question:
Now that we require a minimum packet size, this is no longer true.
Is this no longer true for the algorithms defined in the spec, or no longer true for any packet protection algorithms for any AEAD? It might be worth moving the note to the end of 5.4.1 instead.

approach 2^(-L/2), that is, the birthday bound.  For the algorithms described in this document, that probability is one in 2^64. <del> In some cases, inputs shorter than the full size required by the packet protection algorithm might be used. </del> To prevent an attacker from modifying packet headers, the header is transitively authenticated using packet protection; the entire packet header is part of the authenticated additional data.  Protected