Question:
…transport protocols Also tighten up description of the assumed protocol stack in (what is now) Section 6.1. Note that the UDP and TCP sections have been flipped in order, to improve section flow and clarity.
NAME - If you have a moment before our call on Friday to take a look at this PR, I'd appreciate it. I think it's important to to get that right, in order to address both the INTDIR and TSVART reviews.
From NAME As a bit of transport commentary, I don't find the setup of Section 6 compelling. While UDP is a transport in a technical sense, for the purpose of media, it is almost always a layer upon which the protocol doing the congestion control work runs. To that end, QUIC is just another more standardized case of this, just like SCTP over UDP. Rather than talking about "UDP's behavior" vs "TCP's behavior", I suggest talking about how applications over UDP for media behave vs applications over TCP for media behave.
NAME - I need to look at this more closely. I understand your point about UDP, but (because UDP is not a real transport layer, while TCP and QUIC are), I suspect (without looking closely yet) I'm going to be making sure that I explain this more clearly, without trying to explain the behavior(s) of applications running over TCP and QUIC. As an aside, I'm trying to make this clearer in drafts I'm targeting forwards Media Over QUIC/MOQ, so I actually DO believe you ... Please await results.
A +1 from NAME 's review (URL): Section 6. Evolution of Transport Protocols and Transport Protocol Behaviors I really wonder whether UDP vs. TCP vs. QUIC is actually the relevant distinction. What may actually matter are the transport services provided by the protocol stack (e.g., RFC 8095). I fully agree to a related comment in the INTDIR review.
After looking at this issue. I agree with both NAME and NAME that the emphasis should be on the interaction between (see new section describing this) media transport protocols and the transport services provided by underlying transport protocols. I'm moving the formerly-UDP section below the formerly-TCP section, because it's easier to describe what UDP is NOT doing after describing what TCP IS doing. I'm also moving one paragraph into the formerly-TCP section from Section 6, because it is specific to ABR strategies that assume a reliable transport. More news to come, when I create a PR for this issue ...
We think is now mergrable. Reviewer: Michael Scharf Review result: Ready with Issues This document has been reviewed as part of the transport area review team's ongoing effort to review key IETF documents. These comments were written primarily for the transport area directors, but are copied to the document's authors and WG to allow them to address any issues raised and also to the IETF discussion list for information. When done at the time of IETF Last Call, the authors should consider this review as part of the last-call comments they receive. Please always CC EMAIL if you reply to or forward this review. This informational document surveys network and transport protocol issues that affect quality of experience for streaming applications, in particular video. The overview covers many topics across different technologies. Readers may only be familiar with a subset of these topics and could therefore learn quite a bit from this kind of tutorial. Nonetheless, it remains somewhat unclear what the actual objective of some of the text is. For some topics, quite a bit of technical background is provided, but the discussion is not really comprehensive. In many sections, the document neither derives technical challenges, nor system/protocol requirements, nor practical usage guidance. It is just a kind of tutorial. Quite a bit of text also presents ongoing IETF work, and an RFC with this scope may thus get outdated soon. Section 6.2 deals with topics owned by TCPM. In similar past documents, I have asked for a TCPM presentation prior to an IETF last call in order to ensure that owners of running code are in the loop. I believe this strategy has worked well in the past. Having said this, as TSV-ART reviewer I don't strongly disagree with a publication. All these issues may just be OK for an informational RFC. Some more specific comments: Section 3.2.1. Recognizing Changes from an Expected Baseline This section apparently assumes relatively static path properties, e.g., fixed network connectivity. It would be more useful to analyze the impact of Wifi and 4G/5G networks in one place. Some of this follows later in sections 3.2.1 and Yet, the split between these three sections is unclear. Having a discussion about today's Internet path characteristics and the dynamics at one place could be more useful. The text could also better distinguish between what matters to endpoints and considerations relevant only for middleboxes (such as passive monitors). Section 3.3 Path Requirements The statement "to find the bandwidth requirements for a router on the delivery path" may assume that the bottleneck on the path will be routers. Yet, on a path the bottlenecks could also be a link layer device (e.g., an Ethernet Switch, a Wifi access point, etc.). RFC 6077 (specifically Section 3.1.3) also explains that issue. A better wording may be "to find the bandwidth requirements for a delivery path". Section 3.6. Unpredictable Usage Profiles / Section 3.7. Extremely Unpredictable Usage Profiles I am not fully convinced by the distinction between "unpredictable" and "extremely unpredictable". To me, these two sections could be merged and maybe also be shortened. More specifically, Section 3.7 lists a lot of statistics from the past. What is IMHO a bit missing in Section 3.7 are actual operational considerations. For instance, are there any lessons learnt for the future? Note that Section 3.6 has a reasonable conclusion at the end. Section 4. Latency Considerations and Section 4.1. Ultra Low-Latency I am surprised by the definition "ultra low-latency (less than 1 second)", as well as some of the other numbers. For other real-time communication use cases, "ultra low-latency" would probably imply a latency requirement of the order of one millisecond. For instance, isochronous traffic for motion control in industrial networks may require a latency of 1 ms, and such "ultra-low latency" requirements are discussed elsewhere, e.g., in the DetNet WG. The terminology in this section should be better explained to readers dealing with networked systems with much harder latency requirements. And a reference should be added for these definitions in the context of video streaming. Section 5.5.2 Head-of-Line Blocking If Head-of-Line Blocking is indeed a relevant operational problem, it would be useful to add a corresponding reference (e.g., with measurements). Section 5.5.3. Wide and Rapid Variation in Path Capacity The statement "As many end devices have moved to wireless connectivity for the final hop (Wi-Fi, 5G, or LTE), new problems in bandwidth detction have emerged from radio interference and signal strength effects." could be moved to earlier parts of the document. Quite a bit of new computers apparently only have Wifi connectivity and no Ethernet port, i.e., Wifi may just be their default. Also, wireless may not only be used on the last hop, for instance in meshed setups. This may make the problem even harder. Section 6. Evolution of Transport Protocols and Transport Protocol Behaviors I really wonder whether UDP vs. TCP vs. QUIC is actually the relevant distinction. What may actually matter are the transport services provided by the protocol stack (e.g., RFC 8095). I fully agree to a related comment in the INTDIR review. Section 6.1. UDP and Its Behavior UDP is also used for encrypted tunnels (OpenVPN, Wireguard, etc.). Does encrypted tunneling really have no operational impact on streaming apps other than circuit breakers? (I am thinking about 4K streaming over OpenVPN and the like.) Section 6.2. TCP and Its Behavior This text should IMHO be presented and be discussed in TCPM. Personally, I am not convinced that this document is a good place for discussing the long history of TCP congestion control. If the objective of the document is to provide a tutorial, IMHO it would be more useful to briefly explain the state-of-the-art in year 2022. Some further specific comments on TCP congestion control: It is surprising that RFC 5681 is not referenced.793bis has normative statements regarding congestion control, and most stacks are compatible to what is written in 793bis. Several operating systems use CUBIC by default and support standard Reno. Another general good reference for TCP standards is RFC 7414.Does ECN, DCTCP, etc. really not matter at all, for instance, if a data center hosts video playout servers? Section 6.3. QUIC and Its Behavior As already noted, the discussion about head-of-line-blocking would really benefit from backing by a reference. Section 7. Streaming Encrypted Media As far as I know, streaming users sometimes use encrypted tunnels such as OpenVPN or WireGuard (or IPsec) to access video content. I may miss something, but it is unclear to me how that fits into the categories presented in Section

While QUIC is designed as a general-purpose transport protocol, and can carry different application-layer protocols, the current <del> standardized mapping is for HTTP/3 I-D.ietf-quic-http, which describes how QUIC transport features are used for HTTP.  The convention is for HTTP/3 to run over UDP port 443 Port443 but this is not a strict requirement. </del> <ins> standardized mapping is for HTTP/3 RFC9114, which describes how QUIC transport services are used for HTTP.  The convention is for HTTP/3 to run over UDP port 443 Port443 but this is not a strict requirement. </ins> When HTTP/3 is encapsulated in QUIC, which is then encapsulated in UDP, streaming operators (and network operators) might see UDP <del> traffic patterns that are similar to HTTP(S) over TCP.  Since earlier versions of HTTP(S) rely on TCP, UDP ports may be blocked for any port numbers that are not commonly used, such as UDP 53 for DNS. Even when UDP ports are not blocked and HTTP/3 can flow, streaming operators (and network operators) may severely rate-limit this traffic because they do not expect to see legitimate high-bandwidth traffic such as streaming media over the UDP ports that HTTP/3 is using. </del> <ins> traffic patterns that are similar to HTTP(S) over TCP.  UDP ports may be blocked for any port numbers that are not commonly used, such as UDP 53 for DNS.  Even when UDP ports are not blocked and QUIC packets can flow, streaming operators (and network operators) may severely rate-limit this traffic because they do not expect to see legitimate high-bandwidth traffic such as streaming media over the UDP ports that HTTP/3 is using. </ins> As noted in hol-blocking, because TCP provides a reliable, in-order delivery service for applications, any packet loss for a TCP


Question:
…transport protocols Also tighten up description of the assumed protocol stack in (what is now) Section 6.1. Note that the UDP and TCP sections have been flipped in order, to improve section flow and clarity.
NAME - If you have a moment before our call on Friday to take a look at this PR, I'd appreciate it. I think it's important to to get that right, in order to address both the INTDIR and TSVART reviews.
From NAME As a bit of transport commentary, I don't find the setup of Section 6 compelling. While UDP is a transport in a technical sense, for the purpose of media, it is almost always a layer upon which the protocol doing the congestion control work runs. To that end, QUIC is just another more standardized case of this, just like SCTP over UDP. Rather than talking about "UDP's behavior" vs "TCP's behavior", I suggest talking about how applications over UDP for media behave vs applications over TCP for media behave.
NAME - I need to look at this more closely. I understand your point about UDP, but (because UDP is not a real transport layer, while TCP and QUIC are), I suspect (without looking closely yet) I'm going to be making sure that I explain this more clearly, without trying to explain the behavior(s) of applications running over TCP and QUIC. As an aside, I'm trying to make this clearer in drafts I'm targeting forwards Media Over QUIC/MOQ, so I actually DO believe you ... Please await results.
A +1 from NAME 's review (URL): Section 6. Evolution of Transport Protocols and Transport Protocol Behaviors I really wonder whether UDP vs. TCP vs. QUIC is actually the relevant distinction. What may actually matter are the transport services provided by the protocol stack (e.g., RFC 8095). I fully agree to a related comment in the INTDIR review.
After looking at this issue. I agree with both NAME and NAME that the emphasis should be on the interaction between (see new section describing this) media transport protocols and the transport services provided by underlying transport protocols. I'm moving the formerly-UDP section below the formerly-TCP section, because it's easier to describe what UDP is NOT doing after describing what TCP IS doing. I'm also moving one paragraph into the formerly-TCP section from Section 6, because it is specific to ABR strategies that assume a reliable transport. More news to come, when I create a PR for this issue ...
We think is now mergrable. Reviewer: Michael Scharf Review result: Ready with Issues This document has been reviewed as part of the transport area review team's ongoing effort to review key IETF documents. These comments were written primarily for the transport area directors, but are copied to the document's authors and WG to allow them to address any issues raised and also to the IETF discussion list for information. When done at the time of IETF Last Call, the authors should consider this review as part of the last-call comments they receive. Please always CC EMAIL if you reply to or forward this review. This informational document surveys network and transport protocol issues that affect quality of experience for streaming applications, in particular video. The overview covers many topics across different technologies. Readers may only be familiar with a subset of these topics and could therefore learn quite a bit from this kind of tutorial. Nonetheless, it remains somewhat unclear what the actual objective of some of the text is. For some topics, quite a bit of technical background is provided, but the discussion is not really comprehensive. In many sections, the document neither derives technical challenges, nor system/protocol requirements, nor practical usage guidance. It is just a kind of tutorial. Quite a bit of text also presents ongoing IETF work, and an RFC with this scope may thus get outdated soon. Section 6.2 deals with topics owned by TCPM. In similar past documents, I have asked for a TCPM presentation prior to an IETF last call in order to ensure that owners of running code are in the loop. I believe this strategy has worked well in the past. Having said this, as TSV-ART reviewer I don't strongly disagree with a publication. All these issues may just be OK for an informational RFC. Some more specific comments: Section 3.2.1. Recognizing Changes from an Expected Baseline This section apparently assumes relatively static path properties, e.g., fixed network connectivity. It would be more useful to analyze the impact of Wifi and 4G/5G networks in one place. Some of this follows later in sections 3.2.1 and Yet, the split between these three sections is unclear. Having a discussion about today's Internet path characteristics and the dynamics at one place could be more useful. The text could also better distinguish between what matters to endpoints and considerations relevant only for middleboxes (such as passive monitors). Section 3.3 Path Requirements The statement "to find the bandwidth requirements for a router on the delivery path" may assume that the bottleneck on the path will be routers. Yet, on a path the bottlenecks could also be a link layer device (e.g., an Ethernet Switch, a Wifi access point, etc.). RFC 6077 (specifically Section 3.1.3) also explains that issue. A better wording may be "to find the bandwidth requirements for a delivery path". Section 3.6. Unpredictable Usage Profiles / Section 3.7. Extremely Unpredictable Usage Profiles I am not fully convinced by the distinction between "unpredictable" and "extremely unpredictable". To me, these two sections could be merged and maybe also be shortened. More specifically, Section 3.7 lists a lot of statistics from the past. What is IMHO a bit missing in Section 3.7 are actual operational considerations. For instance, are there any lessons learnt for the future? Note that Section 3.6 has a reasonable conclusion at the end. Section 4. Latency Considerations and Section 4.1. Ultra Low-Latency I am surprised by the definition "ultra low-latency (less than 1 second)", as well as some of the other numbers. For other real-time communication use cases, "ultra low-latency" would probably imply a latency requirement of the order of one millisecond. For instance, isochronous traffic for motion control in industrial networks may require a latency of 1 ms, and such "ultra-low latency" requirements are discussed elsewhere, e.g., in the DetNet WG. The terminology in this section should be better explained to readers dealing with networked systems with much harder latency requirements. And a reference should be added for these definitions in the context of video streaming. Section 5.5.2 Head-of-Line Blocking If Head-of-Line Blocking is indeed a relevant operational problem, it would be useful to add a corresponding reference (e.g., with measurements). Section 5.5.3. Wide and Rapid Variation in Path Capacity The statement "As many end devices have moved to wireless connectivity for the final hop (Wi-Fi, 5G, or LTE), new problems in bandwidth detction have emerged from radio interference and signal strength effects." could be moved to earlier parts of the document. Quite a bit of new computers apparently only have Wifi connectivity and no Ethernet port, i.e., Wifi may just be their default. Also, wireless may not only be used on the last hop, for instance in meshed setups. This may make the problem even harder. Section 6. Evolution of Transport Protocols and Transport Protocol Behaviors I really wonder whether UDP vs. TCP vs. QUIC is actually the relevant distinction. What may actually matter are the transport services provided by the protocol stack (e.g., RFC 8095). I fully agree to a related comment in the INTDIR review. Section 6.1. UDP and Its Behavior UDP is also used for encrypted tunnels (OpenVPN, Wireguard, etc.). Does encrypted tunneling really have no operational impact on streaming apps other than circuit breakers? (I am thinking about 4K streaming over OpenVPN and the like.) Section 6.2. TCP and Its Behavior This text should IMHO be presented and be discussed in TCPM. Personally, I am not convinced that this document is a good place for discussing the long history of TCP congestion control. If the objective of the document is to provide a tutorial, IMHO it would be more useful to briefly explain the state-of-the-art in year 2022. Some further specific comments on TCP congestion control: It is surprising that RFC 5681 is not referenced.793bis has normative statements regarding congestion control, and most stacks are compatible to what is written in 793bis. Several operating systems use CUBIC by default and support standard Reno. Another general good reference for TCP standards is RFC 7414.Does ECN, DCTCP, etc. really not matter at all, for instance, if a data center hosts video playout servers? Section 6.3. QUIC and Its Behavior As already noted, the discussion about head-of-line-blocking would really benefit from backing by a reference. Section 7. Streaming Encrypted Media As far as I know, streaming users sometimes use encrypted tunnels such as OpenVPN or WireGuard (or IPsec) to access video content. I may miss something, but it is unclear to me how that fits into the categories presented in Section

connection's congestion controller, providing some transport-level congestion avoidance measures, which UDP does not. <del> As noted in tcp-behavior, there is an increasing interest in </del> <ins> As noted in reliable-behavior, there is an increasing interest in </ins> congestion control algorithms that respond to delay measurements, instead of responding to packet loss.  These algorithms may deliver an improved user experience, but in some cases, have not responded to


Question:
…transport protocols Also tighten up description of the assumed protocol stack in (what is now) Section 6.1. Note that the UDP and TCP sections have been flipped in order, to improve section flow and clarity.
NAME - If you have a moment before our call on Friday to take a look at this PR, I'd appreciate it. I think it's important to to get that right, in order to address both the INTDIR and TSVART reviews.
From NAME As a bit of transport commentary, I don't find the setup of Section 6 compelling. While UDP is a transport in a technical sense, for the purpose of media, it is almost always a layer upon which the protocol doing the congestion control work runs. To that end, QUIC is just another more standardized case of this, just like SCTP over UDP. Rather than talking about "UDP's behavior" vs "TCP's behavior", I suggest talking about how applications over UDP for media behave vs applications over TCP for media behave.
NAME - I need to look at this more closely. I understand your point about UDP, but (because UDP is not a real transport layer, while TCP and QUIC are), I suspect (without looking closely yet) I'm going to be making sure that I explain this more clearly, without trying to explain the behavior(s) of applications running over TCP and QUIC. As an aside, I'm trying to make this clearer in drafts I'm targeting forwards Media Over QUIC/MOQ, so I actually DO believe you ... Please await results.
A +1 from NAME 's review (URL): Section 6. Evolution of Transport Protocols and Transport Protocol Behaviors I really wonder whether UDP vs. TCP vs. QUIC is actually the relevant distinction. What may actually matter are the transport services provided by the protocol stack (e.g., RFC 8095). I fully agree to a related comment in the INTDIR review.
After looking at this issue. I agree with both NAME and NAME that the emphasis should be on the interaction between (see new section describing this) media transport protocols and the transport services provided by underlying transport protocols. I'm moving the formerly-UDP section below the formerly-TCP section, because it's easier to describe what UDP is NOT doing after describing what TCP IS doing. I'm also moving one paragraph into the formerly-TCP section from Section 6, because it is specific to ABR strategies that assume a reliable transport. More news to come, when I create a PR for this issue ...
We think is now mergrable. Reviewer: Michael Scharf Review result: Ready with Issues This document has been reviewed as part of the transport area review team's ongoing effort to review key IETF documents. These comments were written primarily for the transport area directors, but are copied to the document's authors and WG to allow them to address any issues raised and also to the IETF discussion list for information. When done at the time of IETF Last Call, the authors should consider this review as part of the last-call comments they receive. Please always CC EMAIL if you reply to or forward this review. This informational document surveys network and transport protocol issues that affect quality of experience for streaming applications, in particular video. The overview covers many topics across different technologies. Readers may only be familiar with a subset of these topics and could therefore learn quite a bit from this kind of tutorial. Nonetheless, it remains somewhat unclear what the actual objective of some of the text is. For some topics, quite a bit of technical background is provided, but the discussion is not really comprehensive. In many sections, the document neither derives technical challenges, nor system/protocol requirements, nor practical usage guidance. It is just a kind of tutorial. Quite a bit of text also presents ongoing IETF work, and an RFC with this scope may thus get outdated soon. Section 6.2 deals with topics owned by TCPM. In similar past documents, I have asked for a TCPM presentation prior to an IETF last call in order to ensure that owners of running code are in the loop. I believe this strategy has worked well in the past. Having said this, as TSV-ART reviewer I don't strongly disagree with a publication. All these issues may just be OK for an informational RFC. Some more specific comments: Section 3.2.1. Recognizing Changes from an Expected Baseline This section apparently assumes relatively static path properties, e.g., fixed network connectivity. It would be more useful to analyze the impact of Wifi and 4G/5G networks in one place. Some of this follows later in sections 3.2.1 and Yet, the split between these three sections is unclear. Having a discussion about today's Internet path characteristics and the dynamics at one place could be more useful. The text could also better distinguish between what matters to endpoints and considerations relevant only for middleboxes (such as passive monitors). Section 3.3 Path Requirements The statement "to find the bandwidth requirements for a router on the delivery path" may assume that the bottleneck on the path will be routers. Yet, on a path the bottlenecks could also be a link layer device (e.g., an Ethernet Switch, a Wifi access point, etc.). RFC 6077 (specifically Section 3.1.3) also explains that issue. A better wording may be "to find the bandwidth requirements for a delivery path". Section 3.6. Unpredictable Usage Profiles / Section 3.7. Extremely Unpredictable Usage Profiles I am not fully convinced by the distinction between "unpredictable" and "extremely unpredictable". To me, these two sections could be merged and maybe also be shortened. More specifically, Section 3.7 lists a lot of statistics from the past. What is IMHO a bit missing in Section 3.7 are actual operational considerations. For instance, are there any lessons learnt for the future? Note that Section 3.6 has a reasonable conclusion at the end. Section 4. Latency Considerations and Section 4.1. Ultra Low-Latency I am surprised by the definition "ultra low-latency (less than 1 second)", as well as some of the other numbers. For other real-time communication use cases, "ultra low-latency" would probably imply a latency requirement of the order of one millisecond. For instance, isochronous traffic for motion control in industrial networks may require a latency of 1 ms, and such "ultra-low latency" requirements are discussed elsewhere, e.g., in the DetNet WG. The terminology in this section should be better explained to readers dealing with networked systems with much harder latency requirements. And a reference should be added for these definitions in the context of video streaming. Section 5.5.2 Head-of-Line Blocking If Head-of-Line Blocking is indeed a relevant operational problem, it would be useful to add a corresponding reference (e.g., with measurements). Section 5.5.3. Wide and Rapid Variation in Path Capacity The statement "As many end devices have moved to wireless connectivity for the final hop (Wi-Fi, 5G, or LTE), new problems in bandwidth detction have emerged from radio interference and signal strength effects." could be moved to earlier parts of the document. Quite a bit of new computers apparently only have Wifi connectivity and no Ethernet port, i.e., Wifi may just be their default. Also, wireless may not only be used on the last hop, for instance in meshed setups. This may make the problem even harder. Section 6. Evolution of Transport Protocols and Transport Protocol Behaviors I really wonder whether UDP vs. TCP vs. QUIC is actually the relevant distinction. What may actually matter are the transport services provided by the protocol stack (e.g., RFC 8095). I fully agree to a related comment in the INTDIR review. Section 6.1. UDP and Its Behavior UDP is also used for encrypted tunnels (OpenVPN, Wireguard, etc.). Does encrypted tunneling really have no operational impact on streaming apps other than circuit breakers? (I am thinking about 4K streaming over OpenVPN and the like.) Section 6.2. TCP and Its Behavior This text should IMHO be presented and be discussed in TCPM. Personally, I am not convinced that this document is a good place for discussing the long history of TCP congestion control. If the objective of the document is to provide a tutorial, IMHO it would be more useful to briefly explain the state-of-the-art in year 2022. Some further specific comments on TCP congestion control: It is surprising that RFC 5681 is not referenced.793bis has normative statements regarding congestion control, and most stacks are compatible to what is written in 793bis. Several operating systems use CUBIC by default and support standard Reno. Another general good reference for TCP standards is RFC 7414.Does ECN, DCTCP, etc. really not matter at all, for instance, if a data center hosts video playout servers? Section 6.3. QUIC and Its Behavior As already noted, the discussion about head-of-line-blocking would really benefit from backing by a reference. Section 7. Streaming Encrypted Media As far as I know, streaming users sometimes use encrypted tunnels such as OpenVPN or WireGuard (or IPsec) to access video content. I may miss something, but it is unclear to me how that fits into the categories presented in Section

both were then selected as the default TCP congestion controllers in Linux, and both were deployed globally. <del> The point mentioned in tcp-behavior about TCP congestion controllers being implemented in operating system kernels is different with QUIC. Although QUIC can be implemented in operating system kernels, one of the design goals when this work was chartered was "QUIC is expected to support rapid, distributed development and testing of features," and to meet this expectation, many implementers have chosen to implement QUIC in user space, outside the operating system kernel, and to even distribute QUIC libraries with their own applications. It is worth noting that streaming operators using HTTP/3, carried over QUIC, can expect more frequent deployment of new congestion controller behavior than has been the case with HTTP/1 and HTTP/2, carried over TCP. </del> <ins> The point mentioned in reliable-behavior about TCP congestion controllers being implemented in operating system kernels is different with QUIC.  Although QUIC can be implemented in operating system kernels, one of the design goals when this work was chartered was "QUIC is expected to support rapid, distributed development and testing of features," and to meet this expectation, many implementers have chosen to implement QUIC in user space, outside the operating system kernel, and to even distribute QUIC libraries with their own applications.  It is worth noting that streaming operators using HTTP/3, carried over QUIC, can expect more frequent deployment of new congestion controller behavior than has been the case with HTTP/1 and HTTP/2, carried over TCP. </ins> It is worth considering that if TCP-based HTTP traffic and UDP-based HTTP/3 traffic are allowed to enter operator networks on roughly


Question:
…transport protocols Also tighten up description of the assumed protocol stack in (what is now) Section 6.1. Note that the UDP and TCP sections have been flipped in order, to improve section flow and clarity.
NAME - If you have a moment before our call on Friday to take a look at this PR, I'd appreciate it. I think it's important to to get that right, in order to address both the INTDIR and TSVART reviews.
From NAME As a bit of transport commentary, I don't find the setup of Section 6 compelling. While UDP is a transport in a technical sense, for the purpose of media, it is almost always a layer upon which the protocol doing the congestion control work runs. To that end, QUIC is just another more standardized case of this, just like SCTP over UDP. Rather than talking about "UDP's behavior" vs "TCP's behavior", I suggest talking about how applications over UDP for media behave vs applications over TCP for media behave.
NAME - I need to look at this more closely. I understand your point about UDP, but (because UDP is not a real transport layer, while TCP and QUIC are), I suspect (without looking closely yet) I'm going to be making sure that I explain this more clearly, without trying to explain the behavior(s) of applications running over TCP and QUIC. As an aside, I'm trying to make this clearer in drafts I'm targeting forwards Media Over QUIC/MOQ, so I actually DO believe you ... Please await results.
A +1 from NAME 's review (URL): Section 6. Evolution of Transport Protocols and Transport Protocol Behaviors I really wonder whether UDP vs. TCP vs. QUIC is actually the relevant distinction. What may actually matter are the transport services provided by the protocol stack (e.g., RFC 8095). I fully agree to a related comment in the INTDIR review.
After looking at this issue. I agree with both NAME and NAME that the emphasis should be on the interaction between (see new section describing this) media transport protocols and the transport services provided by underlying transport protocols. I'm moving the formerly-UDP section below the formerly-TCP section, because it's easier to describe what UDP is NOT doing after describing what TCP IS doing. I'm also moving one paragraph into the formerly-TCP section from Section 6, because it is specific to ABR strategies that assume a reliable transport. More news to come, when I create a PR for this issue ...
We think is now mergrable. Reviewer: Michael Scharf Review result: Ready with Issues This document has been reviewed as part of the transport area review team's ongoing effort to review key IETF documents. These comments were written primarily for the transport area directors, but are copied to the document's authors and WG to allow them to address any issues raised and also to the IETF discussion list for information. When done at the time of IETF Last Call, the authors should consider this review as part of the last-call comments they receive. Please always CC EMAIL if you reply to or forward this review. This informational document surveys network and transport protocol issues that affect quality of experience for streaming applications, in particular video. The overview covers many topics across different technologies. Readers may only be familiar with a subset of these topics and could therefore learn quite a bit from this kind of tutorial. Nonetheless, it remains somewhat unclear what the actual objective of some of the text is. For some topics, quite a bit of technical background is provided, but the discussion is not really comprehensive. In many sections, the document neither derives technical challenges, nor system/protocol requirements, nor practical usage guidance. It is just a kind of tutorial. Quite a bit of text also presents ongoing IETF work, and an RFC with this scope may thus get outdated soon. Section 6.2 deals with topics owned by TCPM. In similar past documents, I have asked for a TCPM presentation prior to an IETF last call in order to ensure that owners of running code are in the loop. I believe this strategy has worked well in the past. Having said this, as TSV-ART reviewer I don't strongly disagree with a publication. All these issues may just be OK for an informational RFC. Some more specific comments: Section 3.2.1. Recognizing Changes from an Expected Baseline This section apparently assumes relatively static path properties, e.g., fixed network connectivity. It would be more useful to analyze the impact of Wifi and 4G/5G networks in one place. Some of this follows later in sections 3.2.1 and Yet, the split between these three sections is unclear. Having a discussion about today's Internet path characteristics and the dynamics at one place could be more useful. The text could also better distinguish between what matters to endpoints and considerations relevant only for middleboxes (such as passive monitors). Section 3.3 Path Requirements The statement "to find the bandwidth requirements for a router on the delivery path" may assume that the bottleneck on the path will be routers. Yet, on a path the bottlenecks could also be a link layer device (e.g., an Ethernet Switch, a Wifi access point, etc.). RFC 6077 (specifically Section 3.1.3) also explains that issue. A better wording may be "to find the bandwidth requirements for a delivery path". Section 3.6. Unpredictable Usage Profiles / Section 3.7. Extremely Unpredictable Usage Profiles I am not fully convinced by the distinction between "unpredictable" and "extremely unpredictable". To me, these two sections could be merged and maybe also be shortened. More specifically, Section 3.7 lists a lot of statistics from the past. What is IMHO a bit missing in Section 3.7 are actual operational considerations. For instance, are there any lessons learnt for the future? Note that Section 3.6 has a reasonable conclusion at the end. Section 4. Latency Considerations and Section 4.1. Ultra Low-Latency I am surprised by the definition "ultra low-latency (less than 1 second)", as well as some of the other numbers. For other real-time communication use cases, "ultra low-latency" would probably imply a latency requirement of the order of one millisecond. For instance, isochronous traffic for motion control in industrial networks may require a latency of 1 ms, and such "ultra-low latency" requirements are discussed elsewhere, e.g., in the DetNet WG. The terminology in this section should be better explained to readers dealing with networked systems with much harder latency requirements. And a reference should be added for these definitions in the context of video streaming. Section 5.5.2 Head-of-Line Blocking If Head-of-Line Blocking is indeed a relevant operational problem, it would be useful to add a corresponding reference (e.g., with measurements). Section 5.5.3. Wide and Rapid Variation in Path Capacity The statement "As many end devices have moved to wireless connectivity for the final hop (Wi-Fi, 5G, or LTE), new problems in bandwidth detction have emerged from radio interference and signal strength effects." could be moved to earlier parts of the document. Quite a bit of new computers apparently only have Wifi connectivity and no Ethernet port, i.e., Wifi may just be their default. Also, wireless may not only be used on the last hop, for instance in meshed setups. This may make the problem even harder. Section 6. Evolution of Transport Protocols and Transport Protocol Behaviors I really wonder whether UDP vs. TCP vs. QUIC is actually the relevant distinction. What may actually matter are the transport services provided by the protocol stack (e.g., RFC 8095). I fully agree to a related comment in the INTDIR review. Section 6.1. UDP and Its Behavior UDP is also used for encrypted tunnels (OpenVPN, Wireguard, etc.). Does encrypted tunneling really have no operational impact on streaming apps other than circuit breakers? (I am thinking about 4K streaming over OpenVPN and the like.) Section 6.2. TCP and Its Behavior This text should IMHO be presented and be discussed in TCPM. Personally, I am not convinced that this document is a good place for discussing the long history of TCP congestion control. If the objective of the document is to provide a tutorial, IMHO it would be more useful to briefly explain the state-of-the-art in year 2022. Some further specific comments on TCP congestion control: It is surprising that RFC 5681 is not referenced.793bis has normative statements regarding congestion control, and most stacks are compatible to what is written in 793bis. Several operating systems use CUBIC by default and support standard Reno. Another general good reference for TCP standards is RFC 7414.Does ECN, DCTCP, etc. really not matter at all, for instance, if a data center hosts video playout servers? Section 6.3. QUIC and Its Behavior As already noted, the discussion about head-of-line-blocking would really benefit from backing by a reference. Section 7. Streaming Encrypted Media As far as I know, streaming users sometimes use encrypted tunnels such as OpenVPN or WireGuard (or IPsec) to access video content. I may miss something, but it is unclear to me how that fits into the categories presented in Section

certainly as examining decrypted traffic. Because HTTPS has historically layered HTTP on top of TLS, which is <del> in turn layered on top of TCP, intermediaries do have access to unencrypted TCP-level transport information, such as retransmissions, and some carriers exploited this information in attempts to improve transport-layer performance RFC3135.  The most recent standardized version of HTTPS, HTTP/3 I-D.ietf-quic-http, uses the QUIC protocol RFC9000 as its transport layer.  QUIC relies on the TLS 1.3 initial handshake RFC8446 only for key exchange RFC9001, and encrypts almost all transport parameters itself except for a few invariant header fields.  In the QUIC short header, the only transport-level parameter which is sent "in the clear" is the Destination Connection ID RFC8999, and even in the QUIC long header, the only transport-level parameters sent "in the clear" are the Version, Destination </del> <ins> in turn layered on top of TCP, intermediaries have historically had access to unencrypted TCP-level transport information, such as retransmissions, and some carriers exploited this information in attempts to improve transport-layer performance RFC3135.  The most recent standardized version of HTTPS, HTTP/3 RFC9114, uses the QUIC protocol RFC9000 as its transport layer.  QUIC relies on the TLS 1.3 initial handshake RFC8446 only for key exchange RFC9001, and encrypts almost all transport parameters itself except for a few invariant header fields.  In the QUIC short header, the only transport-level parameter which is sent "in the clear" is the Destination Connection ID RFC8999, and even in the QUIC long header, the only transport- level parameters sent "in the clear" are the Version, Destination </ins> Connection ID, and Source Connection ID.  For these reasons, HTTP/3 is significantly more "opaque" than HTTPS with HTTP/1 or HTTP/2.


Question: I actually used the URL in the draft to clone a copy, so I think this is good to go.

4.1. <del> (To the editor: check this repository URL after the draft is adopted. The working group may create its own repository) </del> <ins> This document is in the Github repository at: </ins> <del> This document is in the Github repository at https://github.com/GrumpyOldTroll/ietf-mops-drafts.  Readers are welcome to open issues and send pull requests for this document. </del> <ins> https://github.com/ietf-wg-mops/draft-ietf-mops-streaming-opcons Readers are welcome to open issues and send pull requests for this document. </ins> Substantial discussion of this document should take place on the MOPS working group mailing list (mops@ietf.org). <ins> Join: https://www.ietf.org/mailman/listinfo/mops Search: https://mailarchive.ietf.org/arch/browse/mops/ </ins> 4.2. Presentations: IETF 105 BOF: https://www.youtube.com/watch?v=4G3YBVmn9Eo&t=47m21s <del> IETF 106 Mops meeting: https://www.youtube.com/ </del> <ins> IETF 106 meeting: https://www.youtube.com/ </ins> watch?v=4_k340xT2jM&t=7m23s <del> List archive search for doc mentions: https://mailarchive.ietf.org/arch/browse/mops/?q=draft-ietf-mops- streaming-opcons </del> 5. This document requires no actions from IANA.


Question:
Discussion from editor conference call: NAME will update this PR with NAME comments
From NAME (URL) Section 7. Streaming Encrypted Media As far as I know, streaming users sometimes use encrypted tunnels such as OpenVPN or WireGuard (or IPsec) to access video content. I may miss something, but it is unclear to me how that fits into the categories presented in Section (Seems closely related to URL )
The purpose of such tunneling is to circumvent geoblocks and access content from other regions - not to protect the media content. Encrypting the content is done by the content owner for its protection.
So, looking at , I'm seeing this text: I think this means that the effect is that the "transport-level information is opaque to intermediaries" effect described in the document applies, with the additional consideration that intermediaries can't see the IP addresses used inside the tunnel, either. I think the same applies to any IPsec-encrypted tunnel. Do we need to say that? And, if so - can we describe this without using product names and descriptions which may age badly in an RFC?
isn't as easy to read, but I think this is also the rough equivalent of IPsec, from a media operator perspective - all of the transport information and the IP addresses used inside the tunnel are obscured.
Just for the record, definitely worth checking as it is related to this discussion: URL Reviewer: Michael Scharf Review result: Ready with Issues This document has been reviewed as part of the transport area review team's ongoing effort to review key IETF documents. These comments were written primarily for the transport area directors, but are copied to the document's authors and WG to allow them to address any issues raised and also to the IETF discussion list for information. When done at the time of IETF Last Call, the authors should consider this review as part of the last-call comments they receive. Please always CC EMAIL if you reply to or forward this review. This informational document surveys network and transport protocol issues that affect quality of experience for streaming applications, in particular video. The overview covers many topics across different technologies. Readers may only be familiar with a subset of these topics and could therefore learn quite a bit from this kind of tutorial. Nonetheless, it remains somewhat unclear what the actual objective of some of the text is. For some topics, quite a bit of technical background is provided, but the discussion is not really comprehensive. In many sections, the document neither derives technical challenges, nor system/protocol requirements, nor practical usage guidance. It is just a kind of tutorial. Quite a bit of text also presents ongoing IETF work, and an RFC with this scope may thus get outdated soon. Section 6.2 deals with topics owned by TCPM. In similar past documents, I have asked for a TCPM presentation prior to an IETF last call in order to ensure that owners of running code are in the loop. I believe this strategy has worked well in the past. Having said this, as TSV-ART reviewer I don't strongly disagree with a publication. All these issues may just be OK for an informational RFC. Some more specific comments: Section 3.2.1. Recognizing Changes from an Expected Baseline This section apparently assumes relatively static path properties, e.g., fixed network connectivity. It would be more useful to analyze the impact of Wifi and 4G/5G networks in one place. Some of this follows later in sections 3.2.1 and Yet, the split between these three sections is unclear. Having a discussion about today's Internet path characteristics and the dynamics at one place could be more useful. The text could also better distinguish between what matters to endpoints and considerations relevant only for middleboxes (such as passive monitors). Section 3.3 Path Requirements The statement "to find the bandwidth requirements for a router on the delivery path" may assume that the bottleneck on the path will be routers. Yet, on a path the bottlenecks could also be a link layer device (e.g., an Ethernet Switch, a Wifi access point, etc.). RFC 6077 (specifically Section 3.1.3) also explains that issue. A better wording may be "to find the bandwidth requirements for a delivery path". Section 3.6. Unpredictable Usage Profiles / Section 3.7. Extremely Unpredictable Usage Profiles I am not fully convinced by the distinction between "unpredictable" and "extremely unpredictable". To me, these two sections could be merged and maybe also be shortened. More specifically, Section 3.7 lists a lot of statistics from the past. What is IMHO a bit missing in Section 3.7 are actual operational considerations. For instance, are there any lessons learnt for the future? Note that Section 3.6 has a reasonable conclusion at the end. Section 4. Latency Considerations and Section 4.1. Ultra Low-Latency I am surprised by the definition "ultra low-latency (less than 1 second)", as well as some of the other numbers. For other real-time communication use cases, "ultra low-latency" would probably imply a latency requirement of the order of one millisecond. For instance, isochronous traffic for motion control in industrial networks may require a latency of 1 ms, and such "ultra-low latency" requirements are discussed elsewhere, e.g., in the DetNet WG. The terminology in this section should be better explained to readers dealing with networked systems with much harder latency requirements. And a reference should be added for these definitions in the context of video streaming. Section 5.5.2 Head-of-Line Blocking If Head-of-Line Blocking is indeed a relevant operational problem, it would be useful to add a corresponding reference (e.g., with measurements). Section 5.5.3. Wide and Rapid Variation in Path Capacity The statement "As many end devices have moved to wireless connectivity for the final hop (Wi-Fi, 5G, or LTE), new problems in bandwidth detction have emerged from radio interference and signal strength effects." could be moved to earlier parts of the document. Quite a bit of new computers apparently only have Wifi connectivity and no Ethernet port, i.e., Wifi may just be their default. Also, wireless may not only be used on the last hop, for instance in meshed setups. This may make the problem even harder. Section 6. Evolution of Transport Protocols and Transport Protocol Behaviors I really wonder whether UDP vs. TCP vs. QUIC is actually the relevant distinction. What may actually matter are the transport services provided by the protocol stack (e.g., RFC 8095). I fully agree to a related comment in the INTDIR review. Section 6.1. UDP and Its Behavior UDP is also used for encrypted tunnels (OpenVPN, Wireguard, etc.). Does encrypted tunneling really have no operational impact on streaming apps other than circuit breakers? (I am thinking about 4K streaming over OpenVPN and the like.) Section 6.2. TCP and Its Behavior This text should IMHO be presented and be discussed in TCPM. Personally, I am not convinced that this document is a good place for discussing the long history of TCP congestion control. If the objective of the document is to provide a tutorial, IMHO it would be more useful to briefly explain the state-of-the-art in year 2022. Some further specific comments on TCP congestion control: It is surprising that RFC 5681 is not referenced.793bis has normative statements regarding congestion control, and most stacks are compatible to what is written in 793bis. Several operating systems use CUBIC by default and support standard Reno. Another general good reference for TCP standards is RFC 7414.Does ECN, DCTCP, etc. really not matter at all, for instance, if a data center hosts video playout servers? Section 6.3. QUIC and Its Behavior As already noted, the discussion about head-of-line-blocking would really benefit from backing by a reference. Section 7. Streaming Encrypted Media As far as I know, streaming users sometimes use encrypted tunnels such as OpenVPN or WireGuard (or IPsec) to access video content. I may miss something, but it is unclear to me how that fits into the categories presented in Section

Internet Transport Protocols" describes the impact of increased encryption of transport headers in general terms. <ins> It is also worth noting that considerations for heavily-encrypted transport protocols also come into play when streaming media is carried over IP-level VPNs and tunnels, with the additional consideration that an intermediary that does not possess credentials allowing decryption will not have visibility to the source and destination IP addresses of the packets being carried inside the tunnel. </ins> 7.2. Although the IETF has put considerable emphasis on end-to-end


Question:
looks good.
From NAME
NAME and NAME - I'm not sure who wrote this text, except I don't think it was me. >And as with other parts of the ecosystem, new technology brings new challenges. For example, with the emergence of ultra-low-latency streaming, responses have to start streaming to the end user while still being transmitted to the cache, and while the cache does not yet know the size of the object. Some of the popular caching systems were designed around cache footprint and had deeply ingrained assumptions about knowing the size of objects that are being stored, so the change in design requirements in long-established systems caused some errors in production. Incidents occurred where a transmission error in the connection from the upstream source to the cache could result in the cache holding a truncated segment and transmitting it to the end user's device. In this case, players rendering the stream often had the video freeze until the player was reset. In some cases the truncated object was even cached that way and served later to other players as well, causing continued stalls at the same spot in the video for all players playing the segment delivered from that cache node. NAME was curious if there's a reasonable reference that we could provide for this.
Yeah, that was me. There was not a public reference for this, alas, just a word of mouth anecdote.
NAME - I agree with you about your comment on Section 3.6. It happens that I got comments that Section 3.6 and 3.7 didn't need to be nearly as detailed as they were, so they were heavily rewritten. The text where "throttled" was used is now in Section 3.6.1, and the emphasis is now on the cause of unexpected traffic profile changes, rather than the ISP response to the unexpected traffic profile changes in this specific incident, so the text that included "throttled" has been removed.
NAME - I agree with your comment on Section 4.4, that some description or reference should be provided for the word "manifest". The next occurrence of "manifest" is in Section 5.2, which does provide a description. I don't think that's quite right - I'd say So I'm changing this definition, and moving it to the first occurrence of "manifest" in Section 4.4. Thanks for helping us fix problems you didn't even tell us about. :upsidedownface:
lgtm

On-demand media streaming refers to the playback of pre-recorded media based on a user's action.  In some cases, on-demand media is produced as a by-product of a live media production, using the same <del> segments as the live event, but freezing the manifest after the live event has finished.  In other cases, on-demand media is constructed out of pre-recorded assets with no streaming necessarily involved during the production of the on-demand content. </del> <ins> segments as the live event, but freezing the manifest that describes the media available from the media server after the live event has finished.  In other cases, on-demand media is constructed out of pre- recorded assets with no streaming necessarily involved during the production of the on-demand content. </ins> On-demand media generally is not subject to latency concerns, but other timing-related considerations can still be as important or even


Question:
looks good.
From NAME
NAME and NAME - I'm not sure who wrote this text, except I don't think it was me. >And as with other parts of the ecosystem, new technology brings new challenges. For example, with the emergence of ultra-low-latency streaming, responses have to start streaming to the end user while still being transmitted to the cache, and while the cache does not yet know the size of the object. Some of the popular caching systems were designed around cache footprint and had deeply ingrained assumptions about knowing the size of objects that are being stored, so the change in design requirements in long-established systems caused some errors in production. Incidents occurred where a transmission error in the connection from the upstream source to the cache could result in the cache holding a truncated segment and transmitting it to the end user's device. In this case, players rendering the stream often had the video freeze until the player was reset. In some cases the truncated object was even cached that way and served later to other players as well, causing continued stalls at the same spot in the video for all players playing the segment delivered from that cache node. NAME was curious if there's a reasonable reference that we could provide for this.
Yeah, that was me. There was not a public reference for this, alas, just a word of mouth anecdote.
NAME - I agree with you about your comment on Section 3.6. It happens that I got comments that Section 3.6 and 3.7 didn't need to be nearly as detailed as they were, so they were heavily rewritten. The text where "throttled" was used is now in Section 3.6.1, and the emphasis is now on the cause of unexpected traffic profile changes, rather than the ISP response to the unexpected traffic profile changes in this specific incident, so the text that included "throttled" has been removed.
NAME - I agree with your comment on Section 4.4, that some description or reference should be provided for the word "manifest". The next occurrence of "manifest" is in Section 5.2, which does provide a description. I don't think that's quite right - I'd say So I'm changing this definition, and moving it to the first occurrence of "manifest" in Section 4.4. Thanks for helping us fix problems you didn't even tell us about. :upsidedownface:
lgtm

Media servers can provide media streams at various bitrates because the media has been encoded at various bitrates.  This is a so-called "ladder" of bitrates that can be offered to media players as part of <del> the manifest that describes the media being requested by the media player, so that the media player can select among the available </del> <ins> the manifest, so that the media player can select among the available </ins> bitrate choices. The media server may also choose to alter which bitrates are made


Question: Adding Olivier's proposed text about the Fast-Close (cfr., URL) and integrated a shorter version of URL Hello, Sorry for being late with this, but the beginning of the fall semester is always too busy for me. During the summer, there was a discussion on the FASTCLOSE option that was triggered by Francois. The current text of RFC6824bis uses the FASTCLOSE option inside ACK packets, but deployment experience has shown that this results in additional complexity since the ACK must be retransmitted to ensure its reliable delivery. Here is a proposed update for section 3.5 that allows a sender to either send FASTCLOSE inside ACK or RST while a receiver must be ready to process both. Changes are marked with Fast Close Regular TCP has the means of sending a reset (RST) signal to abruptly close a connection. With MPTCP, a reguler RST only has the scope of the subflow and will only close the concerned subflow but not affect the remaining subflows. MPTCP's connection will stay alive at the data level, in order to permit break-before-make handover between subflows. It is therefore necessary to provide an MPTCP-level "reset" to allow the abrupt closure of the whole MPTCP connection, and this is the MPFASTCLOSE option. MPFASTCLOSE is used to indicate to the peer that the connection will be abruptly closed and no data will be accepted anymore. The reasons for triggering an MPFASTCLOSE are implementation specific. Regular TCP does not allow sending a RST while the connection is in a synchronized state [RFC0793]. Nevertheless, implementations allow the sending of a RST in this state, if, for example, the operating system is running out of resources. In these cases, MPTCP should send the MPFASTCLOSE. This option is illustrated in Figure 14. 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +---------------+---------------+-------+-----------------------+ +---------------+---------------+-------+-----------------------+ +---------------------------------------------------------------+ Figure 14: Fast Close (MPFASTCLOSE) Option If Host A wants to force the closure of an MPTCP connection, it has two different options : Option A (ACK) : Host A sends an ACK containing the MPFASTCLOSE option on one subflow, containing the key of Host B as declared in the initial connection handshake. On all the other subflows, Host A sends a regular TCP RST to close these subflows, and tears them down. Host A now enters FASTCLOSEWAIT state. Option R (RST) : Host A sends a RST containing the MPFASTCLOSE option on all subflows, containing the key of Host B as declared in the initial connection handshake. Host A can tear the subflows and the connection down immediately. If a host receives a packet with a valid MPFASTCLOSE option, it shall process it as follows : o Upon receipt of an ACK with MPFASTCLOSE, containing the valid key, Host B answers on the same subflow with a TCP RST and tears down all subflows. Host B can now close the whole MPTCP connection (it transitions directly to CLOSED state). o As soon as Host A has received the TCP RST on the remaining subflow, it can close this subflow and tear down the whole connection (transition from FASTCLOSEWAIT to CLOSED states). If Host A receives an MPFASTCLOSE instead of a TCP RST, both hosts attempted fast closure simultaneously. Host A should reply with a TCP RST and tear down the connection. o If Host A does not receive a TCP RST in reply to its MPFASTCLOSE after one retransmission timeout (RTO) (the RTO of the subflow where the MPTCPRST has been sent), it SHOULD retransmit the MPFASTCLOSE. The number of retransmissions SHOULD be limited to avoid this connection from being retained for a long time, but this limit is implementation specific. A RECOMMENDED number is 3. If no TCP RST is received in response, Host A SHOULD send a TCP RST with the MPFASTCLOSE option itself when it releases state in order to clear any remaining state at middleboxes. o Upon receipt of a RST with MPFASTCLOSE, containing the valid key, Host B tears down all subflows. Host B can now close the whole MPTCP connection (it transitions directly to CLOSED state). The burden of the MP_FASTCLOSE is on the host that decides to close the connection (typically the server, but not necessarily). The text above allows it to opt for either ACK or RST without adding complexity to the other side. Another point that might be discussed in RFC6824bis is how a host should react when there are all the subflows associates to a Multipath TCP connection are in the CLOSED state, but the connection is still alive. At this point, it is impossible to send or receive data over the connection. It should attempt to reestablish one subflow to keep the connection alive. Olivier

receiver of the TCP packet (which can be either host). The initial SYN, containing just the MP_CAPABLE header, is used to <del> define the version of MPTCP beign requested, as well as exchanging </del> <ins> define the version of MPTCP being requested, as well as exchanging </ins> flags to negotiate connection features, described later. This option is used to declare the 64-bit keys that the end hosts


Question: Adding Olivier's proposed text about the Fast-Close (cfr., URL) and integrated a shorter version of URL Hello, Sorry for being late with this, but the beginning of the fall semester is always too busy for me. During the summer, there was a discussion on the FASTCLOSE option that was triggered by Francois. The current text of RFC6824bis uses the FASTCLOSE option inside ACK packets, but deployment experience has shown that this results in additional complexity since the ACK must be retransmitted to ensure its reliable delivery. Here is a proposed update for section 3.5 that allows a sender to either send FASTCLOSE inside ACK or RST while a receiver must be ready to process both. Changes are marked with Fast Close Regular TCP has the means of sending a reset (RST) signal to abruptly close a connection. With MPTCP, a reguler RST only has the scope of the subflow and will only close the concerned subflow but not affect the remaining subflows. MPTCP's connection will stay alive at the data level, in order to permit break-before-make handover between subflows. It is therefore necessary to provide an MPTCP-level "reset" to allow the abrupt closure of the whole MPTCP connection, and this is the MPFASTCLOSE option. MPFASTCLOSE is used to indicate to the peer that the connection will be abruptly closed and no data will be accepted anymore. The reasons for triggering an MPFASTCLOSE are implementation specific. Regular TCP does not allow sending a RST while the connection is in a synchronized state [RFC0793]. Nevertheless, implementations allow the sending of a RST in this state, if, for example, the operating system is running out of resources. In these cases, MPTCP should send the MPFASTCLOSE. This option is illustrated in Figure 14. 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +---------------+---------------+-------+-----------------------+ +---------------+---------------+-------+-----------------------+ +---------------------------------------------------------------+ Figure 14: Fast Close (MPFASTCLOSE) Option If Host A wants to force the closure of an MPTCP connection, it has two different options : Option A (ACK) : Host A sends an ACK containing the MPFASTCLOSE option on one subflow, containing the key of Host B as declared in the initial connection handshake. On all the other subflows, Host A sends a regular TCP RST to close these subflows, and tears them down. Host A now enters FASTCLOSEWAIT state. Option R (RST) : Host A sends a RST containing the MPFASTCLOSE option on all subflows, containing the key of Host B as declared in the initial connection handshake. Host A can tear the subflows and the connection down immediately. If a host receives a packet with a valid MPFASTCLOSE option, it shall process it as follows : o Upon receipt of an ACK with MPFASTCLOSE, containing the valid key, Host B answers on the same subflow with a TCP RST and tears down all subflows. Host B can now close the whole MPTCP connection (it transitions directly to CLOSED state). o As soon as Host A has received the TCP RST on the remaining subflow, it can close this subflow and tear down the whole connection (transition from FASTCLOSEWAIT to CLOSED states). If Host A receives an MPFASTCLOSE instead of a TCP RST, both hosts attempted fast closure simultaneously. Host A should reply with a TCP RST and tear down the connection. o If Host A does not receive a TCP RST in reply to its MPFASTCLOSE after one retransmission timeout (RTO) (the RTO of the subflow where the MPTCPRST has been sent), it SHOULD retransmit the MPFASTCLOSE. The number of retransmissions SHOULD be limited to avoid this connection from being retained for a long time, but this limit is implementation specific. A RECOMMENDED number is 3. If no TCP RST is received in response, Host A SHOULD send a TCP RST with the MPFASTCLOSE option itself when it releases state in order to clear any remaining state at middleboxes. o Upon receipt of a RST with MPFASTCLOSE, containing the valid key, Host B tears down all subflows. Host B can now close the whole MPTCP connection (it transitions directly to CLOSED state). The burden of the MP_FASTCLOSE is on the host that decides to close the connection (typically the server, but not necessarily). The text above allows it to opt for either ACK or RST without adding complexity to the other side. Another point that might be discussed in RFC6824bis is how a host should react when there are all the subflows associates to a Multipath TCP connection are in the CLOSED state, but the connection is still alive. At this point, it is impossible to send or receive data over the connection. It should attempt to reestablish one subflow to keep the connection alive. Olivier

3.5. Regular TCP has the means of sending a reset (RST) signal to abruptly <del> close a connection.  With MPTCP, the RST only has the scope of the subflow and will only close the concerned subflow but not affect the remaining subflows.  MPTCP's connection will stay alive at the data level, in order to permit break-before-make handover between </del> <ins> close a connection.  With MPTCP, a regular RST only has the scope of the subflow and will only close the concerned subflow but not affect the remaining subflows.  MPTCP's connection will stay alive at the data level, in order to permit break-before-make handover between </ins> subflows.  It is therefore necessary to provide an MPTCP-level "reset" to allow the abrupt closure of the whole MPTCP connection, and this is the MP_FASTCLOSE option.


Question: Adding Olivier's proposed text about the Fast-Close (cfr., URL) and integrated a shorter version of URL Hello, Sorry for being late with this, but the beginning of the fall semester is always too busy for me. During the summer, there was a discussion on the FASTCLOSE option that was triggered by Francois. The current text of RFC6824bis uses the FASTCLOSE option inside ACK packets, but deployment experience has shown that this results in additional complexity since the ACK must be retransmitted to ensure its reliable delivery. Here is a proposed update for section 3.5 that allows a sender to either send FASTCLOSE inside ACK or RST while a receiver must be ready to process both. Changes are marked with Fast Close Regular TCP has the means of sending a reset (RST) signal to abruptly close a connection. With MPTCP, a reguler RST only has the scope of the subflow and will only close the concerned subflow but not affect the remaining subflows. MPTCP's connection will stay alive at the data level, in order to permit break-before-make handover between subflows. It is therefore necessary to provide an MPTCP-level "reset" to allow the abrupt closure of the whole MPTCP connection, and this is the MPFASTCLOSE option. MPFASTCLOSE is used to indicate to the peer that the connection will be abruptly closed and no data will be accepted anymore. The reasons for triggering an MPFASTCLOSE are implementation specific. Regular TCP does not allow sending a RST while the connection is in a synchronized state [RFC0793]. Nevertheless, implementations allow the sending of a RST in this state, if, for example, the operating system is running out of resources. In these cases, MPTCP should send the MPFASTCLOSE. This option is illustrated in Figure 14. 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +---------------+---------------+-------+-----------------------+ +---------------+---------------+-------+-----------------------+ +---------------------------------------------------------------+ Figure 14: Fast Close (MPFASTCLOSE) Option If Host A wants to force the closure of an MPTCP connection, it has two different options : Option A (ACK) : Host A sends an ACK containing the MPFASTCLOSE option on one subflow, containing the key of Host B as declared in the initial connection handshake. On all the other subflows, Host A sends a regular TCP RST to close these subflows, and tears them down. Host A now enters FASTCLOSEWAIT state. Option R (RST) : Host A sends a RST containing the MPFASTCLOSE option on all subflows, containing the key of Host B as declared in the initial connection handshake. Host A can tear the subflows and the connection down immediately. If a host receives a packet with a valid MPFASTCLOSE option, it shall process it as follows : o Upon receipt of an ACK with MPFASTCLOSE, containing the valid key, Host B answers on the same subflow with a TCP RST and tears down all subflows. Host B can now close the whole MPTCP connection (it transitions directly to CLOSED state). o As soon as Host A has received the TCP RST on the remaining subflow, it can close this subflow and tear down the whole connection (transition from FASTCLOSEWAIT to CLOSED states). If Host A receives an MPFASTCLOSE instead of a TCP RST, both hosts attempted fast closure simultaneously. Host A should reply with a TCP RST and tear down the connection. o If Host A does not receive a TCP RST in reply to its MPFASTCLOSE after one retransmission timeout (RTO) (the RTO of the subflow where the MPTCPRST has been sent), it SHOULD retransmit the MPFASTCLOSE. The number of retransmissions SHOULD be limited to avoid this connection from being retained for a long time, but this limit is implementation specific. A RECOMMENDED number is 3. If no TCP RST is received in response, Host A SHOULD send a TCP RST with the MPFASTCLOSE option itself when it releases state in order to clear any remaining state at middleboxes. o Upon receipt of a RST with MPFASTCLOSE, containing the valid key, Host B tears down all subflows. Host B can now close the whole MPTCP connection (it transitions directly to CLOSED state). The burden of the MP_FASTCLOSE is on the host that decides to close the connection (typically the server, but not necessarily). The text above allows it to opt for either ACK or RST without adding complexity to the other side. Another point that might be discussed in RFC6824bis is how a host should react when there are all the subflows associates to a Multipath TCP connection are in the CLOSED state, but the connection is still alive. At this point, it is impossible to send or receive data over the connection. It should attempt to reestablish one subflow to keep the connection alive. Olivier

is running out of resources.  In these cases, MPTCP should send the MP_FASTCLOSE.  This option is illustrated in tcpm_fastclose. <del> If Host A wants to force the closure of an MPTCP connection, the MPTCP Fast Close procedure is as follows: </del> <ins> If Host A wants to force the closure of an MPTCP connection, it has two different options: </ins> <del> Host A sends an ACK containing the MP_FASTCLOSE option on one subflow, containing the key of Host B as declared in the initial connection handshake.  On all the other subflows, Host A sends a regular TCP RST to close these subflows, and tears them down. Host A now enters FASTCLOSE_WAIT state. </del> <ins> Option A (ACK) : Host A sends an ACK containing the MP_FASTCLOSE option on one subflow, containing the key of Host B as declared in the initial connection handshake.  On all the other subflows, Host A sends a regular TCP RST to close these subflows, and tears them down.  Host A now enters FASTCLOSE_WAIT state. </ins> <del> Upon receipt of an MP_FASTCLOSE, containing the valid key, Host B answers on the same subflow with a TCP RST and tears down all subflows.  Host B can now close the whole MPTCP connection (it transitions directly to CLOSED state). </del> <ins> Option R (RST) : Host A sends a RST containing the MP_FASTCLOSE option on all subflows, containing the key of Host B as declared in the initial connection handshake.  Host A can tear the subflows and the connection down immediately. If a host receives a packet with a valid MP_FASTCLOSE option, it shall process it as follows : Upon receipt of an ACK with MP_FASTCLOSE, containing the valid key, Host B answers on the same subflow with a TCP RST and tears down all subflows.  Host B can now close the whole MPTCP connection (it transitions directly to CLOSED state). </ins> As soon as Host A has received the TCP RST on the remaining subflow, it can close this subflow and tear down the whole


Question: Adding Olivier's proposed text about the Fast-Close (cfr., URL) and integrated a shorter version of URL Hello, Sorry for being late with this, but the beginning of the fall semester is always too busy for me. During the summer, there was a discussion on the FASTCLOSE option that was triggered by Francois. The current text of RFC6824bis uses the FASTCLOSE option inside ACK packets, but deployment experience has shown that this results in additional complexity since the ACK must be retransmitted to ensure its reliable delivery. Here is a proposed update for section 3.5 that allows a sender to either send FASTCLOSE inside ACK or RST while a receiver must be ready to process both. Changes are marked with Fast Close Regular TCP has the means of sending a reset (RST) signal to abruptly close a connection. With MPTCP, a reguler RST only has the scope of the subflow and will only close the concerned subflow but not affect the remaining subflows. MPTCP's connection will stay alive at the data level, in order to permit break-before-make handover between subflows. It is therefore necessary to provide an MPTCP-level "reset" to allow the abrupt closure of the whole MPTCP connection, and this is the MPFASTCLOSE option. MPFASTCLOSE is used to indicate to the peer that the connection will be abruptly closed and no data will be accepted anymore. The reasons for triggering an MPFASTCLOSE are implementation specific. Regular TCP does not allow sending a RST while the connection is in a synchronized state [RFC0793]. Nevertheless, implementations allow the sending of a RST in this state, if, for example, the operating system is running out of resources. In these cases, MPTCP should send the MPFASTCLOSE. This option is illustrated in Figure 14. 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +---------------+---------------+-------+-----------------------+ +---------------+---------------+-------+-----------------------+ +---------------------------------------------------------------+ Figure 14: Fast Close (MPFASTCLOSE) Option If Host A wants to force the closure of an MPTCP connection, it has two different options : Option A (ACK) : Host A sends an ACK containing the MPFASTCLOSE option on one subflow, containing the key of Host B as declared in the initial connection handshake. On all the other subflows, Host A sends a regular TCP RST to close these subflows, and tears them down. Host A now enters FASTCLOSEWAIT state. Option R (RST) : Host A sends a RST containing the MPFASTCLOSE option on all subflows, containing the key of Host B as declared in the initial connection handshake. Host A can tear the subflows and the connection down immediately. If a host receives a packet with a valid MPFASTCLOSE option, it shall process it as follows : o Upon receipt of an ACK with MPFASTCLOSE, containing the valid key, Host B answers on the same subflow with a TCP RST and tears down all subflows. Host B can now close the whole MPTCP connection (it transitions directly to CLOSED state). o As soon as Host A has received the TCP RST on the remaining subflow, it can close this subflow and tear down the whole connection (transition from FASTCLOSEWAIT to CLOSED states). If Host A receives an MPFASTCLOSE instead of a TCP RST, both hosts attempted fast closure simultaneously. Host A should reply with a TCP RST and tear down the connection. o If Host A does not receive a TCP RST in reply to its MPFASTCLOSE after one retransmission timeout (RTO) (the RTO of the subflow where the MPTCPRST has been sent), it SHOULD retransmit the MPFASTCLOSE. The number of retransmissions SHOULD be limited to avoid this connection from being retained for a long time, but this limit is implementation specific. A RECOMMENDED number is 3. If no TCP RST is received in response, Host A SHOULD send a TCP RST with the MPFASTCLOSE option itself when it releases state in order to clear any remaining state at middleboxes. o Upon receipt of a RST with MPFASTCLOSE, containing the valid key, Host B tears down all subflows. Host B can now close the whole MPTCP connection (it transitions directly to CLOSED state). The burden of the MP_FASTCLOSE is on the host that decides to close the connection (typically the server, but not necessarily). The text above allows it to opt for either ACK or RST without adding complexity to the other side. Another point that might be discussed in RFC6824bis is how a host should react when there are all the subflows associates to a Multipath TCP connection are in the CLOSED state, but the connection is still alive. At this point, it is impossible to send or receive data over the connection. It should attempt to reestablish one subflow to keep the connection alive. Olivier

If Host A does not receive a TCP RST in reply to its MP_FASTCLOSE after one retransmission timeout (RTO) (the RTO of the subflow <del> where the MPTCP_RST has been sent), it SHOULD retransmit the </del> <ins> where the MP_FASTCLOSE has been sent), it SHOULD retransmit the </ins> MP_FASTCLOSE.  The number of retransmissions SHOULD be limited to avoid this connection from being retained for a long time, but this limit is implementation specific.  A RECOMMENDED number is 3. If no TCP RST is received in response, Host A SHOULD send a TCP <del> RST itself when it releases state in order to clear any remaining state at middleboxes. </del> <ins> RST with the MP_FASTCLOSE option itself when it releases state in order to clear any remaining state at middleboxes. Upon receipt of a RST with MP_FASTCLOSE, containing the valid key, Host B tears down all subflows.  Host B can now close the whole MPTCP connection (it transitions directly to CLOSED state). </ins> 3.6.


Question: Adding Olivier's proposed text about the Fast-Close (cfr., URL) and integrated a shorter version of URL Hello, Sorry for being late with this, but the beginning of the fall semester is always too busy for me. During the summer, there was a discussion on the FASTCLOSE option that was triggered by Francois. The current text of RFC6824bis uses the FASTCLOSE option inside ACK packets, but deployment experience has shown that this results in additional complexity since the ACK must be retransmitted to ensure its reliable delivery. Here is a proposed update for section 3.5 that allows a sender to either send FASTCLOSE inside ACK or RST while a receiver must be ready to process both. Changes are marked with Fast Close Regular TCP has the means of sending a reset (RST) signal to abruptly close a connection. With MPTCP, a reguler RST only has the scope of the subflow and will only close the concerned subflow but not affect the remaining subflows. MPTCP's connection will stay alive at the data level, in order to permit break-before-make handover between subflows. It is therefore necessary to provide an MPTCP-level "reset" to allow the abrupt closure of the whole MPTCP connection, and this is the MPFASTCLOSE option. MPFASTCLOSE is used to indicate to the peer that the connection will be abruptly closed and no data will be accepted anymore. The reasons for triggering an MPFASTCLOSE are implementation specific. Regular TCP does not allow sending a RST while the connection is in a synchronized state [RFC0793]. Nevertheless, implementations allow the sending of a RST in this state, if, for example, the operating system is running out of resources. In these cases, MPTCP should send the MPFASTCLOSE. This option is illustrated in Figure 14. 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +---------------+---------------+-------+-----------------------+ +---------------+---------------+-------+-----------------------+ +---------------------------------------------------------------+ Figure 14: Fast Close (MPFASTCLOSE) Option If Host A wants to force the closure of an MPTCP connection, it has two different options : Option A (ACK) : Host A sends an ACK containing the MPFASTCLOSE option on one subflow, containing the key of Host B as declared in the initial connection handshake. On all the other subflows, Host A sends a regular TCP RST to close these subflows, and tears them down. Host A now enters FASTCLOSEWAIT state. Option R (RST) : Host A sends a RST containing the MPFASTCLOSE option on all subflows, containing the key of Host B as declared in the initial connection handshake. Host A can tear the subflows and the connection down immediately. If a host receives a packet with a valid MPFASTCLOSE option, it shall process it as follows : o Upon receipt of an ACK with MPFASTCLOSE, containing the valid key, Host B answers on the same subflow with a TCP RST and tears down all subflows. Host B can now close the whole MPTCP connection (it transitions directly to CLOSED state). o As soon as Host A has received the TCP RST on the remaining subflow, it can close this subflow and tear down the whole connection (transition from FASTCLOSEWAIT to CLOSED states). If Host A receives an MPFASTCLOSE instead of a TCP RST, both hosts attempted fast closure simultaneously. Host A should reply with a TCP RST and tear down the connection. o If Host A does not receive a TCP RST in reply to its MPFASTCLOSE after one retransmission timeout (RTO) (the RTO of the subflow where the MPTCPRST has been sent), it SHOULD retransmit the MPFASTCLOSE. The number of retransmissions SHOULD be limited to avoid this connection from being retained for a long time, but this limit is implementation specific. A RECOMMENDED number is 3. If no TCP RST is received in response, Host A SHOULD send a TCP RST with the MPFASTCLOSE option itself when it releases state in order to clear any remaining state at middleboxes. o Upon receipt of a RST with MPFASTCLOSE, containing the valid key, Host B tears down all subflows. Host B can now close the whole MPTCP connection (it transitions directly to CLOSED state). The burden of the MP_FASTCLOSE is on the host that decides to close the connection (typically the server, but not necessarily). The text above allows it to opt for either ACK or RST without adding complexity to the other side. Another point that might be discussed in RFC6824bis is how a host should react when there are all the subflows associates to a Multipath TCP connection are in the CLOSED state, but the connection is still alive. At this point, it is impossible to send or receive data over the connection. It should attempt to reestablish one subflow to keep the connection alive. Olivier

performing subflows or subflows that regularly fail during use, in order to temporarily choose not to use these paths. <ins> 3.11. TCP Fast Open, described in RFC7413, has been introduced with the objective of gaining one RTT before transmitting data.  This is considered a valuable gain as very short connections are very common, especially for HTTP request/response schemes.  It achieves this by sending the SYN-segment together with data and allowing the server to reply immediately with data after the SYN/ACK.  RFC7413 secures this mechanism, by using a new TCP option that includes a cookie which is negotiated in a preceding connection. When using TCP Fast Open in conjunction with MPTCP, there are two key points to take into account, detailed hereafter. 3.11.1. When a TFO client first connects to a server, it cannot immediately include data in the SYN for security reasons RFC7413.  Instead, it requests a cookie that will be used in subsequent connections.  This is done with the TCP cookie request/response options, of resp. 2 bytes and 6-18 bytes (depending on the chosen cookie length). TFO and MPTCP can be combined provided that the total length of their options does not exceed the maximum 40 bytes possible in TCP: In the SYN: MPTCP uses a 4-bytes long MP_CAPABLE option.  The MPTCP and TFO options sum up to 6 bytes.  With typical TCP-options using up to 19 bytes in the SYN (24 bytes if options are padded at a word boundary), there is enough space to combine the MP_CAPABLE with the TFO Cookie Request. In the SYN+ACK: MPTCP uses a 12-bytes long MP_CAPABLE option, but now TFO can be as long as 18 bytes.  Since the maximum option length may be exceeded, it is up to the server to solve this by using a shorter cookie.  As an example, if we consider that 19 bytes are used for classical TCP options, the maximum possible cookie length would be of 7 bytes.  Note that the same limitation applies to subsequent connections, for the SYN packet (because the client then echoes back the cookie to the server).  Finally, if the security impact of reducing the cookie size is not deemed acceptable, the server can reduce the amount of other TCP-options by omitting the TCP timestamps (as outlined in app_options). 3.11.2. MPTCP uses, in the TCP establishment phase, a key exchange that is used to generate the Initial Data Sequence Numbers (IDSNs).  In particular, the SYN with MP_CAPABLE occupies the first octet of the data sequence space.  With TFO, one way to handle the data sent together with the SYN would be to consider an implicit DSS mapping that covers that SYN segment (since there is not enough space in the SYN to include a DSS option).  The problem with that approach is that if a middlebox modifies the TFO data, this will not be noticed by MPTCP because of the absence of a DSS-checksum.  For example, a TCP (but not MPTCP)-aware middlebox could insert bytes at the beginning of the stream and adapt the TCP checksum and sequence numbers accordingly.  With an implicit mapping, this would give to client and server a different view on the DSS-mapping, with no way to detect this inconsistency as the DSS checksum is not present. To solve this, the TFO data should not be considered part of the Data Sequence Number space: the SYN with MP_CAPABLE still occupies the first octet of data sequence space, but then the first non-TFO data byte occupies the second octet.  This guarantees that, if the use of DSS-checksum is negotiated, all data in the data sequence number space is checksummed.  We also note that this does not entail a loss of functionality, because TFO-data is always sent when only one path is active. 3.11.3. The following shows a few examples of possible TFO+MPTCP establishment scenarios. Before a client can send data together with the SYN, it must request a cookie to the server, as shown in Figure fig_tfocookie.  This is done by simply combining the TFO and MPTCP options. Once this is done, the received cookie can be used for TFO, as shown in Figure fig_tfodata.  In this example, the client first sends 20 bytes in the SYN.  The server immediately replies with 100 bytes following the SYN-ACK upon which the client replies with 20 more bytes.  Note that the last segment in the figure has a TCP sequence number of 21, while the DSS subflow sequence number is 1 (because the TFO data is not part of the data sequence number space, as explained in Section tfodata. In Figure fig_tfofallback, the server does not support TFO.  The client detects that no state is created in the server (as no data is acked), and now sends the MP_CAPABLE in the third ack, in order for the server to build its MPTCP context at then end of the establishment.  Now, the tfo data, retransmitted, becomes part of the data sequence mapping because it is effectively sent (in fact re- sent) after the establishment. It is also possible that the server acknowledges only part of the TFO data, as illustrated in Figure fig_tfopartial.  The client will simply retransmit the missing data together with a DSS-mapping. </ins> 4. In order to support multipath operation, the semantics of some TCP


Question:
All protocol request messages contain a field, which allows aggregators to service multiple tasks from a single set of endpoints. The exception is : since it is an HTTP GET request, there is no body, and thus no place for the client to indicate the desired to the aggregator. We don't want to introduce a request body to , because then it would have to be a POST instead of a GET, and that would make it harder for implementations to cache HPKE config values. So instead we have clients encode the task ID in RFC 4648 Base16 (a.k.a. hexadecimal) and provide it as a query parameter. See issue
My goal here is to enable our aggregator implementations in the interop target to use one HPKE config per task. The alternative to this would be for us to use aggregator endpoints that somehow incorporate the task ID, as I described , which is already permitted by the spec as written.
The Base64 alphabet includes , and , which means you have to URL encode (a.k.a. percent encode) Base64 strings before you can use them in a URL, which is a chore and yields less readable URLs (not that 64 character hex strings is especially user-friendly). An alternative would be to use RFC 4648's , omitting the padding characters as suggested in since task IDs are of a known length.
Gotcha. I think the main thing is to be consistent. I would either: Change this PR to use base64url; or Tack on a commit that makes all encodings of the taskID (except those that appear in the body, of course) the same (i.e., hexadecimal). I think (2.) would be my preference.
I agree with the consistency argument. I'm leaning towards using everywhere because it's more compact than Base16/hex, and also I think implementations are less likely to be surprised by case requirements in base64url (i.e., it's obvious that base64 is case sensitive, but you have to read RFC 4648 attentively to notice that it mandates uppercase hex strings).
Base64Url sounds good to me. I took a quick look and it appears that most standard base64 libraries also support the URL encoding.
I will clean this up and merge it once I have implemented it in Janus and convinced myself it's sound.
SGTM, but why not base64 encode the taskID for consistency with URL

ppm-urn-space).  Clients SHOULD display the "detail" field of all errors.  The "instance" value MUST be the endpoint to which the request was targeted.  The problem document MUST also include a <del> "taskid" member which contains the associated PPM task ID, encoded with base64 using the standard alphabet RFC4648 (this value is always known, see task-configuration). </del> <ins> "taskid" member which contains the associated PPM task ID, encoded in Base 64 using the URL and filename safe alphabet with no padding as defined in sections 5 and 3.2 of RFC4648 (this value is always known, see task-configuration). </ins> In the remainder of this document, we use the tokens in the table above to refer to error types, rather than the full URNs.  For


Question:
All protocol request messages contain a field, which allows aggregators to service multiple tasks from a single set of endpoints. The exception is : since it is an HTTP GET request, there is no body, and thus no place for the client to indicate the desired to the aggregator. We don't want to introduce a request body to , because then it would have to be a POST instead of a GET, and that would make it harder for implementations to cache HPKE config values. So instead we have clients encode the task ID in RFC 4648 Base16 (a.k.a. hexadecimal) and provide it as a query parameter. See issue
My goal here is to enable our aggregator implementations in the interop target to use one HPKE config per task. The alternative to this would be for us to use aggregator endpoints that somehow incorporate the task ID, as I described , which is already permitted by the spec as written.
The Base64 alphabet includes , and , which means you have to URL encode (a.k.a. percent encode) Base64 strings before you can use them in a URL, which is a chore and yields less readable URLs (not that 64 character hex strings is especially user-friendly). An alternative would be to use RFC 4648's , omitting the padding characters as suggested in since task IDs are of a known length.
Gotcha. I think the main thing is to be consistent. I would either: Change this PR to use base64url; or Tack on a commit that makes all encodings of the taskID (except those that appear in the body, of course) the same (i.e., hexadecimal). I think (2.) would be my preference.
I agree with the consistency argument. I'm leaning towards using everywhere because it's more compact than Base16/hex, and also I think implementations are less likely to be surprised by case requirements in base64url (i.e., it's obvious that base64 is case sensitive, but you have to read RFC 4648 attentively to notice that it mandates uppercase hex strings).
Base64Url sounds good to me. I took a quick look and it appears that most standard base64 libraries also support the URL encoding.
I will clean this up and merge it once I have implemented it in Janus and convinced myself it's sound.
SGTM, but why not base64 encode the taskID for consistency with URL

MUST be in the same order.  The leader's endpoint MUST be the first in the list.  The order of the "encrypted_input_shares" in a "Report" (see uploading-reports) MUST be the same as the order in <del> which aggregators appear in this list. </del> <ins> which aggregators appear in this list.  Aggregators MAY use the same endpoint for multiple tasks. </ins> "max_batch_lifetime": The maximum number of times a batch of reports may be used in collect requests.