-
Notifications
You must be signed in to change notification settings - Fork 2
/
draft-ietf-opsawg-ipfix-on-path-telemetry-01.xml
1124 lines (848 loc) · 41.9 KB
/
draft-ietf-opsawg-ipfix-on-path-telemetry-01.xml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<?xml version="1.0" encoding="US-ASCII"?>
<!DOCTYPE rfc SYSTEM "rfc2629.dtd">
<?rfc toc="yes"?>
<?rfc tocompact="yes"?>
<?rfc tocdepth="3"?>
<?rfc tocindent="yes"?>
<?rfc symrefs="yes"?>
<?rfc sortrefs="yes"?>
<?rfc comments="yes"?>
<?rfc inline="yes"?>
<?rfc compact="yes"?>
<?rfc subcompact="no"?>
<rfc category="std" docName="draft-ietf-opsawg-ipfix-on-path-telemetry-01"
ipr="trust200902">
<front>
<title abbrev="Export of On-Path Delay in IPFIX">Export of On-Path Delay
in IPFIX</title>
<author fullname="Thomas Graf" initials="T" surname="Graf">
<organization>Swisscom</organization>
<address>
<postal>
<street>Binzring 17</street>
<city>Zurich</city>
<code>8045</code>
<country>Switzerland</country>
</postal>
<email>[email protected]</email>
</address>
</author>
<author fullname="Benoit Claise" initials="B" surname="Claise">
<organization>Huawei</organization>
<address>
<email>[email protected]</email>
</address>
</author>
<author fullname="Alex Huang Feng" initials="A." surname="Huang Feng">
<organization>INSA-Lyon</organization>
<address>
<postal>
<street/>
<city>Lyon</city>
<region/>
<code/>
<country>France</country>
</postal>
<phone/>
<facsimile/>
<email>[email protected]</email>
<uri/>
</address>
</author>
<date day="6" month="February" year="2023"/>
<abstract>
<t>This document introduces new IP Flow Information Export (IPFIX)
information elements to expose the On-Path Telemetry measured delay on
the IOAM transit and decapsulation nodes.</t>
</abstract>
</front>
<middle>
<section anchor="Introduction" title="Introduction">
<t>Network operators want a statistical delay view of their networks.
They want to understand where in the network, for which customer
traffic, how much and why delay is being accummlated. In order to answer
why and where, delay needs to be reported into device and control-plane
context. In order to understand which customer traffic is affected,
delay needs to be reported into customer data-plane context. That
enables network operators to quickly identify when the control-plane
updates the current path with a different next-hop and therefore the
forwarding path changes to different nodes and interfaces, how the path
delay changes for which customer traffic.</t>
<t>With On-Path Telemetry, described in the <xref
target="RFC9232">Network Telemetry Framework</xref> and applied in <xref
target="I-D.ietf-ippm-ioam-deployment">In-situ OAM</xref>, <xref
target="I-D.filsfils-spring-path-tracing">Path Tracing</xref> and <xref
target="I-D.song-opsawg-ifit-framework">In-situ Flow Information
Telemetry</xref>, the path delay between two endpoints is measured by
inserting a timestamp in the packet.</t>
<t>On-Path Telemetry can be distinguished between two modes. Passport
mode, <xref target="RFC9197"/>, where only the last hop in the
forwarding path of the On-Path Telemetry domain exposes all the metrics,
and postcard mode, <xref
target="I-D.song-ippm-postcard-based-telemetry"/>, where the metrics are
also exposed in the transit nodes. In both modes the forwarding path
exposes performance metrics allowing to determine how much delay has
been accumulated on which hop.</t>
<t>This document defines four new IPFIX Information Elements (IEs),
exposing the On-Path delay on IOAM transit and decapsulation nodes,
following the postcard mode principles. Since these IPFIX IEs are
performance metrics <xref target="RFC8911"/>, they must be registered in
the <xref target="IANA-PERF-METRIC">"IANA Performance Metric
Registry</xref>.</t>
<t>Following the guidelines for Registered Performance Metric requesters
and reviewers <xref target="RFC8911"/>, the different characteristics of
the performance metrics (Identifier, Name, URI, Status, Requester,
Revision, Revision Date, Description, etc) must be clearly specified in
the <xref target="IANA-PERF-METRIC">"IANA Performance Metric
Registry</xref> in order for the results of measurements using the
Performance Metrics to be comparable even if they are performed by
different implementations and in different networks. These
characteristics start by selecting a meaningful name, following the
"MetricType_Method_SubTypeMethod_... Spec_Units_Output" naming
convention (See Section 7.1.2 of <xref target="RFC8911"/>).</t>
<t><figure>
<artwork><![CDATA[
+------------------------------------+-------------------------------+
| Performance Metric | IPFIX Information Element |
+------------------------------------+-------------------------------+
|OWDelay_HybridType1_Passive_I |PathDelayMeanDeltaMicroseconds |
|P_RFC[RFC-to-be]_Seconds_Mean (TBD1)|(TBD5) |
+------------------------------------+-------------------------------+
|OWDelay_HybridType1_Passive_I |PathDelayMinDeltaMicroseconds |
|P_RFC[RFC-to-be]_Seconds_Min (TBD2) |(TBD6) |
+------------------------------------+-------------------------------+
|OWDelay_HybridType1_Passive_I |PathDelayMaxDeltaMicroseconds |
|P_RFC[RFC-to-be]_Seconds_Max (TBD3) |(TBD7) |
+------------------------------------+-------------------------------+
|OWDelay_HybridType1_Passive_I |PathDelaySumDeltaMicroseconds |
|P_RFC[RFC-to-be]_Seconds_Sum (TBD4) |(TBD8) |
+------------------------------------+-------------------------------+
Table 1: Correspondance between IPFIX IE and Performance Metric
]]></artwork>
</figure></t>
<t>The delay is measured by calculating the difference between the
timestamp imposed with On-Path Telemetry in the packet at the IOAM
encapsulation node and the timestamp exported in the IPFIX flow record
from the IOAM transit and decapsulation nodes. The lowest, highest,
mean, and/or the sum of measured path delay can be exported, thanks to
the different IPFIX IE specifications.</t>
<figure anchor="topology"
title="Delay use case. Packets flow from host 1 to host 2.">
<artwork align="center"><![CDATA[
On-Path Telemetry Domain
.........................................
. .
. D1 .
. <------> .
. .
. D2 .
. <--------------------> .
. .
. D3 .
. <-----------------------------------> .
. .
(H1) ------ (R1) ------- (R2) ------- (R3) -------- (R4) ------ (H2)
Host 1 Encapsulation Transit Transit Decapsulation Host 2
Node Node 1 Node 2 Node
. .
. .
.........................................
]]></artwork>
</figure>
<t>On the usecase showed in <xref target="topology"/> using On-path
Telemetry to export the delay metrics, the node R2 exports the delay D1,
the node R3 exports the delay D2 and the decapsulation node R4 exports
the total delay D3 using IPFIX.</t>
</section>
<section anchor="notation" title="Terminology">
<t>The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP 14
<xref target="RFC2119"/> <xref target="RFC8174"/> when, and only when,
they appear in all capitals, as shown here.</t>
<t>This document makes use of the terms defined in <xref
target="RFC7011"/> and <xref
target="I-D.ietf-ippm-ioam-deployment"/>.</t>
<t>The following terms are used as defined in <xref
target="RFC7011"/>.</t>
<t><list style="symbols">
<t>IPFIX</t>
<t>IPFIX Information Elements (IEs)</t>
<t>Flow Record</t>
<t>Exporter</t>
</list></t>
<t>The following terms are used as defined in <xref
target="RFC8911"/>.</t>
<t><list style="symbols">
<t>Performance Metric</t>
<t>Registered Performance Metric</t>
<t>Performance Metrics Registry</t>
</list></t>
<t>The following terms are used as defined in <xref
target="I-D.ietf-ippm-ioam-deployment"/>.</t>
<t><list style="symbols">
<t>IOAM encapsulation node</t>
<t>IOAM transit node</t>
<t>IOAM decapsulation node</t>
</list></t>
</section>
<section anchor="PM" title="Performance Metrics">
<t>This section defines and describes the new performance metrics by
applying the template defined in Section 11 of <xref
target="RFC8911"/>.</t>
<section anchor="ip-ow-delay-hybridtype1-passive-reg-entries"
title="IP One-Way Delay Hybrid Type I Passive Performance Metrics">
<t>This section specifies four performance metrics for the Hybrid Type
I Passive assessment of IP One-Way Delay, to be registered in the
<xref target="IANA-PERF-METRIC">"IANA Performance Metric
Registry</xref>.</t>
<t>All column entries besides the ID, Name, Description, and Output
Reference Method categories are the same; thus, this section defines
four closely related performance metrics. As a result, IANA has
assigned corresponding URLs to each of the four registered performance
metrics.</t>
<section numbered="true" title="Summary">
<t>This category includes multiple indexes of the registered
performance metrics: the element ID and Metric Name.</t>
<section title="ID (Identifier)">
<t><insert a numeric Identifier, an integer, TBD></t>
</section>
<section title="Name">
<t>IANA has allocated the numeric Identifiers TBD1-4 for the four
Named Metric Entries in this section</t>
</section>
<section title="Name">
<t>TBD1: OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Mean</t>
<t>TBD2: OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Min</t>
<t>TBD3: OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Max</t>
<t>TBD4: OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Sum</t>
</section>
<section title="URI">
<t>URL: <eref
target="https://www.iana.org/assignments/performance-metrics/OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Mean"/></t>
<t>URL: <eref
target="https://www.iana.org/assignments/performance-metrics/OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Min"/></t>
<t>URL: <eref
target="https://www.iana.org/assignments/performance-metrics/OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Max"/></t>
<t>URL: <eref
target="https://www.iana.org/assignments/performance-metrics/OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Sum"/></t>
</section>
</section>
<section title="Description">
<t>This metric assesses the one-way delay of IP packets constituting
a single connection between two hosts. We consider the measurement
of one-way delay based on a single Observation Point (OP) <xref
target="RFC7011"/> somewhere in the network. The output is the
one-way delay for all successfully forwarded packets expressed as
the <statistic> of their conditional delay distribution, where
<statistic> is one of:</t>
<t><list style="symbols">
<t>Mean</t>
<t>Min</t>
<t>Max</t>
<t>Sum</t>
</list></t>
</section>
<section title="Change Controller">
<t>IETF</t>
</section>
<section title="Version of Registry Format">
<t>1.0</t>
</section>
</section>
<section title="Metric Definition">
<t>This category includes columns to prompt the entry of all necessary
details related to the metric definition, including the immutable
document reference and values of input factors, called "Fixed
Parameters".</t>
<section title="Reference Definition">
<t>Almes, G., Kalidindi, S., Zekauskas, M., and A. Morton, Ed., "A
One-Way Delay Metric for IP Performance Metrics (IPPM)", STD 81, RFC
7679, DOI 10.17487/RFC7679, January 2016,
<https://www.rfc-editor.org/info/rfc7679>. <xref
target="RFC7679"/></t>
<t>Morton, A. and E. Stephan, "Spatial Composition of Metrics", RFC
6049, DOI 10.17487/RFC6049, January 2011,
<https://www.rfc-editor.org/info/rfc6049>. <xref
target="RFC6049"/></t>
<t>Section 3.4 of <xref target="RFC7679"/> provides the reference
definition of the singleton (single value) one-way delay metric.
Section 4.4 of <xref target="RFC7679"/> provides the reference
definition expanded to cover a multi-value sample. Note that terms
such as "singleton" and "sample" are defined in section 2 of <xref
target="RFC2330"/>.</t>
<t>With the OP <xref target="RFC7011"/> typically located between
the hosts participating in the IP connection, the one-way delay
metric requires one individual measurement between the OP and
sourcing host, such that the Spatial Composition <xref
target="RFC6049"/> of the measurements yields a one-way delay
singleton.</t>
</section>
<section title="Fixed Parameters">
<t><figure>
<preamble>Traffic Filters:</preamble>
<artwork><![CDATA[
IPv4 header values:
DSCP: Set to 0
IPv6 header values:
DSCP: Set to 0
Hop Count: Set to 255
Flow Label: Set to 0
Extension Headers: None
]]></artwork>
</figure></t>
</section>
</section>
<section title="Method of Measurement">
<t>This category includes columns for references to relevant sections
of the RFC(s) and any supplemental information needed to ensure an
unambiguous method for implementations.</t>
<section title="Reference Methods">
<t>The foundational methodology for this metric is defined in
section 4 of <xref target="RFC7323"/> using the Timestamps option
with modifications that allow application at a mid-path OP <xref
target="RFC7011"/>.</t>
</section>
<section title="Packet Stream Generation">
<t>N/A</t>
</section>
<section title="Traffic Filtering (Observation) Details">
<t>The Fixed Parameters above give a portion of the Traffic Filter.
Other aspects will be supplied as Runtime Parameters (below).</t>
</section>
<section title="Sampling Distribution">
<t>This metric requires a partial sample of all packets that qualify
according to the Traffic Filter criteria.</t>
</section>
<section title="Runtime Parameters and Data Format">
<t>Runtime Parameters are input factors that must be determined,
configured into the measurement system, and reported with the
results for the context to be complete.</t>
<t>The hybrid type I metering parameters must must be reported to
provide the complete measurement context. As an example, if the
IPFIX metering process is used, then the IPFIX metering process
parameters (IPFIX template record used, potential traffic filters,
and potential sampling method and parameters) that generates the
flow records must be reported to provide the complete measurement
context.</t>
<t><list style="hanging">
<t hangText="Src:">The IP address of the host in the host A Role
(format ipv4&nbhy;address-no-zone value for IPv4 or
ipv6-address-no-zone value for IPv6; see section 4 of <xref
target="RFC6991"/>.</t>
<t hangText="Dst:">The IP address of the host in the host B Role
(format ipv4&nbhy;address-no-zone value for IPv4 or
ipv6-address-no-zone value for IPv6; see section 4 of <xref
target="RFC6991"/>.</t>
<t hangText="TTL or Hop Limit:">Set at desired value.</t>
<t hangText="DSCP:">Set at desired value.</t>
<t hangText="IPv6 Flow Label:">Set at desired value.</t>
<t hangText="Timestamp:">The timestamp when the packet is being
received at IOAM encapsulation node. Format depends on On-Path
Telemetry implementation. For IOAM, Section 4.4.1 of <xref
target="RFC9197"/> describes what kind of timestamps are
supported. Section 4.4.2.3 and 4.4.2.4 describe where the
timestamp is being inserted. For Path Tracing, Section 4.1 of
<xref target="I-D.filsfils-spring-path-tracing"/> describes what
kind of timestamps are supported. Section 9.2 describe the SRH
path tracing TLV where the timestamp is being inserted.</t>
</list></t>
</section>
<section title="Roles">
<t><list style="hanging">
<t hangText="host A:">Launches the IP packet to open the
connection. The Role of "host A" is synonymous with the IP
address used at host A.</t>
<t hangText="host B:">Receives the IP packet to open the
connection. The Role of "host B" is synonymous with the IP
address used at host B.</t>
<t hangText="Encapsulation Node:">Receives the IP packet to open
the connection and encapsulates the timestamp into the packet.
The Role of "Encapsulation Node" is synonymous with the
timestamp inserted in the packet.</t>
<t hangText="Transit Node:">Receives the IP packet to open the
connection and measures the delay between the timestamp in the
packet and the timestamp when the packet was received.</t>
<t hangText="Decapsulation Node:">Receives the IP packet to open
the connection and measures the delay between the timestamp in
the packet and the timestamp when the packet was received and
removes the IOAM header from the packet.</t>
</list></t>
</section>
</section>
<section title="Output">
<t>This category specifies all details of the output of measurements
using the metric.</t>
<section title="Type">
<t>OWDelay Types are discussed in the subsections below.</t>
</section>
<section title="Reference Definition">
<t>For all output types:</t>
<t><list style="hanging">
<t hangText="OWDelay_HybridType1_Passive_IP:">The one-trip delay
of one IP packet is a Singleton</t>
</list></t>
<t>For each <statistic> Singleton one of the following
subsections applies.</t>
<section title="Mean">
<t>The mean SHALL be calculated using the conditional distribution
of all packets with a finite value of one-way delay (undefined
delays are excluded) -- a single value, as follows:</t>
<t>See section 4.1 of <xref target="RFC3393"/> for details on the
conditional distribution to exclude undefined values of delay, and
see section 5 of <xref target="RFC6703"/> for background on this
analysis choice.</t>
<t>See section 4.2.2 of <xref target="RFC6049"/> for details on
calculating this statistic; see also section 4.2.3 of <xref
target="RFC6049"/>.</t>
<t><list style="hanging">
<t hangText="Mean:">The time value of the result is expressed
in units of seconds, as a positive value of type decimal64
with fraction digits = 9 (see section 9.3 of <xref
target="RFC6020"/>) with a resolution of
0.000000001 seconds (1.0 ns), and with lossless
conversion to/from the 64-bit NTP timestamp as per section 6
of <xref target="RFC5905"/>.</t>
</list></t>
</section>
<section title="Min">
<t>The minimum SHALL be calculated using the conditional
distribution of all packets with a finite value of one-way delay
(undefined delays are excluded) -- a single value, as follows:</t>
<t>See section 4.1 of <xref target="RFC3393"/> for details on the
conditional distribution to exclude undefined values of delay, and
see section 5 of <xref target="RFC6703"/> for background on this
analysis choice.</t>
<t>See section 4.3.2 of <xref target="RFC6049"/> for details on
calculating this statistic; see also section 4.3.3 of <xref
target="RFC6049"/>.</t>
<t><list style="hanging">
<t hangText="Min:">The time value of the result is expressed
in units of seconds, as a positive value of type decimal64
with fraction digits = 9 (see section 9.3 of <xref
target="RFC6020"/>) with a resolution of
0.000000001 seconds (1.0 ns), and with lossless
conversion to/from the 64-bit NTP timestamp as per section 6
of <xref target="RFC5905"/>.</t>
</list></t>
</section>
<section title="Max">
<t>The maximum SHALL be calculated using the conditional
distribution of all packets with a finite value of one-way delay
(undefined delays are excluded) -- a single value, as follows:</t>
<t>See section 4.1 of <xref target="RFC3393"/> for details on the
conditional distribution to exclude undefined values of delay, and
see section 5 of <xref target="RFC6703"/> for background on this
analysis choice.</t>
<t>See section 4.3.2 of <xref target="RFC6049"/> for a closely
related method for calculating this statistic; see also section
4.3.3 of <xref target="RFC6049"/>. The formula is as follows:</t>
<t><figure>
<artwork><![CDATA[
Max = (FiniteDelay[j])
such that for some index, j, where 1 <= j <= N
FiniteDelay[j] >= FiniteDelay[n] for all n
]]></artwork>
</figure></t>
<t>where all packets n = 1 through N have finite singleton
delays.</t>
<t><list style="hanging">
<t hangText="Max:">The time value of the result is expressed
in units of seconds, as a positive value of type decimal64
with fraction digits = 9 (see section 9.3 of <xref
target="RFC6020"/>) with a resolution of
0.000000001 seconds (1.0 ns), and with lossless
conversion to/from the 64-bit NTP timestamp as per section 6
of <xref target="RFC5905"/>.</t>
</list></t>
</section>
<section title="Sum">
<t>The sum SHALL be calculated using the conditional distribution
of all packets with a finite value of one-way delay (undefined
delays are excluded) -- a single value, as follows:</t>
<t>See section 4.1 of <xref target="RFC3393"/> for details on the
conditional distribution to exclude undefined values of delay, and
see section 5 of <xref target="RFC6703"/> for background on this
analysis choice.</t>
<t>See section 4.3.5 of <xref target="RFC6049"/> for details on
calculating this statistic. However in this case FiniteDelay or
MaxDelay MAY be used.</t>
<t><list style="hanging">
<t hangText="Sum:">The time value of the result is expressed
in units of seconds, as a positive value of type decimal64
with fraction digits = 9 (see section 9.3 of <xref
target="RFC6020"/>) with a resolution of
0.000000001 seconds (1.0 ns), and with lossless
conversion to/from the 64-bit NTP timestamp as per section 6
of <xref target="RFC5905"/>.</t>
</list></t>
</section>
<section title="Metric Units">
<t>The <statistic> of one-way delay is expressed in seconds,
where <statistic> is one of:</t>
<t><list style="symbols">
<t>Mean</t>
<t>Min</t>
<t>Max</t>
<t>Sum</t>
</list></t>
<t>The one-way delay of the IP connection singleton is expressed
in seconds.</t>
</section>
<section title="Calibration">
<t>Passive Measurements at an OP could be calibrated against an
Active Measurement at host A where the Active Measurement
represents the ground truth.</t>
</section>
</section>
<section title="Administrative Items">
<section title="Status">
<t>Current</t>
</section>
<section title="Requester">
<t>This RFC</t>
</section>
<section title="Revision">
<t>1.0</t>
</section>
<section title="Revision Date">
<t>RFC Date</t>
</section>
</section>
<section title="Comments and Remarks">
<t>none</t>
</section>
</section>
</section>
<section anchor="IE" title="IPFIX Information Elements">
<t>This section defines and describes the new IPFIX IEs.<list
style="hanging">
<t hangText="PathDelayMeanDeltaMicroseconds"><vspace
blankLines="0"/> 16-bit unsigned integer that identifies the mean
path delay in microseconds, between the IOAM encapsulation node and
the local node with the IOAM domain (either an IOAM transit node or
an IOAM decapsulation node).</t>
<t hangText="PathDelayMinDeltaMicroseconds"><vspace blankLines="0"/>
16-bit unsigned integer that identifies the lowest path delay in
microseconds, between the IOAM encapsulation node and the local node
with the IOAM domain (either an IOAM transit node or an IOAM
decapsulation node).</t>
<t hangText="PathDelayMaxDeltaMicroseconds"><vspace blankLines="0"/>
16-bit unsigned integer that identifies the highest path delay in
microseconds, between the IOAM encapsulation node and the local node
with the IOAM domain (either an IOAM transit node or an IOAM
decapsulation node).</t>
<t hangText="PathDelaySumDeltaMicroseconds"><vspace blankLines="0"/>
32-bit unsigned integer that identifies the sum of the path delay in
microseconds, between the IOAM encapsulation node and the local node
with the IOAM domain (either an IOAM transit node or an IOAM
decapsulation node).</t>
</list></t>
</section>
<section anchor="Use-Cases" title="Use Cases">
<t>The measured On-Path delay can be aggregated with Flow Aggregation as
defined in <xref target="RFC7015"/> to the following device and
control-plane dimensions to determine:</t>
<t><list style="symbols">
<t>With node id and egressInterface(IE14), on which node which
logical egress interfaces have been contributing to how much
delay.</t>
<t>With node id and egressPhysicalInterface(253), on which node
which physical egress interfaces have been contributing to how much
delay.</t>
<t>With ipNextHopIPv4Address(IE15) or ipNextHopIPv6Address(IE62),
the forwarding path to which next-hop IP contributed to how much
delay.</t>
<t>With mplsTopLabelIPv4Address(IE47) or srhActiveSegmentIPv6 from
<xref target="I-D.tgraf-opsawg-ipfix-srv6-srh"/>, the forwarding
path to which MPLS top label IPv4 address or SRv6 active segment
contributed to how much delay.</t>
<t>BGP communities are often used for setting a path priority or
service selection. With bgpDestinationExtendedCommunityList(488) or
bgpDestinationCommunityList(485) or
bgpDestinationLargeCommunityList(491) which group of prefixes
accumulated at which node how much delay.</t>
<t>With destinationIPv4Address(13), destinationTransportPort(11),
protocolIdentifier (4) and sourceIPv4Address(IE8), the forwarding
path delay on each node from each IPv4 source address to a specific
application in the network.</t>
</list></t>
<t>Taking figure 1 from section 1 as topology example. Below example
table shows the aggregated delay per each node, egressInterface and
srhActiveSegmentIPv6.</t>
<t><figure>
<artwork><![CDATA[
+------------+------+-----------------+----------------------+
| Path Delay | Node | egressInterface | srhActiveSegmentIPv6 |
+------------+------+-----------------+----------------------+
| 0 ns | R1 | 276 | 2001:db8::4 |
+------------+------+-----------------+----------------------+
| 3122 ns | R2 | 312 | 2001:db8::4 |
+------------+------+-----------------+----------------------+
| 4432 ns | R3 | 27 | 2001:db8::4 |
+------------+------+-----------------+----------------------+
| 7237 ns | R4 | 854 | 2001:db8::4 |
+------------+------+-----------------+----------------------+
Table 2: Example table of measured delay. Ascending by delay.
]]></artwork>
</figure></t>
</section>
<section anchor="IANA" title="IANA Considerations">
<section anchor="PM-IANA" title="Performance Metrics">
<t>This document requests IANA to create new performance metrics under
the "Performance Metrics" registry <xref target="RFC8911"/> with the
values defined in section 2.</t>
</section>
<section anchor="IE-IANA" title="IPFIX Entities">
<t>This document requests IANA to create new IPFIX IEs (see table 3)
under the "IPFIX Information Elements" registry <xref
target="RFC7012"/> available at <xref target="IANA-PERF-METRIC">"IANA
Performance Metric Registry</xref> and assign the following initial
code points.</t>
<t><figure>
<artwork><![CDATA[
+-------+--------------------------------+
|Element| Name |
| ID | |
+-------+--------------------------------+
| TBD5 | PathDelayMeanDeltaMicroseconds |
| | |
+-------+--------------------------------+
| TBD6 | PathDelayMinDeltaMicroseconds |
| | |
+-------+--------------------------------+
| TBD7 | PathDelayMaxDeltaMicroseconds |
| | |
+-------+--------------------------------+
| TBD8 | PathDelaySumDeltaMicroseconds |
| | |
+-------+--------------------------------+
Table 3: Creates IPFIX IEs in the "IPFIX Information Elements" registry
]]></artwork>
</figure></t>
<t>Note to the RFC-Editor:</t>
<t><list style="symbols">
<t>Please replace TBD5 - TBD8 with the values allocated by
IANA</t>
<t>Please replace the [RFC-to-be] with the RFC number assigned to
this document</t>
</list></t>
<section anchor="IANAPathDelayMeanDeltaMicroseconds"
title="PathDelayMeanDeltaMicroseconds">
<dl>
<dt>Name:</dt>
<dd>PathDelayMeanDeltaMicroseconds</dd>
</dl>
<dl>
<dt>ElementID:</dt>
<dd>TBD5</dd>
</dl>
<dl>
<dt>Description:</dt>
<dd>This Information Element identifies the mean path delay
between the IOAM encapsulation node and the local node with the
IOAM domain (either an IOAM transit node or an IOAM decapsulation
node) in microseconds, according to
OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Mean in the
IANA Performance Metric Registry</dd>
</dl>
<dl>
<dt>Abstract Data Type:</dt>
<dd>unsigned32</dd>
</dl>
<dl>
<dt>Data Type Semantics:</dt>
<dd>deltaCounter</dd>
</dl>
<dl>
<dt>Reference:</dt>
<dd>[RFC-to-be], OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Mean in the IANA Performance Metric Registry.</dd>
</dl>
</section>
<section anchor="IANAPathDelayMinDeltaMicroseconds"
title="PathDelayMinDeltaMicroseconds">
<dl>
<dt>Name:</dt>
<dd>PathDelayMinDeltaMicroseconds</dd>
</dl>
<dl>
<dt>ElementID:</dt>
<dd>TBD6</dd>
</dl>
<dl>
<dt>Description:</dt>
<dd>This Information Element identifies the lowest path delay
between the IOAM encapsulation node and the local node with the
IOAM domain (either an IOAM transit node or an IOAM decapsulation
node) in microseconds, according to the
OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Min
in the IANA Performance Metric Registry.</dd>
</dl>
<dl>
<dt>Abstract Data Type:</dt>
<dd>unsigned32</dd>
</dl>
<dl>
<dt>Data Type Semantics:</dt>
<dd>deltaCounter</dd>
</dl>
<dl>
<dt>Reference:</dt>
<dd>[RFC-to-be], OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Min in the IANA Performance Metric Registry.</dd>
</dl>
</section>
<section anchor="IANAPathDelayMaxDeltaMicroseconds"
title="PathDelayMaxDeltaMicroseconds">
<dl>
<dt>Name:</dt>
<dd>PathDelayMaxDeltaMicroseconds</dd>
</dl>
<dl>
<dt>ElementID:</dt>
<dd>TBD7</dd>
</dl>
<dl>
<dt>Description:</dt>
<dd>This Information Element identifies the highest path delay
between the IOAM encapsulation node and the local node with the
IOAM domain (either an IOAM transit node or an IOAM decapsulation
node) in microseconds, according to
OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Max in the
IANA Performance Metric Registry.</dd>
</dl>
<dl>
<dt>Abstract Data Type:</dt>
<dd>unsigned32</dd>
</dl>
<dl>
<dt>Data Type Semantics:</dt>
<dd>deltaCounter</dd>
</dl>
<dl>
<dt>Reference:</dt>
<dd>[RFC-to-be], OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Max in the IANA Performance Metric Registry.</dd>
</dl>
</section>
<section anchor="IANAPathDelaySumDeltaMicroseconds"
title="PathDelaySumDeltaMicroseconds">
<dl>
<dt>Name:</dt>
<dd>PathDelaySumDeltaMicroseconds</dd>
</dl>
<dl>
<dt>ElementID:</dt>
<dd>TBD8</dd>
</dl>
<dl>
<dt>Description:</dt>
<dd>This Information Element identifies the sum of the path delay
between the IOAM encapsulation node and the local node with the
IOAM domain (either an IOAM transit node or an IOAM decapsulation
node) in microseconds, according to
OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Sum in the
IANA Performance Metric Registry.</dd>
</dl>
<dl>
<dt>Abstract Data Type:</dt>
<dd>unsigned64</dd>
</dl>
<dl>
<dt>Data Type Semantics:</dt>
<dd>deltaCounter</dd>
</dl>
<dl>
<dt>Reference:</dt>
<dd>[RFC-to-be], OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Sum in the IANA Performance Metric Registry.</dd>
</dl>
</section>
</section>
</section>
<section anchor="Operational" title="Operational Considerations">
<section anchor="OpsTimeAccuracy" title="Time Accuracy">
<t>The same recommendation as defined in section 4.5 of <xref
target="RFC5153"/> for IPFIX applies in terms of clock precision to
this document as well.</t>
</section>
<section anchor="OpsMeanDelay" title="Mean Delay">
<t>The mean (average) path delay can be calculated by dividing the
PathDelaySumDeltaMicroseconds(TBD5) by the packetDeltaCount(2) at the
IPFIX data collection in order to offload the IPFIX Exporter from
calculating the mean for every Flow at export time.</t>
</section>
<section anchor="OpsReducedSizeEncoding" title="Reduced-size encoding">
<t>Unsigned64 has been chosen as type for
PathDelaySumDeltaMicroseconds to support cases with large delay
numbers and where many packets are being accounted. As an example, a
specific flow record with path delay of 100 microseconds can not
observe more than 42949 packets without overflowing the unsigned32
counter. The procedure described in Section 6.2 of <xref
target="RFC7011"/> could be applied to reduce network bandwidth
between the IPFIX Exporter and Collector if unsigned32 would be large
enough without wrapping around.</t>
</section>
<section anchor="OpsIoamApplication" title="IOAM Application">
<t>This document is applicable in IOAM to the Edge-to-Edge and Direct
Exporting Option-Type.</t>
<t>In case of Edge-to-Edge Option-Type, as described in Section 4.6 of
<xref target="RFC9197"/>, by setting bits 2 and 3, timestamps can be
encoded as defined in Section 4.4.2.3 and 4.4.2.4 of <xref
target="RFC9197"/>.</t>
<t>In case of Direct Exporting Option-Type, as described in Section 2
of <xref target="I-D.ahuang-ippm-dex-timestamp-ext"/>, by
setting Extension-Flags 2 and 3, timestamps can be encoded as defined
in Section 4.4.2.3 and 4.4.2.4 of <xref target="RFC9197"/>.</t>
<t>For Path Tracing, Section 4.1 of <xref
target="I-D.filsfils-spring-path-tracing"/> describes what kind of
timestamps are supported. Section 9.2 describe the SRH path tracing