[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Strange results using Linux DiffServ



I am trying to set up DiffServ on Linux to provide the EF, AF and BE queues.
I have configured the edge routers to mark the packets for the appropriate
flows which is working fine.  I have checked this with tcpdump all the
proper DS marks are being set for the various flows.

Right now I am doing some test on the throughput for each flow and am
getting some strange results.  The links between the core and edge routers
are wireless connections each with 3Mb/s bandwidth.  I have set up an EF
flow and allocated 1.5Mb/s to it.  This flow is bounded and isolated as is
called for the EF flow.

For the four AF flows, I have allocated 256, 512, 1024, 1536Kb/s to
AF1,2,3,4 respectively all of which are bounded.

For the BE flow I have allocated 1Mb/s which is allowed to borrow any
available bandwidth.

I have setup that by default all traffic will be marked for the BE flow so
that I have to explicitly define the traffic that goes through each of the
EF and AF flows.

I want to set this up so that if there is available bandwidth (up to the
3Mb/s) that the BE traffic can utilize this.  However if traffic goes across
the EF or AF flows, then the BE traffic will be throttled down giving
priority to the EF and AF flows according to their priority.

I have setup a test bed with four machines, the core router, one edge router
and two workstations that will transfer information between each other.
Each are connected via 10Mb/s Ethernet links with all systems running Linux.
I have applied the DS-8 patches to the 2.2.17 kernel on the core and edge
routers.

In testing this I see that the traffic is being routed to the appropriate
queues and that the BE traffic will utilize the unused bandwidth.

The problems that I am seeing is that the EF flow will exceed it's 1.5Mb/s
rate which if I understand this correctly should only be allowed up to
1.5Mb/s.  I see the same thing with the AF flows.

When I test each of the various flows individually, they seem constant.
However if I send traffic over multiple flows the transfer rates for all
flows increases dramatically.  For example, if I send data over the EF flow
I see transfer rates of
around 2.8Mb/s (1.3Mb/s over the bounded rate).  Data over the BE flow has
transfer rates of 3.4Mb/s.

Then when I transfer data over both the EF and BE flows at the same time,
the transfer rates for the EF flow goes up to 5Mb/s and the BE flow is at
3.1Mb/s.  This is really strange sine the rates for traffic over a single
flow (no other traffic over the other flows) are closer to the bounded
rates.  Has anyone seen this happen?

Any insight into what I am doing wrong would or how to configure this so
that the EF and AF flows take priority would be appreciated.  Below is the
core router script I use to setup the appropriate flows.

Also I have included the output for the various queues, classes and filters
from tc after each of the transfers.  I have included output from bwmonitor
for eth0 taken during the transfers.

Thanks in advance.

Core Router Script

#! /bin/sh
#
/usr/src/iproute2/tc/tc qdisc add dev eth0 handle 1:0 root dsmark \
                        indices 64 set_tc_index
/usr/src/iproute2/tc/tc filter add dev eth0 parent 1:0 protocol ip \
                        prio 1 tcindex mask 0xfc shift 2
/usr/src/iproute2/tc/tc qdisc add dev eth0 parent 1:0 handle 2:0 cbq \
                        bandwidth 3Mbit allot 1514 cell 8 \
                        avpkt 1000 mpu 64
#
# EF Class
#
/usr/src/iproute2/tc/tc class add dev eth0 parent 2:0 classid 2:1 cbq \
                        bandwidth 3Mbit rate 1536Kbit avpkt 1000 \
                        prio 1 bounded isolated \
                        allot 1514 weight 1 maxburst 10 defmap 1
/usr/src/iproute2/tc/tc qdisc add dev eth0 parent 2:1 pfifo limit 5
/usr/src/iproute2/tc/tc filter add dev eth0 parent 2:0 protocol ip \
                        prio 1 handle 0x2e tcindex classid 2:1 pass_on

/usr/src/iproute2/tc/tc class add dev eth0 parent 2:0 classid 2:2 cbq \
                        bandwidth 3Mbit rate 1536Kbit avpkt 1000 \
                        prio 2 bounded isolated \
                        allot 1514 weight 1 maxburst 10 defmap 1
/usr/src/iproute2/tc/tc filter add dev eth0 parent 2:0 protocol ip \
                        prio 2 handle 0 tcindex mask 0 classid 2:2
#
# AF and BE
#
/usr/src/iproute2/tc/tc qdisc add dev eth0 parent 2:2 handle 3: cbq \
                        bandwidth 1536Kbit allot 1514 cell 8 avpkt 1000 mpu
64
/usr/src/iproute2/tc/tc filter add dev eth0 parent 3:0 protocol ip \
                        prio 1 tcindex mask 0xf0 shift 4 pass_on
#
# AF 1 Class
#
/usr/src/iproute2/tc/tc class add dev eth0 parent 3:0 classid 3:1 cbq \
                        bandwidth 1536Kbit rate 256Kbit avpkt 1000 \
                        prio 5 bounded allot 1514 weight 1 maxburst 21
/usr/src/iproute2/tc/tc filter add dev eth0 parent 3:0 protocol ip \
                        prio 1 handle 1 tcindex classid 3:1
/usr/src/iproute2/tc/tc qdisc add dev eth0 parent 3:1 gred setup \
                        DPs 3 default 2 grio
#
# AF11
#
/usr/src/iproute2/tc/tc filter add dev eth0 parent 1:0 protocol ip \
                        prio 1 handle 10 tcindex classid 1:111
/usr/src/iproute2/tc/tc qdisc change dev eth0 parent 3:1 gred \
                        limit 60KB min 15KB max 45KB burst 20 \
                        avpkt 1000 bandwidth 256Kbit \
                        DP 1 probability 0.02 prio 2
#
# AF12
#
/usr/src/iproute2/tc/tc filter add dev eth0 parent 1:0 protocol ip \
                        prio 1 handle 12 tcindex classid 1:112
/usr/src/iproute2/tc/tc qdisc change dev eth0 parent 3:1 gred \
                        limit 60KB min 15KB max 45KB burst 20 \
                        avpkt 1000 bandwidth 256Kbit \
                        DP 2 probability 0.04 prio 3
#
# AF13
#
/usr/src/iproute2/tc/tc filter add dev eth0 parent 1:0 protocol ip \
                        prio 1 handle 14 tcindex classid 1:113
/usr/src/iproute2/tc/tc qdisc change dev eth0 parent 3:1 gred \
                        limit 60KB min 15KB max 45KB burst 20 \
                        avpkt 1000 bandwidth 256Kbit \
                        DP 3 probability 0.06 prio 4

#
# AF 2 Class
#
/usr/src/iproute2/tc/tc class add dev eth0 parent 3:0 classid 3:2 cbq \
                        bandwidth 1536Kbit rate 512Kbit avpkt 1000 \
                        prio 4 bounded allot 1514 \
                        weight 1 maxburst 21
/usr/src/iproute2/tc/tc filter add dev eth0 parent 3:0 protocol ip \
                        prio 1 handle 2 tcindex classid 3:2
/usr/src/iproute2/tc/tc qdisc add dev eth0 parent 3:2 gred setup \
                        DPs 3 default 2 grio
#
# AF21
#
/usr/src/iproute2/tc/tc filter add dev eth0 parent 1:0 protocol ip \
                        prio 1 handle 18 tcindex classid 1:121
/usr/src/iproute2/tc/tc qdisc change dev eth0 parent 3:2 gred \
                        limit 60KB min 15KB max 45KB burst 20 \
                        avpkt 1000 bandwidth 512Kbit \
                        DP 1 probability 0.02 prio 2
#
# AF22
#
/usr/src/iproute2/tc/tc filter add dev eth0 parent 1:0 protocol ip \
                        prio 1 handle 20 tcindex classid 1:122
/usr/src/iproute2/tc/tc qdisc change dev eth0 parent 3:2 gred \
                        limit 60KB min 15KB max 45KB burst 20 \
                        avpkt 1000 bandwidth 512Kbit \
                        DP 2 probability 0.04 prio 3
#
# AF23
#
/usr/src/iproute2/tc/tc filter add dev eth0 parent 1:0 protocol ip \
                        prio 1 handle 22 tcindex classid 1:123
/usr/src/iproute2/tc/tc qdisc change dev eth0 parent 3:2 gred \
                        limit 60KB min 15KB max 45KB burst 20 \
                        avpkt 1000 bandwidth 512Kbit \
                        DP 3 probability 0.06 prio 4
#
# AF 3 Class
#
/usr/src/iproute2/tc/tc class add dev eth0 parent 3:0 classid 3:3 cbq \
                        bandwidth 1536Kbit rate 768Kbit avpkt 1000 \
                        prio 3 bounded allot 1514 \
                        weight 1 maxburst 21
/usr/src/iproute2/tc/tc filter add dev eth0 parent 3:0 protocol ip \
                        prio 1 handle 3 tcindex classid 3:3
/usr/src/iproute2/tc/tc qdisc add dev eth0 parent 3:3 gred setup \
                        DPs 3 default 2 grio
#
# AF31
#
/usr/src/iproute2/tc/tc filter add dev eth0 parent 1:0 protocol ip \
                        prio 1 handle 26 tcindex classid 1:131
/usr/src/iproute2/tc/tc qdisc change dev eth0 parent 3:3 gred \
                        limit 60KB min 15KB max 45KB burst 20 \
                        avpkt 1000 bandwidth 768Kbit \
                        DP 1 probability 0.02 prio 2
#
# AF32
#
/usr/src/iproute2/tc/tc filter add dev eth0 parent 1:0 protocol ip \
                        prio 1 handle 28 tcindex classid 1:132
/usr/src/iproute2/tc/tc qdisc change dev eth0 parent 3:3 gred \
                        limit 60KB min 15KB max 45KB burst 20 \
                        avpkt 1000 bandwidth 768Kbit \
                        DP 2 probability 0.04 prio 3
#
# AF33
#
/usr/src/iproute2/tc/tc filter add dev eth0 parent 1:0 protocol ip \
                        prio 1 handle 30 tcindex classid 1:133
/usr/src/iproute2/tc/tc qdisc change dev eth0 parent 3:3 gred \
                        limit 60KB min 15KB max 45KB burst 20 \
                        avpkt 1000 bandwidth 768Kbit \
                        DP 3 probability 0.06 prio 4
#
# AF 4 Class
#
/usr/src/iproute2/tc/tc class add dev eth0 parent 3:0 classid 3:4 cbq \
                        bandwidth 1536Kbit rate 1024Kbit avpkt \
                        1000 prio 2 bounded allot 1514 \
                        weight 1 maxburst 21
/usr/src/iproute2/tc/tc filter add dev eth0 parent 3:0 protocol ip \
                        prio 1 handle 4 tcindex classid 3:4
/usr/src/iproute2/tc/tc qdisc add dev eth0 parent 3:4 gred setup \
                        DPs 3 default 2 grio
#
# AF41
#
/usr/src/iproute2/tc/tc filter add dev eth0 parent 1:0 protocol ip \
                        prio 1 handle 34 tcindex classid 1:141
/usr/src/iproute2/tc/tc qdisc change dev eth0 parent 3:4 gred \
                        limit 60KB min 15KB max 45KB burst 20 \
                        avpkt 1000 bandwidth 1024Kbit \
                        DP 1 probability 0.02 prio 2
/usr/src/iproute2/tc/tc filter add dev eth0 parent 1:0 protocol ip \
                        prio 1 handle 36 tcindex classid 1:142
/usr/src/iproute2/tc/tc qdisc change dev eth0 parent 3:4 gred \
                        limit 60KB min 15KB max 45KB burst 20 \
                        avpkt 1000 bandwidth 1024Kbit \
                        DP 2 probability 0.04 prio 3
/usr/src/iproute2/tc/tc filter add dev eth0 parent 1:0 protocol ip \
                        prio 1 handle 38 tcindex classid 1:143
/usr/src/iproute2/tc/tc qdisc change dev eth0 parent 3:4 gred \
                        limit 60KB min 15KB max 45KB burst 20 \
                        avpkt 1000 bandwidth 1024Kbit \
                        DP 3 probability 0.06 prio 4

# BE
/usr/src/iproute2/tc/tc class add dev eth0 parent 3:0 classid 3:5 cbq \
                        bandwidth 1536Kbit rate 1Mbit avpkt 1000 \
                        prio 7 allot 1514 weight 1 \
                        maxburst 21 borrow
/usr/src/iproute2/tc/tc filter add dev eth0 parent 3:0 protocol ip \
                        prio 1 handle 0 tcindex classid 3:5
/usr/src/iproute2/tc/tc qdisc add dev eth0 parent 3:5 red \
                        limit 60KB min 15KB max 45KB burst 20 \
                        avpkt 1000 bandwidth 1024Kbit \
                        probability 0.4


RESULTS

5MB transfer over the EF flow only


bwmonitor output for eth0

980879258 1592032 67392
980879259 2834840 91128
980879260 2883288 81448
980879261 2859064 84880
980879262 2834840 79304
980879263 2883288 76000
980879264 2846952 80168
980879265 2871176 80128
980879266 2871176 75040
980879267 2737944 71552
980879268 2616824 67056
980879269 2846952 75360
980879270 2871176 77248
980879271 2895400 75520
980879272 2628936 71808
980879273 1708200 49008

Output from TC

 qdisc red 804d: limit 60Kb min 15Kb max 45Kb ewma 3 Plog 17 Scell_log 12
 Sent 7425 bytes 99 pkts (dropped 0, overlimits 0)

 qdisc gred 804c:
 DP:1 (prio 2) Average Queue 0b Measured Queue 0b
         Packet drops: 0 (forced 0 early 0)
         Packet totals: 0 (bytes 0)
 limit 60Kb min 15Kb max 45Kb ewma 3 Plog 21 Scell_log 12
 DP:2 (prio 3) Average Queue 0b Measured Queue 0b
         Packet drops: 0 (forced 0 early 0)
         Packet totals: 0 (bytes 0)
 limit 60Kb min 15Kb max 45Kb ewma 3 Plog 20 Scell_log 12
 DP:3 (prio 4) Average Queue 0b Measured Queue 0b
         Packet drops: 0 (forced 0 early 0)
         Packet totals: 0 (bytes 0)
 limit 60Kb min 15Kb max 45Kb ewma 3 Plog 19 Scell_log 12
 Sent 0 bytes 0 pkts (dropped 0, overlimits 0)

 qdisc gred 804b:
 DP:1 (prio 2) Average Queue 0b Measured Queue 0b
         Packet drops: 0 (forced 0 early 0)
         Packet totals: 0 (bytes 0)
 limit 60Kb min 15Kb max 45Kb ewma 3 Plog 21 Scell_log 12
 DP:2 (prio 3) Average Queue 0b Measured Queue 0b
         Packet drops: 0 (forced 0 early 0)
         Packet totals: 0 (bytes 0)
 limit 60Kb min 15Kb max 45Kb ewma 3 Plog 20 Scell_log 12
 DP:3 (prio 4) Average Queue 0b Measured Queue 0b
         Packet drops: 0 (forced 0 early 0)
         Packet totals: 0 (bytes 0)
 limit 60Kb min 15Kb max 45Kb ewma 3 Plog 19 Scell_log 12
 Sent 0 bytes 0 pkts (dropped 0, overlimits 0)

 qdisc gred 804a:
 DP:1 (prio 2) Average Queue 0b Measured Queue 0b
         Packet drops: 0 (forced 0 early 0)
         Packet totals: 0 (bytes 0)
 limit 60Kb min 15Kb max 45Kb ewma 3 Plog 21 Scell_log 13
 DP:2 (prio 3) Average Queue 0b Measured Queue 0b
         Packet drops: 0 (forced 0 early 0)
         Packet totals: 0 (bytes 0)
 limit 60Kb min 15Kb max 45Kb ewma 3 Plog 20 Scell_log 13
 DP:3 (prio 4) Average Queue 0b Measured Queue 0b
         Packet drops: 0 (forced 0 early 0)
         Packet totals: 0 (bytes 0)
 limit 60Kb min 15Kb max 45Kb ewma 3 Plog 19 Scell_log 13
 Sent 0 bytes 0 pkts (dropped 0, overlimits 0)

 qdisc gred 8049:
 DP:1 (prio 2) Average Queue 0b Measured Queue 0b
         Packet drops: 0 (forced 0 early 0)
         Packet totals: 0 (bytes 0)
 limit 60Kb min 15Kb max 45Kb ewma 3 Plog 21 Scell_log 14
 DP:2 (prio 3) Average Queue 0b Measured Queue 0b
         Packet drops: 0 (forced 0 early 0)
         Packet totals: 0 (bytes 0)
 limit 60Kb min 15Kb max 45Kb ewma 3 Plog 20 Scell_log 14
 DP:3 (prio 4) Average Queue 0b Measured Queue 0b
         Packet drops: 0 (forced 0 early 0)
         Packet totals: 0 (bytes 0)
 limit 60Kb min 15Kb max 45Kb ewma 3 Plog 19 Scell_log 14
 Sent 0 bytes 0 pkts (dropped 0, overlimits 0)

 qdisc cbq 3: rate 1536Kbit cell 8b mpu 64b (bounded,isolated) prio
no-transmit/8 weight 1536Kbit allot 1514b
level 1 ewma 5 avpkt 1000b maxidle 158us
 Sent 7425 bytes 99 pkts (dropped 0, overlimits 0)
  borrowed 0 overactions 0 avgidle 4166 undertime 0

 qdisc pfifo 8048: limit 5p
 Sent 5348203 bytes 3559 pkts (dropped 97, overlimits 0)

 qdisc cbq 2: rate 3Mbit cell 8b mpu 64b (bounded,isolated) prio
no-transmit/8 weight 3Mbit allot 1514b
level 1 ewma 5 avpkt 1000b maxidle 79us
 Sent 5355670 bytes 3659 pkts (dropped 97, overlimits 12972)
  borrowed 0 overactions 0 avgidle 2083 undertime 0

 qdisc dsmark 1: indices 0x0040 set_tc_index
 Sent 5355670 bytes 3659 pkts (dropped 97, overlimits 0)


class cbq 3: root rate 1536Kbit cell 8b mpu 64b (bounded,isolated) prio
no-transmit/8 weight 1536Kbit allot 1514b
level 1 ewma 5 avpkt 1000b maxidle 158us
 Sent 11100 bytes 105 pkts (dropped 0, overlimits 0)
  borrowed 0 overactions 0 avgidle 3788 undertime 0
class cbq 3:1 parent 3: leaf 8049: rate 256Kbit cell 8b (bounded) prio 5/5
allot 1514b
level 0 ewma 5 avpkt 1000b maxidle 24104us
 Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
  borrowed 0 overactions 0 avgidle 631894 undertime 0
class cbq 3:2 parent 3: leaf 804a: rate 512Kbit cell 8b (bounded) prio 4/4
allot 1514b
level 0 ewma 5 avpkt 1000b maxidle 9641us
 Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
  borrowed 0 overactions 0 avgidle 252757 undertime 0
class cbq 3:3 parent 3: leaf 804b: rate 768Kbit cell 8b (bounded) prio 3/3
allot 1514b
level 0 ewma 5 avpkt 1000b maxidle 4820us
 Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
  borrowed 0 overactions 0 avgidle 126378 undertime 0
class cbq 3:4 parent 3: leaf 804c: rate 1Mbit cell 8b (bounded) prio 2/2
allot 1514b
level 0 ewma 5 avpkt 1000b maxidle 2409us
 Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
  borrowed 0 overactions 0 avgidle 63188 undertime 0
class cbq 3:5 parent 3: leaf 804d: rate 1Mbit cell 8b prio 7/7 allot 1514b
level 0 ewma 5 avpkt 1000b maxidle 2409us
 Sent 11100 bytes 105 pkts (dropped 0, overlimits 0)
  borrowed 0 overactions 0 avgidle 51491 undertime 0
class cbq 2: root rate 3Mbit cell 8b mpu 64b (bounded,isolated) prio
no-transmit/8 weight 3Mbit allot 1514b
level 1 ewma 5 avpkt 1000b maxidle 79us
 Sent 42 bytes 1 pkts (dropped 0, overlimits 0)
  borrowed 0 overactions 0 avgidle 2083 undertime 0
class cbq 2:1 parent 2: leaf 8048: rate 1536Kbit cell 8b (bounded,isolated)
prio 1/1 allot 1514b
level 0 ewma 5 avpkt 1000b maxidle 949us
 Sent 5348203 bytes 3559 pkts (dropped 97, overlimits 13649)
  borrowed 0 overactions 1143 avgidle 110 undertime 0
class cbq 2:2 parent 2: leaf 3: rate 1536Kbit cell 8b (bounded,isolated)
prio 2/2 allot 1514b
level 0 ewma 5 avpkt 1000b maxidle 949us
 Sent 11100 bytes 105 pkts (dropped 0, overlimits 0)
  borrowed 0 overactions 0 avgidle 18803 undertime 0

filter parent 1: protocol ip pref 1 tcindex hash 64 mask 0x00fc shift 2
fall_through
filter parent 1: protocol ip pref 1 tcindex handle 0x000a classid 1:111
filter parent 1: protocol ip pref 1 tcindex handle 0x000c classid 1:112
filter parent 1: protocol ip pref 1 tcindex handle 0x000e classid 1:113
filter parent 1: protocol ip pref 1 tcindex handle 0x0012 classid 1:121
filter parent 1: protocol ip pref 1 tcindex handle 0x0014 classid 1:122
filter parent 1: protocol ip pref 1 tcindex handle 0x0016 classid 1:123
filter parent 1: protocol ip pref 1 tcindex handle 0x001a classid 1:131
filter parent 1: protocol ip pref 1 tcindex handle 0x001c classid 1:132
filter parent 1: protocol ip pref 1 tcindex handle 0x001e classid 1:133
filter parent 1: protocol ip pref 1 tcindex handle 0x0022 classid 1:141
filter parent 1: protocol ip pref 1 tcindex handle 0x0024 classid 1:142
filter parent 1: protocol ip pref 1 tcindex handle 0x0026 classid 1:143

Transfer rate achived

EF - 3.2Mb/s


5MB transfer over the AF41 flow only


bwmonitor output for eth0

980879709 160712 6872
980879710 1563064 45864
980879711 1260280 40624
980879712 1393512 39280
980879713 1284504 35936
980879714 1647864 46592
980879715 1526744 48416
980879716 2035448 58032
980879717 1672088 52120
980879718 1659976 46672
980879719 2011224 56352
980879720 2132344 68136
980879721 1296616 35152
980879722 1768984 67072
980879723 1829544 62880
980879724 1805320 65312
980879725 1974888 69840
980879726 1599416 56440
980879727 1550968 55256
980879728 1563080 54040
980879729 1490408 58920
980879730 1332952 36320
980879731 1357176 39712
980879732 1393512 39520
980879733 1490408 42080
980879734 1684200 46512
980879735 1127048 32080
980879736 1393512 40752
980879737 133640 12368

Output from TC

qdisc red 804d: limit 60Kb min 15Kb max 45Kb ewma 3 Plog 17 Scell_log 12
 Sent 21570 bytes 171 pkts (dropped 0, overlimits 0)

 qdisc gred 804c:
 DP:1 (prio 2) Average Queue 0b Measured Queue 0b
         Packet drops: 0 (forced 0 early 0)
         Packet totals: 3575 (bytes 5390829)
 limit 60Kb min 15Kb max 45Kb ewma 3 Plog 21 Scell_log 12
 DP:2 (prio 3) Average Queue 0b Measured Queue 0b
         Packet drops: 0 (forced 0 early 0)
         Packet totals: 0 (bytes 0)
 limit 60Kb min 15Kb max 45Kb ewma 3 Plog 20 Scell_log 12
 DP:3 (prio 4) Average Queue 0b Measured Queue 0b
         Packet drops: 0 (forced 0 early 0)
         Packet totals: 0 (bytes 0)
 limit 60Kb min 15Kb max 45Kb ewma 3 Plog 19 Scell_log 12
 Sent 5390829 bytes 3575 pkts (dropped 0, overlimits 0)

 qdisc gred 804b:
 DP:1 (prio 2) Average Queue 0b Measured Queue 0b
         Packet drops: 0 (forced 0 early 0)
         Packet totals: 0 (bytes 0)
 limit 60Kb min 15Kb max 45Kb ewma 3 Plog 21 Scell_log 12
 DP:2 (prio 3) Average Queue 0b Measured Queue 0b
         Packet drops: 0 (forced 0 early 0)
         Packet totals: 0 (bytes 0)
 limit 60Kb min 15Kb max 45Kb ewma 3 Plog 20 Scell_log 12
 DP:3 (prio 4) Average Queue 0b Measured Queue 0b
         Packet drops: 0 (forced 0 early 0)
         Packet totals: 0 (bytes 0)
 limit 60Kb min 15Kb max 45Kb ewma 3 Plog 19 Scell_log 12
 Sent 0 bytes 0 pkts (dropped 0, overlimits 0)

 qdisc gred 804a:
 DP:1 (prio 2) Average Queue 0b Measured Queue 0b
         Packet drops: 0 (forced 0 early 0)
         Packet totals: 0 (bytes 0)
 limit 60Kb min 15Kb max 45Kb ewma 3 Plog 21 Scell_log 13
 DP:2 (prio 3) Average Queue 0b Measured Queue 0b
         Packet drops: 0 (forced 0 early 0)
         Packet totals: 0 (bytes 0)
 limit 60Kb min 15Kb max 45Kb ewma 3 Plog 20 Scell_log 13
 DP:3 (prio 4) Average Queue 0b Measured Queue 0b
         Packet drops: 0 (forced 0 early 0)
         Packet totals: 0 (bytes 0)
 limit 60Kb min 15Kb max 45Kb ewma 3 Plog 19 Scell_log 13
 Sent 0 bytes 0 pkts (dropped 0, overlimits 0)

 qdisc gred 8049:
 DP:1 (prio 2) Average Queue 0b Measured Queue 0b
         Packet drops: 0 (forced 0 early 0)
         Packet totals: 0 (bytes 0)
 limit 60Kb min 15Kb max 45Kb ewma 3 Plog 21 Scell_log 14
 DP:2 (prio 3) Average Queue 0b Measured Queue 0b
         Packet drops: 0 (forced 0 early 0)
         Packet totals: 0 (bytes 0)
 limit 60Kb min 15Kb max 45Kb ewma 3 Plog 20 Scell_log 14
 DP:3 (prio 4) Average Queue 0b Measured Queue 0b
         Packet drops: 0 (forced 0 early 0)
         Packet totals: 0 (bytes 0)
 limit 60Kb min 15Kb max 45Kb ewma 3 Plog 19 Scell_log 14
 Sent 0 bytes 0 pkts (dropped 0, overlimits 0)

 qdisc cbq 3: rate 1536Kbit cell 8b mpu 64b (bounded,isolated) prio
no-transmit/8 weight 1536Kbit allot 1514b
level 1 ewma 5 avpkt 1000b maxidle 158us
 Sent 5412399 bytes 3746 pkts (dropped 0, overlimits 2528)
  borrowed 0 overactions 0 avgidle 4166 undertime 0

 qdisc pfifo 8048: limit 5p
 Sent 5348203 bytes 3559 pkts (dropped 97, overlimits 0)

 qdisc cbq 2: rate 3Mbit cell 8b mpu 64b (bounded,isolated) prio
no-transmit/8 weight 3Mbit allot 1514b
level 1 ewma 5 avpkt 1000b maxidle 79us
 Sent 10760812 bytes 7310 pkts (dropped 97, overlimits 22652)
  borrowed 0 overactions 0 avgidle 2083 undertime 0

 qdisc dsmark 1: indices 0x0040 set_tc_index
 Sent 10760812 bytes 7310 pkts (dropped 97, overlimits 0)


class cbq 3: root rate 1536Kbit cell 8b mpu 64b (bounded,isolated) prio
no-transmit/8 weight 1536Kbit allot 1514b
level 1 ewma 5 avpkt 1000b maxidle 158us
 Sent 5416104 bytes 3752 pkts (dropped 0, overlimits 0)
  borrowed 0 overactions 0 avgidle 3788 undertime 0
class cbq 3:1 parent 3: leaf 8049: rate 256Kbit cell 8b (bounded) prio 5/5
allot 1514b
level 0 ewma 5 avpkt 1000b maxidle 24104us
 Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
  borrowed 0 overactions 0 avgidle 631894 undertime 0
class cbq 3:2 parent 3: leaf 804a: rate 512Kbit cell 8b (bounded) prio 4/4
allot 1514b
level 0 ewma 5 avpkt 1000b maxidle 9641us
 Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
  borrowed 0 overactions 0 avgidle 252757 undertime 0
class cbq 3:3 parent 3: leaf 804b: rate 768Kbit cell 8b (bounded) prio 3/3
allot 1514b
level 0 ewma 5 avpkt 1000b maxidle 4820us
 Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
  borrowed 0 overactions 0 avgidle 126378 undertime 0
class cbq 3:4 parent 3: leaf 804c: rate 1Mbit cell 8b (bounded) prio 2/2
allot 1514b
level 0 ewma 5 avpkt 1000b maxidle 2409us
 Sent 5390829 bytes 3575 pkts (dropped 0, overlimits 2719)
  borrowed 0 overactions 214 avgidle 23670 undertime 0
class cbq 3:5 parent 3: leaf 804d: rate 1Mbit cell 8b prio 7/7 allot 1514b
level 0 ewma 5 avpkt 1000b maxidle 2409us
 Sent 25275 bytes 177 pkts (dropped 0, overlimits 0)
  borrowed 0 overactions 0 avgidle 51339 undertime 0
class cbq 2: root rate 3Mbit cell 8b mpu 64b (bounded,isolated) prio
no-transmit/8 weight 3Mbit allot 1514b
level 1 ewma 5 avpkt 1000b maxidle 79us
 Sent 210 bytes 5 pkts (dropped 0, overlimits 0)
  borrowed 0 overactions 0 avgidle 2083 undertime 0
class cbq 2:1 parent 2: leaf 8048: rate 1536Kbit cell 8b (bounded,isolated)
prio 1/1 allot 1514b
level 0 ewma 5 avpkt 1000b maxidle 949us
 Sent 5348203 bytes 3559 pkts (dropped 97, overlimits 13649)
  borrowed 0 overactions 1143 avgidle 110 undertime 0
class cbq 2:2 parent 2: leaf 3: rate 1536Kbit cell 8b (bounded,isolated)
prio 2/2 allot 1514b
level 0 ewma 5 avpkt 1000b maxidle 949us
 Sent 5416104 bytes 3752 pkts (dropped 0, overlimits 7615)
  borrowed 0 overactions 540 avgidle 18787 undertime 0

filter parent 1: protocol ip pref 1 tcindex hash 64 mask 0x00fc shift 2
fall_through
filter parent 1: protocol ip pref 1 tcindex handle 0x000a classid 1:111
filter parent 1: protocol ip pref 1 tcindex handle 0x000c classid 1:112
filter parent 1: protocol ip pref 1 tcindex handle 0x000e classid 1:113
filter parent 1: protocol ip pref 1 tcindex handle 0x0012 classid 1:121
filter parent 1: protocol ip pref 1 tcindex handle 0x0014 classid 1:122
filter parent 1: protocol ip pref 1 tcindex handle 0x0016 classid 1:123
filter parent 1: protocol ip pref 1 tcindex handle 0x001a classid 1:131
filter parent 1: protocol ip pref 1 tcindex handle 0x001c classid 1:132
filter parent 1: protocol ip pref 1 tcindex handle 0x001e classid 1:133
filter parent 1: protocol ip pref 1 tcindex handle 0x0022 classid 1:141
filter parent 1: protocol ip pref 1 tcindex handle 0x0024 classid 1:142
filter parent 1: protocol ip pref 1 tcindex handle 0x0026 classid 1:143


Transfer rates achived

AF4 - 2.1Mb/s


5MB transfer over the AF41 and EF flows at the same time


bwmonitor output for eth0

980880075 2539264 69088
980880076 7352616 183440
980880077 7631200 180960
980880078 7764432 191520
980880079 7146720 170832
980880080 7098272 172160
980880081 7497968 181376
980880082 7849216 185904
980880083 4761520 126456
980880084 6783136 164048
980880085 1829552 55392
980880086 1829544 48160
980880087 1672088 74160
980880088 2314024 68632
980880089 1151272 33696
980880090 1611528 44896
980880091 1454072 37360
980880092 1417736 41696
980880093 1599416 47616
980880094 1357176 34480
980880095 1611528 52792
980880096 1696312 43624
980880097 1926440 65848
980880098 1768984 52440
980880099 1166216 38848

Output from TC

qdisc red 805f: limit 60Kb min 15Kb max 45Kb ewma 3 Plog 17 Scell_log 12
 Sent 3681 bytes 48 pkts (dropped 0, overlimits 0)

 qdisc gred 805e:
 DP:1 (prio 2) Average Queue 0b Measured Queue 0b
         Packet drops: 0 (forced 0 early 0)
         Packet totals: 3977 (bytes 6009808)
 limit 60Kb min 15Kb max 45Kb ewma 3 Plog 21 Scell_log 12
 DP:2 (prio 3) Average Queue 0b Measured Queue 0b
         Packet drops: 0 (forced 0 early 0)
         Packet totals: 0 (bytes 0)
 limit 60Kb min 15Kb max 45Kb ewma 3 Plog 20 Scell_log 12
 DP:3 (prio 4) Average Queue 0b Measured Queue 0b
         Packet drops: 0 (forced 0 early 0)
         Packet totals: 0 (bytes 0)
 limit 60Kb min 15Kb max 45Kb ewma 3 Plog 19 Scell_log 12
 Sent 6009808 bytes 3977 pkts (dropped 0, overlimits 0)

 qdisc gred 805d:
 DP:1 (prio 2) Average Queue 0b Measured Queue 0b
         Packet drops: 0 (forced 0 early 0)
         Packet totals: 0 (bytes 0)
 limit 60Kb min 15Kb max 45Kb ewma 3 Plog 21 Scell_log 12
 DP:2 (prio 3) Average Queue 0b Measured Queue 0b
         Packet drops: 0 (forced 0 early 0)
         Packet totals: 0 (bytes 0)
 limit 60Kb min 15Kb max 45Kb ewma 3 Plog 20 Scell_log 12
 DP:3 (prio 4) Average Queue 0b Measured Queue 0b
         Packet drops: 0 (forced 0 early 0)
         Packet totals: 0 (bytes 0)
 limit 60Kb min 15Kb max 45Kb ewma 3 Plog 19 Scell_log 12
 Sent 0 bytes 0 pkts (dropped 0, overlimits 0)

 qdisc gred 805c:
 DP:1 (prio 2) Average Queue 0b Measured Queue 0b
         Packet drops: 0 (forced 0 early 0)
         Packet totals: 0 (bytes 0)
 limit 60Kb min 15Kb max 45Kb ewma 3 Plog 21 Scell_log 13
 DP:2 (prio 3) Average Queue 0b Measured Queue 0b
         Packet drops: 0 (forced 0 early 0)
         Packet totals: 0 (bytes 0)
 limit 60Kb min 15Kb max 45Kb ewma 3 Plog 20 Scell_log 13
 DP:3 (prio 4) Average Queue 0b Measured Queue 0b
         Packet drops: 0 (forced 0 early 0)
         Packet totals: 0 (bytes 0)
 limit 60Kb min 15Kb max 45Kb ewma 3 Plog 19 Scell_log 13
 Sent 0 bytes 0 pkts (dropped 0, overlimits 0)

 qdisc gred 805b:
 DP:1 (prio 2) Average Queue 0b Measured Queue 0b
         Packet drops: 0 (forced 0 early 0)
         Packet totals: 0 (bytes 0)
 limit 60Kb min 15Kb max 45Kb ewma 3 Plog 21 Scell_log 14
 DP:2 (prio 3) Average Queue 0b Measured Queue 0b
         Packet drops: 0 (forced 0 early 0)
         Packet totals: 0 (bytes 0)
 limit 60Kb min 15Kb max 45Kb ewma 3 Plog 20 Scell_log 14
 DP:3 (prio 4) Average Queue 0b Measured Queue 0b
         Packet drops: 0 (forced 0 early 0)
         Packet totals: 0 (bytes 0)
 limit 60Kb min 15Kb max 45Kb ewma 3 Plog 19 Scell_log 14
 Sent 0 bytes 0 pkts (dropped 0, overlimits 0)

 qdisc cbq 3: rate 1536Kbit cell 8b mpu 64b (bounded,isolated) prio
no-transmit/8 weight 1536Kbit allot 1514b
level 1 ewma 5 avpkt 1000b maxidle 158us
 Sent 6013489 bytes 4025 pkts (dropped 0, overlimits 4826)
  borrowed 0 overactions 0 avgidle 4166 undertime 0

 qdisc pfifo 805a: limit 5p
 Sent 5341992 bytes 3538 pkts (dropped 57, overlimits 0)

 qdisc cbq 2: rate 3Mbit cell 8b mpu 64b (bounded,isolated) prio
no-transmit/8 weight 3Mbit allot 1514b
level 1 ewma 5 avpkt 1000b maxidle 79us
 Sent 11355481 bytes 7563 pkts (dropped 57, overlimits 11680)
  borrowed 0 overactions 0 avgidle 2083 undertime 0

 qdisc dsmark 1: indices 0x0040 set_tc_index
 Sent 11355481 bytes 7563 pkts (dropped 57, overlimits 0)


class cbq 3: root rate 1536Kbit cell 8b mpu 64b (bounded,isolated) prio
no-transmit/8 weight 1536Kbit allot 1514b
level 1 ewma 5 avpkt 1000b maxidle 158us
 Sent 6017192 bytes 4031 pkts (dropped 0, overlimits 0)
  borrowed 0 overactions 0 avgidle 3556 undertime 0
class cbq 3:1 parent 3: leaf 805b: rate 256Kbit cell 8b (bounded) prio 5/5
allot 1514b
level 0 ewma 5 avpkt 1000b maxidle 24104us
 Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
  borrowed 0 overactions 0 avgidle 631894 undertime 0
class cbq 3:2 parent 3: leaf 805c: rate 512Kbit cell 8b (bounded) prio 4/4
allot 1514b
level 0 ewma 5 avpkt 1000b maxidle 9641us
 Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
  borrowed 0 overactions 0 avgidle 252757 undertime 0
class cbq 3:3 parent 3: leaf 805d: rate 768Kbit cell 8b (bounded) prio 3/3
allot 1514b
level 0 ewma 5 avpkt 1000b maxidle 4820us
 Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
  borrowed 0 overactions 0 avgidle 126378 undertime 0
class cbq 3:4 parent 3: leaf 805e: rate 1Mbit cell 8b (bounded) prio 2/2
allot 1514b
level 0 ewma 5 avpkt 1000b maxidle 2409us
 Sent 6009808 bytes 3977 pkts (dropped 0, overlimits 4988)
  borrowed 0 overactions 285 avgidle 10299 undertime 0
class cbq 3:5 parent 3: leaf 805f: rate 1Mbit cell 8b prio 7/7 allot 1514b
level 0 ewma 5 avpkt 1000b maxidle 2409us
 Sent 7384 bytes 54 pkts (dropped 0, overlimits 0)
  borrowed 0 overactions 0 avgidle 47027 undertime 0
class cbq 2: root rate 3Mbit cell 8b mpu 64b (bounded,isolated) prio
no-transmit/8 weight 3Mbit allot 1514b
level 1 ewma 5 avpkt 1000b maxidle 79us
 Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
  borrowed 0 overactions 0 avgidle 2083 undertime 0
class cbq 2:1 parent 2: leaf 805a: rate 1536Kbit cell 8b (bounded,isolated)
prio 1/1 allot 1514b
level 0 ewma 5 avpkt 1000b maxidle 949us
 Sent 5341992 bytes 3538 pkts (dropped 57, overlimits 4856)
  borrowed 0 overactions 411 avgidle 24911 undertime 0
class cbq 2:2 parent 2: leaf 3: rate 1536Kbit cell 8b (bounded,isolated)
prio 2/2 allot 1514b
level 0 ewma 5 avpkt 1000b maxidle 949us
 Sent 6017192 bytes 4031 pkts (dropped 0, overlimits 4352)
  borrowed 0 overactions 314 avgidle 14367 undertime 0

filter parent 1: protocol ip pref 1 tcindex hash 64 mask 0x00fc shift 2
fall_through
filter parent 1: protocol ip pref 1 tcindex handle 0x000a classid 1:111
filter parent 1: protocol ip pref 1 tcindex handle 0x000c classid 1:112
filter parent 1: protocol ip pref 1 tcindex handle 0x000e classid 1:113
filter parent 1: protocol ip pref 1 tcindex handle 0x0012 classid 1:121
filter parent 1: protocol ip pref 1 tcindex handle 0x0014 classid 1:122
filter parent 1: protocol ip pref 1 tcindex handle 0x0016 classid 1:123
filter parent 1: protocol ip pref 1 tcindex handle 0x001a classid 1:131
filter parent 1: protocol ip pref 1 tcindex handle 0x001c classid 1:132
filter parent 1: protocol ip pref 1 tcindex handle 0x001e classid 1:133
filter parent 1: protocol ip pref 1 tcindex handle 0x0022 classid 1:141
filter parent 1: protocol ip pref 1 tcindex handle 0x0024 classid 1:142
filter parent 1: protocol ip pref 1 tcindex handle 0x0026 classid 1:143


Transfer rates achived

EF - 5.5Mb/s
AF4 - 2.4Mb/s

Thanks
William