Posts from ‘Frame-Relay’
Legacy Frame-Relay Traffic Shaping Basics
This blog post is taken from the INE Resources area Understanding Frame-Relay Traffic Shaping presentation by Brian Dennis.
Overview
Frame-Relay traffic shaping is designed to control the amount of traffic the router sends out of an interface or out of a particular DLCI. Common reasons for Frame-Relay traffic shaping are:
- It allows the router to conform to the rate subscribed with the service provider
- It allows for the throttling of a higher speed site (768K) so that it does not overrun a lower speed site (64K)
Traffic shaping is designed to delay excess traffic, whereas policing is designed to drop excess traffic.
Terminology
- Available Rate (AR) – the actual physical speed of the interface; on a DCE serial interface this is determined by the configured clock rate. On a DTE serial interface, it is determined by the received clock rate. A router will always (by default) try to send out at the AR regardless of the interface bandwidth. AR is also commonly referred to as port speed, line rate, or access rate.
You just cannot assume anything when you sit for your Version 4.0 CCIE R&S Lab Exam. One of the former assumptions we could make with Version 3.x was that our Frame Relay Switch is going to be just fine and dandy. Therefore, if you examined your PVC health (status) and you saw DELETED, you could immediately inspect your Frame Relay map statements, or your frame-relay interface-dlci commands for a typo in the DLCI.
But in this new exam (Troubleshooting section or Configuration section), nothing is off limits from your problem scope. OK, well, to be more accurate, most Layer 1 issues are still indeed out of scope. In fact, in the Troubleshooting section, Layer 1 really cannot be an issue since the devices we are troubleshooting are actually virtual routers. You cannot even run up against a bad cable there! But still, there is a lot more that we can be asked to troubleshoot than in the past. And if you think about the Core Knowledge section, they could even ask Layer 1 troubleshooting-related questions there instead!
In this blog post (dedicated to my current Advanced Troubleshooting Bootcamp Live Class), we will examine Frame Relay troubleshooting where the Frame Relay Switch rears its rather ugly head.
Tags: ccie, exam, lab, practice, troubleshooting
Following along the idea of pinging yourself (depending on where you live, there may actually be laws against that!), there are some further problems if we happen to be running PPP over Frame-Relay.
I will stick with the same pretty graphic that Anthony provided in his entry between R3 and R5!
So let’s start with some basic Frame-Relay configuration and move up from there.
R3
int s1/0
encap frame
no shut
R5
int s0/0/0
encap frame
no shut
INE-R3(config-if)#do sh frame pvc | in ACT DLCI = 301, DLCI USAGE = UNUSED, PVC STATUS = INACTIVE, INTERFACE = Serial1/0 DLCI = 302, DLCI USAGE = UNUSED, PVC STATUS = INACTIVE, INTERFACE = Serial1/0 DLCI = 304, DLCI USAGE = UNUSED, PVC STATUS = INACTIVE, INTERFACE = Serial1/0 DLCI = 305, DLCI USAGE = UNUSED, PVC STATUS = ACTIVE, INTERFACE = Serial1/0 INE-R3(config-if)# INE-R5(config-if)#do sh frame pvc | in ACT DLCI = 501, DLCI USAGE = LOCAL, PVC STATUS = INACTIVE, INTERFACE = Serial0/0/0 DLCI = 502, DLCI USAGE = LOCAL, PVC STATUS = INACTIVE, INTERFACE = Serial0/0/0 DLCI = 503, DLCI USAGE = LOCAL, PVC STATUS = ACTIVE, INTERFACE = Serial0/0/0 DLCI = 504, DLCI USAGE = LOCAL, PVC STATUS = INACTIVE, INTERFACE = Serial0/0/0 DLCI = 513, DLCI USAGE = LOCAL, PVC STATUS = INACTIVE, INTERFACE = Serial0/0/0 INE-R5(config-if)#
I’m at least active between R3 and R5 now, so this is a good thing! Now, to configure PPPoFR (let’s say our requirement was to authenticate the routers) we set up a Virtual Template… Let’s start with the basics and we’ll add complications in a little bit! 
You might have a request in the Core Knowledge, Troubleshooting, or the Configuration section to have a Cisco router be able to PING its own Frame-Relay IP address. In this blog post, we will make sure we can accomplish this. Here is the topology that we will use:
Sample Trouble Ticket
Your junior administrator has reported that R1 and R3 are unable to establish their EIGRP adjacency.
The first thing I want to do is engage in some “ground up” troubleshooting here. How are Layers 1 and 2 starting at R1?
R1#show ip int brief Interface IP-Address OK? Method Status Protocol FastEthernet0/0 unassigned YES unset administratively down down Serial0/0 178.10.0.1 YES manual up up
Tags: ccie, lab, practice, strategy, troubleshooting
Note: The following post is an excerpt from the full QoS section of IEWB-RS VOL1 version 5.
Peak shaping may look confusing at first sight; however, its function becomes clear once you think of oversubscription. As we discussed before, oversubscription means selling customers more bandwidth than a network can supply, hoping that not all connections would use their maximum sending rate at the same time. With oversubscription, traffic contract usually specifies three parameters: PIR, CIR and Tc – peak rate, committed rate and averaging time interval for rate measurements. The SP allows customers to send traffic at rates up to PIR, but only guarantees CIR rate in case of network congestion. Inside the network SP uses any of the max-min scheduling procedures to implement bandwidth sharing in such manner that oversubscribed traffic has lower preference than conforming traffic. Additionally, the SP generally assumes that customers respond to notifications of traffic congestion in the network (either explicit, such as FECN/BECN/TCP ECN or implicit such as packet drops in TCP) by slowing down sending rate.
Commonly, customers implement traffic shaping to conform to traffic contract, and provider uses traffic policing to enforce the contract. If a contract specifies PIR, then it makes sense for customer to shape traffic at PIR rate. However, this makes difficult to deduce CIR value just by looking at the router configuration. In some circumstances, like with Frame-Relay networks, a secondary parameter, known as minCIR, may help to understand the configuration quickly. In general, it would benefit to see CIR and PIR in the shaping configuration at the same time. This is exactly the idea behind shape peak. When you configure
shape peak <CIR> <Bc> <Be>
the actual maximum sending rate is limited to:
PIR = CIR*(1+Be/Bc).
That is, each time interval Tc=Bc/CIR the shaper allows sending up to Bc+Be bits of data. By default, if you omit the value for Be, it equals to Bc and thus PIR=2*CIR by default. However, due to some IOS show output discrepancy, this is NOT reflected in “show” command output, unless you explicitly specify the Be value in command line. With shape peak configured this way, you can see both CIR as the “average rate” and PIR as the “target rate” when issuing “show policy-map” command.
Rack1R6#show policy-map interface fastEthernet 0/0.146
FastEthernet0/0.146
Service-policy output: POLICY_VLAN146_OUT
Class-map: HTTP (match-all)
6846 packets, 4065413 bytes
5 minute offered rate 63000 bps, drop rate 0 bps
Match: access-group 180
Traffic Shaping
Target/Average Byte Sustain Excess Interval Increment
Rate Limit bits/int bits/int (ms) (bytes)
128000/64000 1600 6400 6400 100 1600
...
All other shaping functions remain the same as with the classic GTS – shape peak is just more suited for use with oversubscription scenarios. Also, in Frame-Relay networks you may want to use configuration similar to the following to respond to congestion notifications:
shape peak <CIR> <Bc> <Be> shape adaptive <CIR>
To illustrate the use of shape peak, let’s look at the following scenario. Here, R4 serves two customers (R1 and R6) sending their traffic across one serial link of 128Kbps between R4 and R5. The fictive ISP sells 128Kbps (PIR) to each of the customers, guaranteeing only 64Kbps (CIR). Let’s assume the measurement interval of 100ms for this configuration. The serial link, which is the oversubscribed resource, uses WFQ for fair bandwidth sharing between two flows.
R1:
access-list 180 permit tcp any eq 80 any
!
class-map HTTP
match access-group 180
!
policy-map POLICY_VLAN146_OUT
class HTTP
shape peak 64000 6400 6400
!
interface FastEthernet 0/0
service-policy output POLICY_VLAN146_OUT
R6:
access-list 180 permit tcp any eq 80 any
!
class-map HTTP
match access-group 180
!
policy-map POLICY_VLAN146_OUT
class HTTP
shape peak 64000 6400 6400
!
interface FastEthernet 0/0.146
service-policy output POLICY_VLAN146_OUT
R4:
!
! All HTTP traffic
!
ip access-list extended HTTP
permit tcp any eq 80 any
!
class-map HTTP
match access-group name HTTP
!
! Traffic from R1 and R6 respectively
!
ip access-list extended FROM_R1
permit ip host 155.1.146.1 any
!
ip access-list extended FROM_R6
permit ip host 155.1.146.6 any
!
!
!
class-map FROM_R1
match access-group name FROM_R1
!
class-map FROM_R6
match access-group name FROM_R6
!
! Subrate policers
!
policy-map SUBRATE_POLICER
class FROM_R1
police cir 64000 bc 3200 pir 128000 be 6400
conform-action set-prec-transmit 1
exceed-action set-prec-transmit 0
violate-action drop
class FROM_R6
police cir 64000 bc 3200 pir 128000 be 6400
conform-action set-prec-transmit 1
exceed-action set-prec-transmit 0
violate-action drop
!
! Policer configuration using MQC syntax.
!
policy-map POLICE_VLAN146
class HTTP
service-policy SUBRATE_POLICER
!
interface FastEthernet 0/1
service-policy input POLICE_VLAN146
The idea is to allow R1 and R6 send up to 128Kbps if there is enough bandwidth on the serial link. However, if both of the sources start streaming at the same time, the SP may only guarantee up to 64Kbps to each of sending routers. The implementation meters each flow against 64Kbps and 128Kbps meters, and marks all conforming traffic with IP precedence of 1. All exceeding traffic is marked with IP precedence of 0. Since the serial link uses WFQ, we conclude that traffic marked with IP precedence of zero has lower scheduling weight. Thus, if IP precedence 1 traffic exist on the link, it is given preference over low-priority traffic (precedence 0).
To verify our configuration in action, start downloading a large file from R1 across R4 and see the statistics on R1 and R4:
Rack1R4#show policy-map interface fastEthernet 0/1
FastEthernet0/1
Service-policy input: POLICE_VLAN146
Class-map: HTTP (match-all)
20451 packets, 12066090 bytes
30 second offered rate 126000 bps, drop rate 0 bps
Match: access-group name HTTP
Service-policy : SUBRATE_POLICER
Class-map: FROM_R1 (match-all)
20451 packets, 12066090 bytes
30 second offered rate 126000 bps, drop rate 0 bps
Match: access-group name FROM_R1
police:
cir 64000 bps, bc 3200 bytes
pir 128000 bps, be 6400 bytes
conformed 11113 packets, 6556670 bytes; actions:
set-prec-transmit 1
exceeded 9338 packets, 5509420 bytes; actions:
set-prec-transmit 0
violated 0 packets, 0 bytes; actions:
drop
conformed 64000 bps, exceed 62000 bps, violate 0 bps
Class-map: FROM_R6 (match-all)
0 packets, 0 bytes
30 second offered rate 0 bps, drop rate 0 bps
Match: access-group name FROM_R6
police:
cir 64000 bps, bc 3200 bytes
pir 128000 bps, be 6400 bytes
conformed 0 packets, 0 bytes; actions:
set-prec-transmit 1
exceeded 0 packets, 0 bytes; actions:
set-prec-transmit 0
violated 0 packets, 0 bytes; actions:
drop
conformed 0 bps, exceed 0 bps, violate 0 bps
Class-map: class-default (match-any)
0 packets, 0 bytes
30 second offered rate 0 bps, drop rate 0 bps
Match: any
!
! The above statistics demonstrate that R1 uses almost all available bandwidth
! From the output below we can see that R1 is set to CIR 64Kbps and PIR 128Kbs.
! We may also notice that shaper was active for some time, delaying hundreds of
! exceeding packets. This usually happens in the beginning of TCP session when
! sendger aggressively increases sending rate.
!
Rack1R1#show policy-map interface fastEthernet 0/0
FastEthernet0/0
Service-policy output: POLICY_VLAN146_OUT
Class-map: HTTP (match-all)
3225 packets, 1897929 bytes
30 second offered rate 124000 bps, drop rate 0 bps
Match: access-group 180
Traffic Shaping
Target/Average Byte Sustain Excess Interval Increment
Rate Limit bits/int bits/int (ms) (bytes)
128000/64000 1600 6400 6400 100 1600
Adapt Queue Packets Bytes Packets Bytes Shaping
Active Depth Delayed Delayed Active
- 0 3225 1897929 348 205320 no
Class-map: class-default (match-any)
29 packets, 4378 bytes
30 second offered rate 0 bps, drop rate 0 bps
Match: any
Now start another file transfer, this time from R6 down to a host behind, R5 across the serial link. This will make both flows compete for the link bandwidth, and result in fair sharing of the link bandwidth. Now verify the policer statistics once again:
Rack1R4#show policy-map interface fastEthernet 0/1
FastEthernet0/1
Service-policy input: POLICE_VLAN146
Class-map: HTTP (match-all)
35113 packets, 20715559 bytes
30 second offered rate 126000 bps, drop rate 0 bps
Match: access-group name HTTP
Service-policy : SUBRATE_POLICER
Class-map: FROM_R1 (match-all)
29986 packets, 17691740 bytes
30 second offered rate 63000 bps, drop rate 0 bps
Match: access-group name FROM_R1
police:
cir 64000 bps, bc 3200 bytes
pir 128000 bps, be 6400 bytes
conformed 18466 packets, 10894940 bytes; actions:
set-prec-transmit 1
exceeded 11520 packets, 6796800 bytes; actions:
set-prec-transmit 0
violated 0 packets, 0 bytes; actions:
drop
conformed 63000 bps, exceed 0 bps, violate 0 bps
Class-map: FROM_R6 (match-all)
5127 packets, 3023819 bytes
30 second offered rate 63000 bps, drop rate 0 bps
Match: access-group name FROM_R6
police:
cir 64000 bps, bc 3200 bytes
pir 128000 bps, be 6400 bytes
conformed 5124 packets, 3022049 bytes; actions:
set-prec-transmit 1
exceeded 3 packets, 1770 bytes; actions:
set-prec-transmit 0
violated 0 packets, 0 bytes; actions:
drop
conformed 63000 bps, exceed 0 bps, violate 0 bps
Class-map: class-default (match-any)
0 packets, 0 bytes
30 second offered rate 0 bps, drop rate 0 bps
Match: any
!
! Verify statistics for both traffic shapers on R1 and R6. Both are set for PIR=128Kbps.
! However, metered rate is close to CIR, and the shaping is inactive. The sending rate
! went down thanks to TCP implicit congestion management procedure, that makes protocol
! sending rate adaptive to congestion in networks.
!
Rack1R6#show policy-map interface fastEthernet 0/0.146
FastEthernet0/0.146
Service-policy output: POLICY_VLAN146_OUT
Class-map: HTTP (match-all)
6846 packets, 4065413 bytes
5 minute offered rate 63000 bps, drop rate 0 bps
Match: access-group 180
Traffic Shaping
Target/Average Byte Sustain Excess Interval Increment
Rate Limit bits/int bits/int (ms) (bytes)
128000/64000 1600 6400 6400 100 1600
Adapt Queue Packets Bytes Packets Bytes Shaping
Active Depth Delayed Delayed Active
- 0 6846 4065413 3 1782 no
Class-map: class-default (match-any)
191 packets, 43930 bytes
5 minute offered rate 0 bps, drop rate 0 bps
Match: any
Rack1R1#show policy-map interface fastEthernet 0/0
FastEthernet0/0
Service-policy output: POLICY_VLAN146_OUT
Class-map: HTTP (match-all)
33062 packets, 19505469 bytes
30 second offered rate 63000 bps, drop rate 0 bps
Match: access-group 180
Traffic Shaping
Target/Average Byte Sustain Excess Interval Increment
Rate Limit bits/int bits/int (ms) (bytes)
128000/64000 1600 6400 6400 100 1600
Adapt Queue Packets Bytes Packets Bytes Shaping
Active Depth Delayed Delayed Active
- 0 33062 19505469 2632 1552858 no
Class-map: class-default (match-any)
7641 packets, 7385752 bytes
30 second offered rate 0 bps, drop rate 0 bps
Match: any
Now let’s confirm that WFQ is actually working on the serial interface between R4 and R5 and provides truly fair division of the bandwidth:
Rack1R4#show queueing interface serial 0/1
Interface Serial0/1 queueing strategy: fair
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: weighted fair
Output queue: 12/1000/64/0 (size/max total/threshold/drops)
Conversations 2/3/256 (active/max active/max total)
Reserved Conversations 0/0 (allocated/max allocated)
Available Bandwidth 96 kilobits/sec
(depth/weight/total drops/no-buffer drops/interleaves) 6/16192/0/0/0
Conversation 134, linktype: ip, length: 580
source: 155.1.146.1, destination: 155.1.58.8, id: 0xEB41, ttl: 254,
TOS: 32 prot: 6, source port 80, destination port 11001
(depth/weight/total drops/no-buffer drops/interleaves) 6/16192/0/0/0
Conversation 192, linktype: ip, length: 580
source: 155.1.146.6, destination: 155.1.108.10, id: 0x70CA, ttl: 254,
TOS: 32 prot: 6, source port 80, destination port 11002
To summarize, shape peak is a special form of shaping specifically adapted to configure oversubscription scenarios. All other properties of GTS remains the same.
Tags: adaptive, iewb, oversubscription, sample, shape-peak, shaping
Can you please help me understand use of Frame-Relay Interface-dlci command. It’s getting mysterious for me day by day as I am studying FR. The reason being is I earlier thought that I should only use this command on FR point to point subinterface. As Point to Point subinterface don’t allow us to put Frame relay map statements. Also in such case Inverse arp should be turned off. But while I was going through Cisco’s FR documentation on website I saw that in almost all examples they used interface dlci command on interface not on sub interface and also without turning off inverse arp. So the question now is if inverse arp is turned on then as per my understanding we need not to put this command as it will discover dlci settings through lmi signals automatically.
Kindly explain Interface Dlci command to me…
When I saw this post, I got to thinking a little bit. Mostly about the fact that the interface-dlci appears to be a much more misunderstood command than I ever gave it credit for! (poor thing…)
The quick answer is that the “frame-relay inteface-dlci” command simply says “This DLCI goes here” to the router.
On a physical interface, this command is largely irrelevant (more in a minute) because ALL DLCIs are assigned to the physical interface by default. If you are ever interested, concerned or otherwise bored, just check out “show frame-relay pvc” and you will see where they are assigned.
So in the case of sub-interfaces, there is no automagical assignment of DLCI numbers. Even if your subinterface number and DLCI number are the same. That’s just a sign of being anal-retentive (or as we consultants call it, “good at documentation”) or a little OCD. But you can technically have DLCI 100 on subinterface Serial 0/0.223. Kinda strange, but perfectly workable!
So whenever you have a subinterface, you need to do SOMETHING to tell the router “this DLCI goes here”.
So now let’s look at the next portion: Mapping. Layer3 to Layer2 mapping in particular. We can learn about L3-L2 mapping via Inverse ARP. This is on by default, but frowned upon in the CCIE Realm! “Show frame-relay map” will let you know if you have learned any addresses dynamically or not.
So if we DID allow Inverse ARP, whether our subinterfaces were point-to-point or multipoint ones, we COULD just use the “frame-relay interface-dlci” command and nothing else. (Yes, I know inverse ARP requests are not sent by default on subinterfaces, but responses still are. Watch your debugs!)
So the Interface-DLCI command assigned the PVC to a subinterface. Inverse ARP then took care of the mapping. What if we aren’t allowed to use Inverse ARP? Like for the CCIE lab? Ok, what are our options? Well, the “frame-relay map” command is the most obvious and well known. That works very well. The “frame-relay map” command both assigned L3-L2 mapping AND says “this DLCI goes here” all in one command!
Unless of course you are on a point-to-point subinterface. As you pointed out, you can’t use the map command there! But that’s ok, it’s not needed anyway! Point-to-point links have a different way of thinking. They view the world as “If it’s not my address it must be yours” and sends things out.
So that covers our two primary issues with PVC operations in Frame Relay. #1 is assigning the DLCI to an interface (no magic). #2 is the L3-L2 mapping to make IP actually work!
The last part I want to add (re: my “more in a minute” above) was that the “frame-relay inteface-dlci” command also serves another purpose which sometimes gets confusing in terms of where we just got through with things!
So where we left things with “frame-relay interface-dlci” commands:
1. Definitely used on point-to-point subinterfaces
2. Can be used on multipoint subinterfaces if Inverse ARP works
3. Not used on physical interfaces because all DLCIs belong there by default.
Now, just to mess with that logic a bit. When studying frame-relay, and particularly by the time you get into QoS configurations you will become familiar with frame-relay map-classes. Map classes can be assigned to an interface or subinterface without any problem. When this occurs, the information in the map-class gets appled to EVERY DLCI on that interface or subinterface.
So what happens if you have different QoS parameters for different PVCs that just happen to be on the same interface/subinterface? Hmmmm… Well, in comes the “frame-relay inteface-dlci” command again! See, it really IS a cool command!
The “frame-relay map” command does not have any parameter for adding a map-class. After you hit enter on your “frame-relay interface-dlci” command though, you’ll get a new sub-command prompt. Try using “?” here. You’ll see that you have the opportunity to specify a separate map-class for each and every DLCI that you have.
So if you see “frame-relay interface-dlci” commands on a physical interface. Or if you see them AND a “frame-relay map” command under a multipoint subinterface, this is the reason why. If you use the “frame-relay inteface-dlci” command AND the “frame-relay map” command for the same PVC, you will need to make sure the “frame-relay map” command comes first. Otherwise the router will express its displeasure!
So there are some very simple, but also some very powerful things the little “frame-relay interface-dlci” command does. Hopefully that will help you take some of the mystery out of things!
Tags: dlci, inverse-arp, layer2, mapping
More updates have been posted tonight to the CCIE Routing & Switching Lab Workbook Volume 1 Version 5.0 Beta on the member’s site. Bridging & Switching, Frame Relay, IP Routing, RIP, and EIGRP are now posted. Also there is a new feature that indicates when the files were updated last, so you will always know if you have the most current version.
The files are available as regular PDFs now instead of locklizard, and include both a visible watermark at the bottom and the new “hidden” steganographic watermarking system. Please post all questions about the content to IEOC.
Stay tuned, more updates will be posted in the new few days!
Hi Brian,
I ran into these nasty frame relay mappings during an initial lab set-up and was wondering why they are there, (even with inverse-arp disabled), and what they are actually doing. I was able to remove them only after writing my configuration to memory, and then performing a reload of the router.
Router(config-if)#do show frame map Serial0/0 (up): ip 0.0.0.0 dlci 113(0x71,0x1C10) broadcast, CISCO, status defined, inactive Serial0/0 (up): ip 0.0.0.0 dlci 105(0x69,0x1890) broadcast, CISCO, status defined, active Serial0/0 (up): ip 0.0.0.0 dlci 104(0x68,0x1880) broadcast, CISCO, status defined, active Serial0/0 (up): ip 0.0.0.0 dlci 103(0x67,0x1870) broadcast, CISCO, status defined, activeThanks,
Josh
Hi Josh,
This is actually an error relating to AutoInstall over Frame Relay. When the router boots up and does not have a configuration file saved in NVRAM, it attempts to run autoinstall to automatically find an IP address and download a config. The first thing the router does is to detect the encapsulation on its WAN interfaces, which in this case is Frame Relay. Once the router finds that it’s running Frame Relay, it attempts to send a config request via TFTP. In order to do this it needs an IP address, so it sends a BOOTP request out all DLCIs. Since the router doesn’t know what the unicast IP addresses are on the other ends of the circuits, it creates IPv4 mappings to 0.0.0.0 for all circuits and includes the “broadcast” keyword on them. This allows the router to encapsulate the BOOTP request out all DLCIs.
If you haven’t actually configured IP helper-address or a BOOTP server, the operation will fail. The result of this that we see is that when Frame Relay is re-enabled on the interfaces the mappings to 0.0.0.0 reappear. In some versions of IOS this can be fixed by removing Frame Relay and re-applying it, for example:
router#config t router(config)#interface s0/0 router(config-if)#encapsulation ppp router(config-if)#encapsulation frame-relay router(config-if)#end router#
In most versions however this does not work. Therefore the way to fix this is just to have the router not do autoinstall on bootup. Since the router does autoinstall because it doesn’t have a config saved in memory, the only way to 100% fix it is to save your config to NVRAM (wr m), and to reload.
Tags: bootp, dynamic, helper-address, inverse-arp, mapping
IEWB-RS Volume I Version 5 Frame Relay labs are now posted on the members site. Please post all questions and comments about it here.
Happy labbing!
Loading ...
