Sunday, March 29, 2009

Game Plan for tomorrow

Here is how I plan to attack the lab.

1. Contrary to popular strategy, I am NOT going to do a read-through the first time. In the past I've found doing a read-through to be a waste of time because my short-term memory just ain't that good.

2. If there are troubleshooting tasks, review the config and spend 30 minutes or so trying to spot them.

3. Attack L2, IGP, and EGP immediately after. I'll go ahead and configure authentication/encryption type tasks that aren't required for core connectivity, but I won't hesitate to skip them if I hit a snag.

4. Diagram entire BGP/VPN topology. Note that I wait until core is complete before doing this. I like having a good feel for the core before starting the diagram.

5. Complete the VPN configuration. While I'm comfortable with TE now, I'll consider skipping it if it's just a few points because it just seems to have a habit of breaking other things. If I can have VPN completed before lunch, I'll be extremely happy because that should give me plenty of time to polish off the other tasks and do verifications.

6. Diagram and complete multicast. Nothing too fancy of the diagram, I just find it greatly helps me spot rpf and inter-AS issues when I have a separate mcast diagram drawn out. Unfortunately, I know how nasty multicast can be from the R&S lab, so I need prepared to give up these points altogether if need be.

7. Complete the rest of the lab. Consider skipping any traffic filtering related security tasks altogether. If I'm feeling confident in my points, I'm not even going to bother with them because I'm just too afraid of breaking my core. Seems like there's always a routing protocol or tunnel involved that get broken. If I must filter, I'm going to throw an explicit deny log at the end of the access-list so I can see everything that's getting dropped.

At this point, hopefully I have at least an hour left to go back through the lab, step by step, to verify everything. When I passed R&S, I found at least 6 points doing this that were due to stupid omissions (e.g. only applying QoS policy to R5 instead of R5 and R6).

Then, with 30 minutes left, I won't make any more configuration changes unless ABSOLUTELY necessary.

My speed has been fine though, so I'll be a little surprised if I don't have the whole lab wrapped up with 2 hours to spare. If I do, I'll probably keep doing verifications and annoy the proctor with the pickiest little questions I can think of until the labs over or the proctor kicks me out.

And here's what worries me:
Layer 2:
PPPoE. I've done a few basic configs, but it seems like it could get pretty complicated.
l2tp/AToM. There really isn't much to it, so it shouldn't give me any problems.
ATM: I have the basics down, but if some knobs get thrown my way I could have issues.

IGP: IS-IS knobs make me a little nervous.

EGP: I could have spent more time on advanced BGP features like orf and fast external failover. But these seem to be covered well in the doccd.

MPLS: ldp knobs, such as funky neighbor attributes, label filtering, and label retention

VPN: Making sure to spot if/where I need to use domain-id, sham links, and soo. Broken LSPs can be nasty to figure out too.

Multicast:
I'm happy with MDTs and bgp ipv4 multicast now. I'm still a little uncomfortable with MSDP. I think I just have to keep in mind that if I don't have a pim path between two hosts, I'll need two RPs and will need to join them with MSDP.

QoS: They always seem to find something nasty I haven't thought of. I'll just need to spend a little extra time verifying that what I put on is working properly

Security: Bottom line, don't break the core

System Management and IP Services: Gotta find those knobs.

MPLS VPN Inter-AS Feature Guide

This document could be a life saver tomorrow. It's located under the 12.0S(29) feature guides.

http://www.cisco.com/en/US/docs/ios/12_0s/feature/guide/fsiasleb.html

It has a wealth of information on the send label command, route reflectors, and peering non-adjacent vpnv4 routers.

Saturday, March 28, 2009

IE SP Vol 2 Lab 10

Ok, I've changed my mind and decided to give lab 10 another try.

4:27pm lab started
4:35pm 1.1 complete, 3
4:43pm 1.2 complete, 3
4:45pm 1.3 complete, 3
4:49pm 1.4 complete, 3
Layer 2 complete, 12/12, 0:22

5:07pm 1.1 complete, 3
5:13pm 1.2 complete, 3
5:15pm 1.3 complete, 3
IGP complete, 9/9, 0:26

5:24pm 3.1 complete, 4
5:32pm 3.2 complete, 3 -- probably gonna need some next-hop-self's later but I'll add them when req'd.
EGP complete, 7/7, 0:17

5:42pm 4.1 complete, 3
6:01pm 4.2 complete, 3
6:09pm 4.3 complete, 3
6:24pm 4.4 complete, 3
MPLS complete, 12/12, 0:52

8:20am back from break
8:33am vpn diagram complete
8:49am 5.1 complete, 3
8:57am 5.2 complete, 3
9:58am 5.3 0/4 -- this one completely blew my mind. So it IS possible to have an interface participate in multiple vrf's using the vrf selection source command. I'm glad I decided to do lab 10 or I would have had no idea about this if I run into it tomorrow.

10:21am back from break
10:54am 4/4. Got this one, and it works beautifully! In summary, vrf leaking occurs on the way out so traffic can leave the vrf and go out to the global routing table. But on the way back in, the PE router needs to know which traffic goes to which vrf. The vrf selection commands enable the router to put the flows into the proper vrf on the way back to the P network. This essentially requires three steps.

1. Create the vrf and assicated rd and rt's on the PE router
2. Leak the vrf routes out to the global routing table for the downstream C networks
3. Inject the upstream C routes into the global routing table so the downstream C networks have reachability back
4. Specify which source networks are associated with which vrfs via the vrf selection source command
5. Enable ip vrf select source and ip vrf receive on the PE upstream interface
6. Resolve any next-hop rechability and MPLS LSP issues.

Sounds like a lot, but after doing a couple it's really not too bad.

11:22am 5.5 complete, 4, trick here was default-information-originate on R4 ipv4 address family.
11:48am 5.6 not complete, 0/4. I got hung up because the debug ip packet didn't show my packets flowing. All I needed to do was to advertise my link in bgp to BB1, but I never bothered to try because I thought something else was broken. Most likely these packets were being cef switched so I didn't even know they were flowing. Painful 4 points to give up...

VPN complete, 14/22, 2:05, 4:02 total

6.1 not complete, 0/3. They got me on this one. I tried to put this on the TE tunnel and the pim adjacencies wouldn't come up. I considered creating a gre tunnel, but figured there was some knob that would allow pim to talk across the TE tunnel. Nope, a gre tunnel was the solution.

6.2 not complete, 0/3. This makes sense at least, now. The new loopback can't pass the rpf check since R4 doesn't have a route to it. We don't want to add full ip reachability, we just want to pass the rpf check. So we add a bgp multicast peer and add the loopback there, so R4 can do the rpf check on the new loopback. Got it.

12:27pm 6.3 complete, 3/3. Have to carry the ipv4 multicast route out to R1.

Multicast complete, 3/9, 0:39

12:42pm 7.1 complete, 3/3
12:51pm 7.2 complte, 3/3
12:54pm 7.3 complete, 3/3
1:00pm 7.4 not complete, 0/3, they got me again, I forgot about the nat. Sad, because I even had NAT written on the diagram.
QoS complete, 9/12, 0:33

1:17pm 8.1 complete, 2/2. More trickery, do to this, besides mpls, ttl expiration messages must also be filtered. The solutions guide didn't catch onto this even, as it only prevents traffic that passes through the TE tunnels from showing the hops. Not all traffic is going through the tunnels!

1:21pm 8.2 not complete, 0/3. I misunderstood and didn't filter the tunnels too. I'm sure a customer would just love an ISP who filtered all tunneling from their network lol.

1:26pm 9.1 not complete, 0/3. I was looking for something totally different.

1:30pm 9.2 complete, 3/3. For once I didn't forget to enable traps.

1:31pm 10.1 complete, 2/2. Scan time

Lab 10 complete, 71/100, 5:45

Wow, not bad for a difficulty 9 lab. There were a few new concepts in this lab that was was able to catch on to and am sure I would be able to configure them next time. I'm actually feeling really good right about now.

Friday, March 27, 2009

IE SP Vol 2 Lab 7 Repeat

Here goes my 2nd attempt at lab 7.

2:15pm lab started
2:25pm 1.1 complete, 3 points
2:29pm 1.2 complete, 2 points
2:34pm 1.3 complete, 3 points
2:36pm 1.4 complete, 3 points
2:42pm 1.5 complete, 3 points
L2 complete, 14/14, 0:27

Taking CoD break

12:40am back from break
12:50am 2.1 complete, 3 points
12:57am 2.2 complete, 2 points
1:00am 2.3 complete, 3 points
1:01am 2.4 complete, 2 points
IGP complete, 10/10, 0:21

1:21am 3.1 complete, 3 points
1:33am 3.2 complete, 2 points
1:46am 3.3 complete, 2 points
1:52am 3.4 complete, 3 points
1:54am 3.5 complete, 3 points
2:03am 3.6 complete, 3 points
EGP complete, 16/16, 1:02

2:13am 4.1 complete, 3 points
MPLS complete, 3/3, 0:10

2:30am vpn diagram complete
2:39am 5.1 complete 3
2:54am 5.2 complete 3
3:22am 5.3 complete 3
3:42am 5.4 complete 3
3:46am 5.5 complete 3
4:22am 5.6 incomplete, 0/4 looked up the answer. This one still baffles me.
4:59am 5.7 complete, 3
5:14am 5.8 complete, 3
VPN mostly complete, 21/25, 3:01 (too long!!!)

5:24am 6.1 complete, 3 watch rfp checks on TE tunnel
5:44am 6.2 not complete, 0, not sure what vlan 7 is. Needed mpls ip on TE tunnel (AGAIN!!!)
6:15am 6.3 not complete, 0, but I did eventually get it working!
Multicast barely complete, 3/9, 1:01 (still making progress...)

6:25am 7.1 complete, 3
6:52am 7.2 not complete. 0/3 I have it configured right but the marking doesn't seem to be occurring.
7:11am 7.3 complete, 3
QoS complete, 6/9, 0:54

10:34am back from break
11:25am 8.1 complete, 3 points--too much time!
11:30am 8.2 wrong, 0/2--no ip unreachable on null0 interfaces, I never would have thought of that!
11:34am 8.3 complete, 3.
Security complete, 6/8, 1:00

11:43am 9.1 not complete, missed snmp ifindex persist and didn't use absolute
11:49am 10.1 not complete, needed to use nat instead of policy routing

Lab 7 complete, 8:11, 79%
The single biggest thing that worries me about this lab is task 5.6. I understand that a route is needed for the PE router so the mpls bindings occur. What is beyond me is why on earth redistribute connected works. Just as I'm typing that, I actually bother to do a sh ip route on R4. Sure enough, 131.1.5.5 shows up as connected, due to it being a peer neighbor route. Ok, I feel better now.

Thursday, March 26, 2009

IE SP Vol 2 Lab 4 Repeat

I'm going to try to do a repeat of labs 4, 7, and 8 before my big day. Here it goes...

4:44am lab started

Layer 2
4:55am 1.1 complete. 3 Vtp was acting up and I had to bounce both sides of the trunk int
4:58am 1.2 complete 3
5:04am 1.3 complete 3
5:07am 1.4 complete 3
Layer 2 complete, 12/12, 0:23

IGP
5:10am 2.1 complete 2
5:21am 2.2 complete 3, frame-rel map clns broad
5:25am 2.3 complete 2
6:06am 2.4 complete, 2 --got distracted
6:16am 2.5 complete, 3
IGP complete, 12/12, 1:09

EGP
6:27am 3.1 complete, 3
6:40am 3.2 complete, 4
EGP complete, 7/7, 0:34

MPLS
6:43am 4.1 complete, 3
6:49am 4.2 complete, 2
6:54am 4.3 complete, 2
7:01am 4.4 complete, 3
7:26am 4.5 complete, 3
MPLS complete, 13/13, 0:46

taking lunch
10:33am back from lunch
10:43am vpn diagram complete
10:47am 5.1 complete, 3 points
10:56am 5.2 complete, 3 points
11:00am 5.3 complete, 3 points
11:18am 5.4 complete, 3 points
11:34am 5.5 complete, 3 points
11:53am 5.6 complete, 3 points
12:41pm 5.7 skipped, the export map won't take
BGP almost complete, 18/21, 2:08. Too much time spent troubleshooting 5.7!

taking another lunch.
3:56pm back from lunch
4:10pm 6.1 complete, 3 points
4:13pm 6.2 complete, 2 points
4:17pm 6.3 complete, 3 points
5:06pm 6.4 not complete, 0 points, needed to make msdp router pim dr.
for the second time, task 6.5 killed r1.
Multicast complete, 8/14, 1:10

Total time so far: 6:10

QoS:
7.1 no issues, 3 points, 0:15
7.2 no issues, 3 points, 0:30
QoS complete, 6/6, 0:45

Security
8.1 no issues, 3 points, 0:05
8.2 no issues, 3 points, 0:05
Security complete, 6/6, 0:10

System Management
9.1 0/0, 0:10, banner motd, and no motd on other ports
9.2 2/2, 0:05
9.3 3/3, 0:10
System Management complete, 5/8, 0:25

IP Services
10.1 2/2, 0:10

Lab complete, 88%. Of the 12 points I missed, at least 6 seem to be due to dynamips issues. The solutions guide shows I was doing mdt's and the nat exercise correctly, but they just didn't want to come up.

By far my biggest weakness is multicast. I'll take a stab at lab 7 tomorrow and will again spend a lot of time on the mulitcast portion. I think I remembered most of the key concepts today, I just ran into some small implementation issues.

Sunday, March 22, 2009

IE SP Vol 2 Lab 10

Here we go, the last of the labs I haven't looked at yet.

12:38am lab started

Layer 2
12:44am 1.1 complete, 3
12:48am 1.2 complete, 3
12:52am 1.3 complete, 3
1:02am 1.4 complete, 3, assuming pvc command satisfies requirement
Layer 2 complete, 12/12, 0:24

IGP
1:12am 2.1 complete, 3
1:28am 2.2 complete, 3. Unable to authen RIP. Verified no spaces, etc, assuming BB issue.
1:33am 2.3 complete, 3 points
IGP complete, 3/3, 0:31

EGP
1:48am 3.1 complete, 3. Tried confederations but ran into issues so went with local-as.

3:30am back from work break
3:46am 3.2 complete, 3 points
EGP complete, 6/6, 0:31

MPLS
4:03am 4.1 complete, maybe--te w/o mpls on each interface

Well, after 2.5 hours of working on 4.2, I give up. As far as I can tell, I had everything right 2 hours ago. I ran into an issue where I was somehow missing my loopback 0 ip address, which wouldn't allow me to bring the tunnels up. I added this, and my tunnel through R9 worked fine. but the dynamic tunnel won't come up. I've checked and rechecked my interfaces, but it just won't work.

Saturday, March 21, 2009

IE CoD Service Provider Multicast

Since Multicast is obviously my weakest point now, I've decided to review the IE CoD for Multicast.

1. Don't touch mVPN until each PIM domain has been verified
2. Use debug ip pim to find "RPF lookup failed" messages
3. Enable pim between msdp peers so rp information can be exchanged
4. For multicast testing, getting a ping reply is not a great test. It is only really necessary to verify the downstream traffic is getting to the host. To get icmp replies, it may be necessary to enable multicast routing on the destination and enabling pim there, ensuring it is not the pim dr.
5. mBGP is solely for the purpose of advertising source ips for rpf lookup. Verify using show ip rpf.