Least Sucky Backbone Provider

Good morning,
	I'm considering dropping Cogent completely out of my
transit mix, as
the number of outages and problems they have been
experienced over the past
year has reached an unacceptable level. It has gotten to the
point that we
their BGP session is shutdown for longer periods than it is
on. Based on the
availability of on-net fiber in my facility, I have narrowed
the field to
the following candidates:

1. Level 3
2. MCI/Verizon
3. AT&T

I'm looking for comments from actual customers of the above
providers in
relation to;

1. Network reliability and performance
2. Responsiveness to outages
3. Proactive notification of network maintenance

95% of our traffic mix is US48 in nature, so International
routes are not a
huge decision point.

We had the same issues with Cogent .. I feel your pain...

level(3) has always been good for us - very few issues and
their support
has been great from our perspective.

MCI/Verizon did not work well for us at all - their network
was solid
and customer service wasn't too bad ... our problem was that
less than
20% of our traffic was preferred via MCI's routes.  Funny
how one of the
largest networks in the world was the least attractive BGP
wise....   of
course everyone's network is different with every provider
so YMMV....

Never dealt with AT&T as we're based in Canada...;)

Paul


-----Original Message-----
From: owner-nanogmerit.edu [mailto:owner-nanogmerit.edu] On Behalf Of
Gregory Boehnlein
Sent: Monday, November 05, 2007 10:52 AM
To: nanogmerit.edu
Subject: Least Sucky Backbone Provider


Good morning,
	I'm considering dropping Cogent completely out of my
transit
mix, as the number of outages and problems they have been
experienced
over the past year has reached an unacceptable level. It has
gotten to
the point that we their BGP session is shutdown for longer
periods than
it is on. Based on the availability of on-net fiber in my
facility, I
have narrowed the field to the following candidates:

1. Level 3
2. MCI/Verizon
3. AT&T

I'm looking for comments from actual customers of the above
providers in
relation to;

1. Network reliability and performance
2. Responsiveness to outages
3. Proactive notification of network maintenance

95% of our traffic mix is US48 in nature, so International
routes are
not a huge decision point.









------------------------------------------------------------
----------------

"The information transmitted is intended only for the
person or entity to which it is addressed and contains
confidential and/or privileged material. If you received
this in error, please contact the sender immediately and
then destroy this transmission, including all attachments,
without copying, distributing or disclosing same. Thank
you."

Have only had experience of Level3 & MCI/Verizon in the
UK

I prefer Level3 due to the following...

Scale of the Network
Host lots of big content providers across the Globe
Very few outages (1 in 12months) on the UK backbone
Customer support was very good
Always an account manager to assist with any issues.

Hope this helps


Stephen Bailey - Lead Technical Services Specialist
IS Network Services
FUJITSU Services

Fujitsu Services Limited, Registered in England no 96056,
Registered
Office 22 Baker Street, bond, W1U 3BW

This e-mail is only for the use of its intended recipient. 
Its contents
are subject to a duty of confidence and may be privileged. 
Fujitsu
Services does not guarantee that this e-mail has not been
intercepted
and amended or that it is virus-free.

-----Original Message-----
From: owner-nanogmerit.edu [mailto:owner-nanogmerit.edu] On Behalf Of
Paul Stewart
Sent: 05 November 2007 16:04
To: Gregory Boehnlein; nanogmerit.edu
Subject: RE: Least Sucky Backbone Provider


We had the same issues with Cogent .. I feel your pain...

level(3) has always been good for us - very few issues and
their support
has been great from our perspective.

MCI/Verizon did not work well for us at all - their network
was solid
and customer service wasn't too bad ... our problem was that
less than
20% of our traffic was preferred via MCI's routes.  Funny
how one of the
largest networks in the world was the least attractive BGP
wise....   of
course everyone's network is different with every provider
so YMMV....

Never dealt with AT&T as we're based in Canada...;)

Paul
 

-----Original Message-----
From: owner-nanogmerit.edu [mailto:owner-nanogmerit.edu] On Behalf Of
Gregory Boehnlein
Sent: Monday, November 05, 2007 10:52 AM
To: nanogmerit.edu
Subject: Least Sucky Backbone Provider


Good morning,
	I'm considering dropping Cogent completely out of my
transit
mix, as the number of outages and problems they have been
experienced
over the past year has reached an unacceptable level. It has
gotten to
the point that we their BGP session is shutdown for longer
periods than
it is on. Based on the availability of on-net fiber in my
facility, I
have narrowed the field to the following candidates:

1. Level 3
2. MCI/Verizon
3. AT&T

I'm looking for comments from actual customers of the above
providers in
relation to;

1. Network reliability and performance
2. Responsiveness to outages
3. Proactive notification of network maintenance

95% of our traffic mix is US48 in nature, so International
routes are
not a huge decision point.







 

------------------------------------------------------------
------------
----

"The information transmitted is intended only for the
person or entity
to which it is addressed and contains confidential and/or
privileged
material. If you received this in error, please contact the
sender
immediately and then destroy this transmission, including
all
attachments, without copying, distributing or disclosing
same. Thank
you."

On Nov 5, 2007, at 9:51 AM, Gregory Boehnlein wrote:

> 	I'm considering dropping Cogent completely out of my
transit mix, as
> the number of outages and problems they have been
experienced over  
> the past
> year has reached an unacceptable level. It has gotten
to the point  
> that we
> their BGP session is shutdown for longer periods than
it is on.

There can still be problems over peering links between your
new Stable  
Carrier and some Unstable Carrier.  If such a problem
affects your  
traffic, then you may want to tell your carrier not to
advertise your  
prefixes to the Unstable Carrier at all, which you can do
using  
redistribution/traffic engineering communities.  If your
Stable  
Carrier doesn't support these, and your tolerance for pain
is less  
than theirs, then you may even have to shut down your
connections to  
Stable Carrier to keep Unstable Carrier's problem from
affecting your  
own customers.

So who supports these communities?  A cursory reading of 
www.onesc.net/communities 
  says:

-L3 supports TE-community-based prepends to all its peers
(?)
-Savvis to Tier-1s plus Cogent, XO, Telia
-GBLX to Tier-1s plus Cogent, XO, Telia
-Telia Sonera to Tier-1s plus Cogent, XO, TWTC
-Qwest to Tier-1s plus XO
-Sprint to Tier-1s
-XO to some Tier-1s (lacks Qwest, NTT)

Networks which may not support redist/TE communities:

Verizon
NTT/Verio
AT&T
TWTC
Cogent

(let me know if I'm in error)

These communities are useful when there are networks that 
(A) are  
unstable, and (B) carry/send important traffic to your
network.   I'd  
look for support of traffic engineering via redist.
communities when   
shopping for a "stable" carrier.


Bradley

On 2007-11-05-10:51:58, Gregory Boehnlein <daminnacs.net> wrote:
> I'm considering dropping Cogent completely [...]

Always a good idea.

> 1. Level 3
> 2. MCI/Verizon
> 3. AT&T
> 
> I'm looking for comments from actual customers of the
above providers in
> relation to;
> 
> 1. Network reliability and performance

As Vijay reminds us time and time again, engineering a
large,
reliable, network isn't particularly difficult these days. 
Indeed,
none of the candidates you name above suffer from major
reliability
problems.

> 2. Responsiveness to outages
> 3. Proactive notification of network maintenance

All large providers lack in these areas, some more than
others.  Even
with preferred support, it's not uncommon to get asked if
you get dial
tone on your OC-48, or if 10GE is "like a T1" -- I
do, weekly.  Plan
accordingly.  

With that in mind, key differentiators I'd focus on when
selecting a
transit provider include provisioning intervals,
tools/automation,
routing policy/feature support, and reachability to specific
ASNs.

I'd summarize the above vendors as follows.  Please forgive
the
rambling, and if you deem any of this off topic, kindly hit
the 'd'
key and spare us the chatter.  (Me personally, I consider
vendor
reviews and pseudo-arch discussions like this fascinating
and acutely
on-topic, though I can see where others may disagree...)

Level(3) (AS 3356, not legacy Wiltel, Broadwing): All in
all,
thoroughly "gets it".  Robust implementation of
inbound and outbound
BGP communities; prefix-list auto-generation off IRR;
working
blackhole community; IPv6 support, though tunneled.  Support
folk are
smarter than average; provisioning times are slower than
average.
Large collection of "eyeball" customers.

Verizon Business (AS 701, formerly UUNET, MCI, et al): Solid
as a
rock, though beginning to show its age.  Supports a
blackhole
community (kudos to cmorrow, et al, for setting the trend
there),
though few/coarse others outbound.  No inbound communities;
1995
called and asked for its as-path filters back .  Older
equipment
(Juniper M40, Cisco 12008 w/ E0-E3 cards, ...) is still
common in the
edge, thus availability of 10GE customer ports is sparse
outside of
specific hotels.  Presents frequently on, but is not yet
equipped to
offer, IPv6 customer connectivity.  Significant eyeball
base,
specifically Verizon DSL and FTTx customers.

AT&T (AS 7018): Solid connectivity and architecture;
sharp folk who
are also active in the NANOG community (tscholl, ren, jayb,
...).
Significant eyeball base as represented by AT&T (SBC,
Ameritech,
BellSouth) DSL/FTTx customers and various cable MSOs, though
the
latter is slowly dwindling.  With that said, it is important
to
realize that their commodity IP product is tailored towards
enterprises with leased lines, not your typical NANOG/SP
demographic.
Accordingly, some friendly advice here would be to lay out
your
specific requirements (wrt communities, prefix listing,
source address
verification, IP ACLs, dampening, ...) as a part of the
contract/RFP
process, lest you might find yourself frustrated by various
defaults.

HTH,
-a (speaking on behalf of himself only)