This has got to stop [telecom]

Nothing going on there is even within the FCC's actual jurisdiction, let alone in violation of any rule. After all, this is about content distribution, not telecom. Netflix doesn't own wire and doesn't even want to rent any. It's just servers in data centers, a big computing and content-leasing company. There is always the possibility of an antitrust issue, but what is VZ doing that competes with Netflix?

The problem with Netflix is that they assume that the cost of Internet bandwidth is *precisely* zero. They come across this because the retail price of wireline bandwidth -- that is, what most ISPs charge home subscribers per byte transferred -- is not metered, and thus zero. To be sure, for most ISPs (urban cable and telco), the cost per bit has been so low that it really wasn't worth fretting about. The cost of providing service was in the fixed physical plant, not the usage.

But Netflix takes that "too small to meter" cost, which turns out to be not quite zero after all, and multiplies it many fold. It puts so much load on the network that it alone can force an upgrade of routers, an increase in the number of switching systems or multiplexing nodes. And it does this via streaming, which is relatively inelastic, compared at least to TCP, which the network was after all designed around.

TCP uses capacity elastically: Dropped packets cause it to slow down, creating an endless sawtooth pattern of end-to-end rates. So it can adapt from dial-up speed to gigabit fiber. Streaming however needs a minimum speed. Netflix, to their credit, does attempt to monitor the connection and regularly adjust its streaming rate, and video quality, to what works. But it still imposes a massive load on ISPs, which can displace or degrade other applications.

So ISPs naturally want to be compensated for the load. And Netflix doesn't want to pay. Netflix thinks its content is so groovy that ISPs should be willing to upgrade their networks to accommodate its demands. ISPs think otherwise.

This is the whole point of the traditional Internet business model -- it is a voluntary agreement to exchange traffic for mutual benefit. And mutual benefit often involves money to create that mutual balance. Frankly I think the cable companies should be required to let Netflix rent space on their Video on Demand servers, so that users can watch cable shows the way their choice of diety intended them to. And then the Internet can do the stuff that only it can do, not waste the bulk of its capacity on TV. But it isn't working out that way. Netflix is thus trying to use regulatory pressure to change the Internet's business model to favor it. They want the Internet's wholesale prices to be regulated so that TV distributors pay ISPs nothing. And once the FCC starts regulating that, who knows what other aspects of crazy telecom rate regulation will show up? Sorry, but I'd rather let Netflix play by the Internet's rules than make the Internet kowtow to it.

Please do not use the word "invalid" in domain names. The Majordomo server which distributes this email reacts to "invalid" by removing the sender's name from the email.

It's OK to obfuscate your email address (see the F.A.Q.), but putting "invalid" into your domain name causes more problems than it solves.

Bill Horne Moderator

Horse hockey.

Netflix is not using Verizon's bandwidth. Verizon's own subscribers -- who have already paid for that bandwidth -- are using what is theirs, and Verizon is only dragging Netflix's name into the fray to distract us all from the simple fact that Verizon is trying to cheat its paying customers.

What this tells me is that there is too little competition in the telecom industry -- otherwise Verizon would never dare try to throttle any site -- and the feds not only should be vetoing the pending ComCast merger and any more like it, they should be breaking up the existing giants, especially Verizon and AT&T, into smaller companies once more.

This is one reason VPN providers are becoming so popular.

Stick your NetFlix in a VPN tunnel and most times you eliminate those pesky problems like interruptions.

There are free as well as paid VPN providers on the market making it possible to test with a free VPN provider and later switching to a paid one should you want to spend some cash or get different "pay to play" features.

Check out the article and especially the video at this link for an example of NetFlix over a VPN on a Verizon feed.

I agree it's only a work around but it is one way to get what you want with out dealing with Verizon's crap.

Um, no. Netflix is a content-distribution network. (To the extent that there are "servers in data centers", Netflix leases them by the hour from Amazon.) In some locations they use third-party CDNs, but their big push over the past few years has been to transition this service to their own platform (thereby cutting out middlemen like Akamai).

Verizon is a big cable-TV MSO, in case you haven't noticed. Their video (and VOD) business competes *directly* with Netflix.

Wrong again. They assume that the cost of Internet bandwidth *within a provider's network* is the responsibility of that provider, to be recovered by the ISP directly from the ISP's customers. They are of course willing to interconnect with providers, at locations of the providers' choosing, when that would be mutually beneficial. They are even willing to colocate their content servers on a provider's network

-- that is, after all, what a CDN's business is -- when doing so would provide better service to their customers.

Depends on the congestion-control algorithm you use, but accurate to a first approximation.

No, Netflix understands that its service is one of the principal drivers for the ISPs' customers to upgrade their service to faster, more expensive tiers, and therefore thinks it reasonable to expect the ISPs to invest some of this additional revenue into upgrading their backhaul networks. The ISPs, on the other hand, have seen the upsell fees as essentially free profit, which they can easily gull from customers who don't realize that the speed of their access link has no connection with the speed of the transit network, and who don't read the contracts that say the provider is under no obligation to engineer its transit network in a way that allows it to deliver that bandwidth, not even statistically.

Netflix is doing the public a service by standing up on this principle. They don't have to do so, and arguably it would benefit them considerably in the long term to cave -- because that would make "the next Netflix" a near-impossibility without a huge up-front investment in bribes to access providers, permanently locking out all but the most serious competition. Which is of course exactly what the cable duopoly would like: they realize they can't turn back the clock and crush Netflix in infancy, but they can certainly do so for anyone else who might want to compete with them.

Unlike most people, where I live, I have the choice of three cable MSOs: Comcast, Verizon, and RCN. If I actually cared about Netflix (I'm not a subscriber) I would certainly consider the quality of service offered by each provider for Netflix streaming in my decision as to whose Internet service to buy. Most people don't have that option, which is presumably why Netflix did a deal with Comcast but is holding out for better terms from Verizon -- most of Comcast's customers are in monopoly territory, whereas all of Verizon's FiOS customers have the choice of another cableco (often Comcast).

-GAWollman

Of course they could charge by the bit, but people wouldn't like that either.

So while you pay for the peak, the price is based on a low average, and if the average rises, somebody has to pay. Better Netflix than the rest of us who don't watch it often.

Then the problem is a conflict of interest, and VZ should be forced to sell off either its content-resale business, or its telecom business.

Indeed, a general rule that no firm may be in both businesses is probably the best way to handle the antitrust issue across the board. Let the data carriers divide themselves into common and private carriers as the passenger- and freight-transportation industries have done.

What I said, servers in data centers, whether their own or leased. They don't own transmission facilities (telecommunications).

To the extent that they are protecting their MSO business using their telecom monopoly resources, then there could be an antitrust issue. Verizon doesn't own significant content the way some companies do, though. Not that Netflix does much either.

Which is based on the assumption that it's zero. Providers such as Verizon have costs, even for distribution, so an application that makes a sea change in those costs should pony up, because otherwise those of Verizon's customers who don't want to be IP couch potatoes will have to pay the added costs forced on them by a service they don't use. That strikes me as a valid business decision to leave to Verizon.

This only applies to large providers. I know of many smaller ISPs who couldn't get Netflix into their own networks on a dare or via a bribe. And those are the ones with the highest upstream costs.

Unfortunately, the retail wireline ISP market grew up here charging for burst rate, not total carried load. Netflix doesn't need 100 Mbps; it works fine on a 10 Mbps circuit. So the usual provider tiering doesn't really come into play here except at the bottom of the ladder (ADSL). Even the bottom cable modem tier is plenty fast for Netflix in HD.

That assumes that the beneficial innovations are the ones that need gobs of bits, like Netflix, rather than having fewer, more valuable, bits. Instead, real innovation, of the type that needs low latency or low loss, may be crowded out in a sea of cat videos.

We are, I presume, neighbors; I have the same three choices. I do sometimes watch Netflix, though they don't have much quality streaming other than a couple of TV shows (like Orange is the New Black). Their online movie choice is pathetic, no doubt due to licensing problems. So I'm likely to dump Comcast for RCN not because of Netflix (I suspect Comcast now has better peering with them) but because Comcast's customer service quality has deteriorated over the past few years from "fair to middlin', usually OK" to "remember Chrysler?". I may post an article about that one of these days, but what I've written on it so far is, well, bigger than most Digest posts.

Fred Goldstein wrote: :On 8/18/2014 9:24 PM, Garrett Wollman wrote: :> In article , :> Fred Goldstein wrote: :>> Nothing going on there is even within the FCC's actual jurisdiction, let :>> alone in violation of any rule. After all, this is about content :>> distribution, not telecom. Netflix doesn't own wire and doesn't even :>> want to rent any. It's just servers in data centers, a big computing and :>> content-leasing company. :> :> Um, no. Netflix is a content-distribution network. (To the extent :> that there are "servers in data centers", Netflix leases them by the :> hour from Amazon.) In some locations they use third-party CDNs, but :> their big push over the past few years has been to transition this :> service to their own platform (thereby cutting out middlemen like :> Akamai). :>

:What I said, servers in data centers, whether their own or leased. They :don't own transmission facilities (telecommunications).

Netflix does own (or more likely, has leased, but the difference doesn't matter to us) quite a lot of fiber.

:> Wrong again. They assume that the cost of Internet bandwidth :> *within a provider's network* is the responsibility of that :> provider, to be recovered by the ISP directly from the :> ISP's customers

:Which is based on the assumption that it's zero. Providers such as :Verizon have costs, even for distribution, so an application that :makes a sea change in those costs should pony up, because otherwise :those of Verizon's customers who don't want to be IP couch potatoes :will have to pay the added costs forced on them by a service they :don't use. That strikes me as a valid business decision to leave to :Verizon.

Verizon, comcast, et al, have internal networks that are capable of transporting the amount of data they're selling to their customers. They have made that information available publically, sometimes in a weird effort to blame netflix. In some places, there may be busy hour congestion, but there's not much of it, and it's not a real problem for most customers. Netflix is quite willing to pay the costs of getting the bits to the edge of Verizon network, Verizon just wants to get paid twice to deliver them.

:This only applies to large providers. I know of many smaller ISPs who :couldn't get Netflix into their own networks on a dare or via a bribe. :And those are the ones with the highest upstream costs.

Totally false. Netflix is willing to peer with anyone who meets certain technical requirements, and who is located in one of the (large number) of places they're located in. If you don't meet the technical requirements, your not a real network, and you probably aren't peering with anyone. It's very much in netflix's (and other people who deliver big streams of data to end customers) interest to peer with anyone who asks.

...

"The places they're" ... *already* ... "located in"? That does a lot of good (not) for rural providers, who are the ones that can't get Netflix except via backbone. For these ISPs, many of whom depend on fixed wireless, backbone connections are outrageously expensive due to middle mile -- the link from their area to a backbone node -- costs. The middle mile often has to come from the ILEC, at outrageous rates, since there isn't much competition to rural areas, unless the stimulus build (BTOP) added some.

Yet the NN crowd proposes rules that would require wireless network operators in Wyoming or Maine or rural upstate NY to pay for the capacity to carry however much TV their subscribers might want to watch. You can denigrate them as "not real networks" but they're the only Internet access their customers have. For TV, they at least have satellite, but that's lousy for Internet. It's not the simple cases like Verizon FiOS that should determine the rules, it's the tough ones.

Slightly revised to clarify a quote

Bill Horne Moderator

Per Fred Goldstein:

Wouldn't the only diff be that, with NN, they would need to explicitly bill the user? i.e. set up plans similar to cell phone data plans where the users who use more pay more... and maybe expanded to also account for bandwidth used?

In article you write:

That assumes the bandwidth is there. Wireless systems are not like fiber systems. With fiber, you have practically unlimited internal bandwidth, while with rural wireless, adding capacity costs real money. Having participated in a local project to find money to build out fixed wireless to the hilly rural parts of the county not covered by the urban cable systems, I can say that it is very unlikely that the customers would be willing to pay the real costs for the amount of bandwidth they'd use, and even so, a couple of high bandwidth users could suck up all the bandwidth in their part of the network and lock everyone else out.

R's, John

From: Fred Goldstein

If you run an ISP and you signed up your customers by promising them "Unlimited Internet", then you find out that it costs you more to provide that service than you had expected, because they're watching Netflix on three TVs while they're running 35 simultaneous torrents of pr0n and they're watching 10-hour YouTube videos on seven different computers behind their NAT, then guess what? You are the one who miscalculated.

Sucks to be you.

If you signed your customers up with a TOS (which nobody reads) that says you can throttle their speeds after some point which is entirely up to you to set, then guess what? They should have read the TOS.

Sucks to be them.

Why is any of this controversial?

There's a fine line between "unexpected" and "unacceptable" traffic, so please recall that many previous start-up companies, such as Google, took advantage of the common resource that is the Internet, without receiving demands that /they/ contribute more to pay for the physical layer. End users did not receive unexpected charges from ISPs because they used Napster to download MP3 files, nor when they downloaded Windows updates, Adobe Acrobat updates, Symantec Anti-virus updates, or McAfee software.

In other words, if streaming content is an issue of /degree/ instead of /type/, then I must ask if you feel ILECs are entitled to limit the /amount/ of such traffic so as to keep adequate bandwidth available for the other services the ILEC's users are paying to have available, such as CNN, or MSNBC, etc.

Why? Because "I" designed a system with the best information available to me at the time? Your logic infers a belief that every driver should be forced to buy a semi-tractor as a family car, since they are more crashworthy than a compact sedan, but that's not the way we do things: both companies and consumers take risks in accordance with their own estimates of value and costs.

Well, that's not always reasonable or workable. If "they" didn't read the "TOS" in their flood insurance contracts, then "they" are without food, medicine, or homes the day after hurricane Katrina left New Orleans and Mississippi.

Except, it doesn't work that way.

Some follow-on events are too big to predict, and that means that responsible adults must do their best to adapt to an unforseen (and, I submit, unforseeable) situation that demands large-scale adjustments to the underlying paradigm.

Because limiting the bandwidth available to streaming content is a major shift in the business model previously available to entre- preneurs offering a product or service over the Internet: that of "end user pays" for *all* transport costs. However, nothing happens in a vacuum: we all know that NetFlix and other streaming video services have been more successful than most predicted. That success, however, does not entitle NetFlix to externalize more of its costs onto Internet users whom don't subscribe to its service.

I'd like to clarify this debate, so please answer these questions:

1. Verizon's voice networks are designed for "busy-hour" *business* loads: assuming the data networks are also able to carry that kind of load, do you feel that streaming content providers should, or should not, be allowed to make use of that capacity (during off-hours) without paying more?

If not, is Verizon, or /any/ other ISP, entitled to demand ongoing direct revenue from streaming providers, only a one-time payment to cover the upgrade costs, a regulated rate-of-return income that tracks bandwidth use attributable to streaming content, or nothing? What about network-of-networks players such as Layer 3?

1. Should Verizons be subject to "must carry" rules when streaming content providers use a system (the Internet) that was originally equipped with 50 kilobit-per-second links, to deliver multi-gigabit-per-second content, for profit? After all, Verizon shareholders had to pay to construct whatever capability is currently available, even if it was intended for a different class of traffic: should Verizon's investors be required to pay for upgrades and maintenance to pipes which most of Verizon's users will never employ?
2. How do you feel Verizon should be entitled to view the physical layer of the network? Does a regulated common carrier have an obligation to design, provision, construct, and maintain a Terabyte-per-second fiber link to every home or office, on the grounds that nobody else was ever asked to contribute to the costs before? Bill

Because the number of ISPs, especially high-speed ones, on the market is still so small that there's not enough competition to keep TOSes fair. And that situation, as long as it persists, means there needs to be continued regulation, both of the anti-predation and antitrust kinds.

But the problem will solve itself in time, *if* regulation is done in a way that doesn't prevent new companies from entering the field (nor prevent even the existing large companies from building into each other's territories and competing with each other).