I’ve spent most of my adult life assuming that technology was in an endless upward spiral that would always provide me with a never ending supply of a) fast computers, b) cheap storage, and c) massive bandwidth. You can imagine my surprise when I recently discovered that we are apparently on the verge of a global bandwidth shortage. If true, that would certainly change most of my assumptions about the future of media, computing, and civilization as we know it.
My first inkling that we might have a serious problem came last week when a Google representative, speaking at the Cable Europe Congress, announced:
The Web infrastructure, and even Google’s (infrastructure) doesn’t scale. It’s not going to offer the quality of service that consumers expect.
One cable executive called it “the best news of the day”. Why is it that good news for cable companies is always bad news for consumers? And why is it that Google of all companies is just now getting around to breaking this news to us? Seems like they might have mentioned this before they spent a fortune buying YouTube.
I was still pondering how serious this bandwidth shortage might be when I read that a bird flu pandemic could bring the Internet to it’s knees.
How would a flu pandemic affect the Internet? Easy. Remember years ago when we were told that one day we’d all be able to telecommute to our jobs? Despite the fact that most employees now have broadband and a home computer that prediction never really came to pass. Well, it turns out that your boss was just waiting for the plague. Once the bird flu hits we’ll all be expected to telecommute. The current thinking is that a massive influx of telecommuters making VPN connections to their offices would have a substantial impact on available bandwidth. When that happens, it’s goodbye online video. Just don’t forget to clock out at the end of the day.
It wasn’t all that long ago that we were talking about a global bandwidth glut. During the dot com boom companies actually went under by overbuilding infrastructure. So what’s changed? In a word, everything.
P2P applications like BitTorrent are partially to blame. The firm CacheLogic has estimated that P2P applications use between 60 and 80 percent of capacity on consumer ISP networks. Others have suggested that spam comprises 80% of all Internet traffic. Between all of the spam and video file sharing that leaves -40% to -60% of Internet capacity for everything else we do online. Based on those numbers it’s amazing you’re even able to read this article.
Is our dependence on ubiquitous broadband a fatal flaw in our plans for the future, or are these dire warnings overstated? Chances are we won’t really know for sure until something unexpected happens.
In the meantime, the mere thought of a bandwidth shortage probably has the old-line media types jumping for joy. It’s not just the cable operators who would stand to benefit if the Internet collapsed. The broadcast networks would be free to ignore the Internet and go back to business as usual, newspapers and magazines might start seeing subscriber growth, and phone companies could go back to their switched networks and forget about all of this VoIP stuff. In other words, it would be the 1980’s all over again.
And all of those cool new media services that you were planning on using? Turns out they could be a lot less cool and a lot more expensive than you were thinking they might be.
Then again, this could be an elaborate ploy by Google to buy more time to ramp up their IP television service and catch the cable operators off guard when the service is finally ready.
Or not. That’s the worst part of this story. We don’t really know anything for sure.
How far would you be willing to go to ensure there was enough bandwidth to go around? Would you willingly limit your Internet access to a small selection of lower bandwidth applications like email and limited web browsing? Would you stop playing online games? Stop using P2P networks or watching online videos?
Would today’s Internet even be worth using if there were bandwidth limitations?
A genuine bandwidth shortage might spur us to get serious about solving the spam problem, which would in turn do something to alleviate the shortage.
A high percentage of web page related traffic is actually the pictures that go on those pages. Which are nice, but in many cases not necessary. Many websites and blogs could redesign a bit and reduce their bandwidth requirements by half or more.
A real bandwidth shortage could cause more webmasters to get serious about HTTP compression, which has been around for years, and which dramatically reduces the amount of data required to represent the text parts of a web page.
In short, even if bandwidth were to stagnate at today’s levels (and I don’t really think it will), the bandwidth we have could be used a lot more efficiently without having much practical impact on the internet and the way it’s used.
I think this story stinks of terror ! Please goerge w bush of the internet scare the people more !
How about mentioning the fact that the isp have yet to fulfill their commitment to the citizens of these united states and provide faster reliable internet ! How about mentioning the fact that we the people gave these isp companies tax breaks to do this that I have mentioned !
No, instead you wish to blame the people and help further the web2.0 government controlled internet.
You are indeed a moron or are a paid off writer.
Damn Ken, you’re good. How’d you find out about my Republican party affiliation and the fact that I secretly work for Dick Cheney, AT&T, and Halliburton. No one (and I mean NO ONE) else knew about that. Now the world knows. Thanks a lot!
This isn’t so much about instilling terror as it is a hypothetical question that puts together issues raised by a couple of seemingly unrelated stories that I found curious. Some people call this process blogging.
You may notice that I’m perplexed and somewhat skeptical because some of the sources seem to contradict each other (if everyone is right we’re using something like 300% of total internet traffic).
And by the way, my official title is “George W Bush of the Internets”. Note that Internets are always plural.
WI – good answers. I would add that Google and other search engines could easily cut back substantially on their crawling activities. There’s an awful lot of non-human internet traffic that we could do without. The robots might have a different opinion though.
This is ridiculous. There may be a shortage on Google’s resources (which some would easily consider as “the Internet is broken” or “global bandwidth shortage”), because everybody is relying upon their services.
But this has nothing to do with the bandwidth of Internet inter-connections or even customerISP connections. That bird flu pandemic story has the same flaw: Before the bandwidth of “the Internet” would be exhausted, it would be the companies’ servers and THEIR connections that would be overloaded.
Usually when i read about this kind of stuff i’m always left to wonder if it’s really just an overused method of getting away with charging more for nothing. I concur in that i always thought technology was an infinate realm of progression, but i feel that the efforts aren’t ever in the right place. it’s always going to be about who can make more money. and as long as spam makes money, there will always be a way to clog our local switches. But how much data per user is a lot? There are just too many variables to look at, and the service providers’ solution is simply to build bigger. and offer more tier based billing. For me to have to limit my internet usage is absolutely unacceptable. Why is it my problem? Broadband is already over priced, and crippled as it is, so aren’t we already forced to limit our usage?
There’s something not mentioned here. Could it not be the case that the companies that are actually selling (and therefore maintaining) the Internet to both: Google/YouTube and average users, aren’t doing so responsibly? We are only in this position due to extreme overselling and it should not matter whether customers of these services are using P2P or distributing loads of video – these customers are simply paying the rates and using the services they have paid for. The cable companies are struggling to keep up, yet keeping quiet because they don’t want to draw attention to this extreme overselling.
its knees
it’s = it is
its = belonging to it
Now that Kirk has been outed as a Republican, I feel free to present other little-known facts about him. As a matter of fact, here are the Top Ten Things that You Didn’t Know About Kirk Biglione:
10. Once stood in line for a week for Morrissey tickets.
09. Was originally named “Curt,” but changed his name in honor of his favorite TV character – Captain Kirk
08. Thinks that single-malt Scotch “tastes gross.”
07. Made out with Courtney Love in 1984. And again in 1986.
06. Has never actually finished The Illuminatus! Trilogy or Gravitys Rainbow
05. Calls Jim Jarmusch movies “slow and preachy.”
04. Thinks that Jazz isn’t nearly musical enough because “you can’t sing along with it.”
03. Has never actually left his house.
02. Thinks that every word that Stephen Colbert says is true.
And the Number One Thing That You Didn’t Know About Kirk Biglione . . .
01. He really really really really really really really really loves his Zune.
Uh, thanks Pedant. I’m trying to use all the apostrophes I can while I still have the bandwidth.
Jim, you will soon regret posting that list.
Pedant — we do try to get Kirk to take care with apostrophes, but, frankly, he’s recently presented strong evidence to support his theory that spelling is relative. Given that he was right on that point, I am hesitant to correct his posts. Though, in the grander scheme of things, I’m not sure what your point is (unless you have a mission to save the apostrophe from misuse, in which case I totally support your cause, we need more punctuation czars).
Since Kirk began discussing this topic, I’ve been absorbing and thinking, and would remind all the broadband users of the world that…not everyone is a broadband user, and some who are use cable modems that distribute connectivity in a less-than-equal sort of way. Even DSL isn’t as wide open as it seems.
The major companies have not made adequate investment in serious infrastructure — and while I believe it’s in their best interest to do so, I have to be realistic and understand that these companies are more about increasing shareholder price than communications. The Google executive who made the comment about infrastructure (noting this applied to both Google and internet as a whole) made a good point: think about the flow of online traffic when a major power outage happens. Or when a major telephony hub goes offline. Or, horrors!, when a virus attack occurs — massive amounts of data flowing slows us all down.
Sure, maybe the hypothetical posed by Kirk may never happen. Or maybe it will only happen in certain regions. But shouldn’t we start anticipating massive demand (the media companies certainly are) and insisting that we, the consumer and taxpayer, be provided with the biggest pipes we can?
Interesting topic here. Some aspects of a pandemic that are not considered here. When the first stream of rumours is coming about a flu that is making a lot of victims, there indeed will be a run on information at the Internets. But very soon people will decide not to turn up at work and we will run out of basic commodities like electricity, food and security. So a lot of internet users won’t be able to get their computers switched on and I am curious about the way providers are going to handle this situation and whether they feel any responsibility by then to do as much as they can.
I can also imagine some government taskforce produces a virus in order to select a certain group of users who are allowed to continue the use of the Internet while others are not.
Bit paranoid maybe, but life is hard and then you die.
Time to start buying stocks of the basic internet cable companies.
Western Infedels why are you looking at dropping pictures from websites. I mean, web traffic is like
Time to start buying stocks of the basic internet cable companies.
Western Infedels why are you looking at dropping pictures from websites. I mean, web traffic is like less then 10% of all, try decreasing on the greater usage parts of the net, changing http from “9%” to “7%” is not worth the trouble…
How much of all traffic is legal?
Current protocols and physical links are fairly inefficent and can’t operate effectively when data throughput gets to a high percentage of possible peak bandwidth. As usual to get a bit more data down the pipe they put the data rate up by 10X and move the bottleneck.
More efficient use of bandwidth, charging all net users for true bandwidth usage, making better use of off peak downloads, charging for data quality including data rate, block size, jitter, delay, etc.
As we approach the online video age the real problems are being overlooked, bandaids don’t work in the long run. Google is putting massive server farms all over the world and now owns a LOT of total bandwidth. But the total end to end solution is not being resolved and can’t with current technology.
I agree about Spam and BitTorrent using up bandwidth. The spam problem must be solved, hopefully as people ditch their XP machines and move to Vista we will see less machines infected with spam sending viruses. Fingures crossed. As for bit torrent, is it the technology of the usage that you find wrong? Are you’re saying that people downloading large videoes is wrong? Or are you saying that the method of splitting files into small bits between thousands of users is wrong.
Also, where do you mention usenet? Most ISPs Usenet servers have terabyes of crap pass through them daily, most of it usually available on BitTorrent networks too.
Action Plan:
– get rid of spam
– limit bit torent useage to off peak times
– ban flash!
– get ISPs to look again at what binary newsgroups they carry.
– encourage people to upload large files to a web site and link to them, rather than CC an email with a 10MB (+ encoding) file to 20 people
Just mt 3 cents worth
Marc
marc, you are moron.
do you really think that vista is going to solve problems with virus infections? how, really? you are listening to bill gates way too much.
Well… Comcast’s response to this has been to disconnect HSI users a month after a single phone call. That’s it. And if you ask for a number of how much you “should” use they simply tell you they can’t and you should just use the internet less. I’ve run into this problem. While there isn’t a limit with my “unlimited use for a flat monthly fee” account, Comcast has been aggresively disconnecting HSI users and give absured numbers to describe activities nobody would do (unless you are a spammer).
Suggesting a disconnected HSI user sent 13 million emails that month or 256,000 photos is rediculous. Who talks like that when describing bandwidth? Other companies such as Cox Communications state you can sign up for an account and download 40 Gigs a month, 60 Gigs and so on. This is something that makes more sense to people in general. (yes, some people don’t know what a gig is, I know ::grinz::)
Anyway, I maintain a blog with my personal experience. I’m hoping to get the word out and encourage competition as I believe it would reduce the likelyhood a Company such as Comcast from simply booting HSI users as quickly. My blog is at http://comcastissue.blogspot.com and I’ve documented my experience as accurately as I can.
I kind of disagree. Besides bandwidth, you have to take in consideration the compression improvements of the last couple of years. This same advantages were the ones that allowed us to have video with good audio and video quality over the internet.
Regards
Here’s another scary, extra paranoid thought.
If there was a global bandwidth shortage, what would then stop the government from declaring it a “scare public resource” and using that rationale to try and put the internet under FCC regulation?
Every commenter above has proven their dismay at the fact that they really don’t have control.
The web needs a minor face-lift with regards to who has control of the value of my personal digital contact points.
Here’s an example: Let’s say you’re a friendly person – as are most. You have a big driveway near a sports arena – which most don’t! So, when a game is on you let your friends park at your place – they’re elated – you rock! BUT, enter the dark side – what would happen if you let AOL control who parks in your yard? A) You can be assured at the next game there would be no room for your friends B) You can also bet that vehicles with billboard ads on them will also start to turn up in your yard, and finally C) Because its first come first serve you can be sure that the guy pedaling sexual enhancement products will sneak in around 3am daunting a big plastic model on his roof.
Property is property. Whether it’s your home, your car, your website, your inbox, your VoIP number, or your attention – It’s all about who controls your personal value.
Thanks for that thought Jim. We all know Ted Stevens is going to be named the official Internet Czar shortly. Turns out it’s going to be one of those lifetime positions.
Frank, your story scares me. Legitimate services like VoIP, iTunes, Amazon Unbox, and just about everything else every major corporation has planned for the 21st century, requires lots of bandwidth. I wonder if they’d be so quick to cut you off if you were paying for Comcast VoIP? You could always claim you just make a lot of phone calls.
Marc, I’m not saying BitTorrent is good or bad, just noting that others have said that the sharing of large video files may be part of the problem (if there really is a problem). BitTorrent is a great way to distribute large files and there are many legitimate uses for that technology. Also, I’m with Shamus. Windows Vista isn’t going to solve any of our problems.
I’ve a comment on Argonaut’s statement regarding compression – I was in the fiber business during the bubble. Hundreds of millions were invested in laying fiber at that time and then the collapse came. Now get this – All the money that was invested in the installation of this fiber was lost in the bankruptcy courts. Then, the fiber laid dark for a few years. Then, savvy investors came along and started picking up for pennies on the dollar and turns it on.
Believe me – Since the collapse of the bubble there have been millions of miles of dark fiber lit – in fact, over the past 2 yrs Google has been the largest investor of dark fiber in the country. – Now go back to my original post and think “value”. Why would Google purchase all this fiber? What is Google expert at when it comes to generating revenue?
So Frank, what do you think Google is up to? Are they buying up bandwidth to ensure they have enough for future service offerings? Or do you think they’ll ultimately get into the bandwidth business (arguably they’re already in that business).
And are statements like the one made during the European Cable Congress part of a disinformation campaign?
Interesting dilemma, indeed!
IMHO, the NAP’S that control the core backbone in the USA should quickly upgrade routers to 10G and scale to 100G as the IEEE ratifies the new standard (& as hardware becomes available from the likes of CISCO, nortel, Juniper, etc.)
10GB pipes already exist in sizeable numbers and it is critical for the backbone to grow accordingly. This does take a $h1tlo@d of $$$$$, however. The end user will end up coughing up more bucks to cover the necessary infrastructure upgrades; passed along to ISP’S and thru taxes.
As long as bottlenecks occur below or at the ISP layers (not core backbone) things will work out.
People won’t use less: we NEED more as a whole! But its’ cool…ISP’S can limit your throughput and even block bandwidth intensive traffic. It’s written in SLA’s & AUP’s already…. Let’s hope it doesn’t come to regulation…
Kirk,
Google is the king of monetizing the value of personal information. Every privacy statement written by them allows for the connecting of information between various searches, portals and applications. They then use that information to create a very valuable keyword-bidding offering. At last look they were generating about $4M/day. I’m certain it’s more than that now.
Now, look at the history. Google has played a key part in the roll out of WiFi in San-Francisco. They give you free WiFi in exchange for access to your surfed data. (You can only access the Google WiFi net via their encrypted plug-in. – they assure your information is private, at least until it hits their servers.) They then tuck that data away with everything else they know about you. Also, Google WiFi, at some point, will include Google Talk.
My guess is that they will start opening up free WiFi in all major metro areas.
Think about it, in 2006 nearly $400B was spent in the US on reaching the consumer with advertising. There are only, what… 350M Americans?
The consumer simply has no idea just how valuable their information is!
ryan,
That’s my point – regarding giving the consumer Personal Value Control. Then ISPs won’t regulate you they will work hard to give you what you want.
Think of it like this – NBC works hard to give you what they want because they don’t want you to switch to CBS. And the local broadcaster does the same.
If your email address had increasing value (and it no doubt would if it were under your control), and the ISP were a part of that value chain (which I am sure you would let them be – for a price ;-), then the ISP would walk over hot coals to keep you happy. You want more bandwidth, you got it! You want more features – we’re working on it! etc etc etc….(Instead of now – “The average hold time is 37 minutes – but you are a valued customer”.
I belive that even if this happens, we’ll be able to relay in new and more effective compression technologies, wich in the end would let us do the same thing we do now, but with less bandwidth.
It would be nice though if we didn’t have to gio through something like this to get our hands on compression methods that already exist.
I work for an ISP where we have seen around 80% of traffic being bitttorent or file sharing. The way around this is we have implementeed traffic shapers to limit bittorrent and other severe offenders to essentially a scavenger class of service. This allows other real time traffic to get through. We actually don’t base our usage percentage on file sharing traffic due to the fact that it is not cost effective and would eat up any amount of bandwidth we threw at it (mostly literally speaking, but I am sure there are limits). Supply and demand work for bandwidth just like anything, so this is holy a false or at least a temporary problem.
Many pay on per MB basis around the world, so yes, the Internet is worth using if there were bandwidth limitations. The world will never run out of bandwidth because bandwidth providers will just start increasing prices, which in turn, will lead to more economic usage of the bandwidth. Just like the gas prices and its shortage.
T1 Service
This may not be only a scare tactic. There could be some basis behind the accusation that the Internet would come to its knees during a bird flu epidemic. I agree with some of the comments that a bird flu epidemic would not bring down the Internet. Its just that it could be entirely possible that a surge if Internet traffic from another source could bring the Internet tumbling down. The Internet is already being strained by the new in flux of Internet video media and the large bandwidth usage of VoIP and P2P applications. If for some reason these were to spike, I could see the performance of the Internet dropping drastically as the ISPs struggle to meet the needs of its users.
I would attempt to at least alleviate it by developing a mandatory program that runs a peer to peer layer of the internet, causing a sort of heat death, so to speak. All the lines that are hardly used by clients compared to the use server lines use will be activated and equal out the flow of information by taking information from peers as often as possible.
This is because big corporations want to keep customers but not drop the money to upgrade equipment and lines. If google can do it so can the big boys. They are already Throttling customers speeds and limiting there bandwidth. Comcast limits customers to 250 gigs a month. The Solution is upgrade your service, start designing websites based on compression and when making files available to the public zip them up. Compression is not the devil it is an internet saint.