Revealed: ISPs Already Violating Net Neutrality To Block Encryption And Make Everyone Less Safe Online

from the scary-news dept

One of the most frequent refrains from the big broadband players and their friends who are fighting against net neutrality rules is that there's no evidence that ISPs have been abusing a lack of net neutrality rules in the past, so why would they start now? That does ignore multiple instances of violations in the past, but in combing through the comments submitted to the FCC concerning net neutrality, we came across one very interesting one that actually makes some rather stunning revelations about the ways in which ISPs are currently violating net neutrality/open internet principles in a way designed to block encryption and thus make everyone a lot less secure. The filing comes from VPN company Golden Frog and discusses "two recent examples that show that users are not receiving the open, neutral, and uninterrupted service to which the Commission says they are entitled."

The first example you may have actually heard about. It got some attention back in July, when entrepreneur Colin Nederkoorn released a video showing how Verizon was throttling his Netflix connection, which was made obvious when he logged into a VPN and suddenly his Netflix wasn't stuttering and the throughput was much higher. That video got a lot of attention (over half a million views) and highlighted the nature of the interconnection fight in which Verizon is purposely allowing Netflix streams coming via Level 3 to clog. As most people recognize, in a normal scenario, using a VPN should actually slow down your connection somewhat thanks to the additional encryption. However, the fact that it massively sped up the Netflix connection shows just how much is being throttled when Verizon knows it's Netflix traffic. Nederkoorn actually was using Golden Frog's VyprVPN in that video, so it actually makes Golden Frog look good -- but the company notes that it really shows one way in which "internet access providers are 'mismanaging' their networks to their own users' detriment."

But the second example Golden Frog provides is much scarier and much more pernicious, and it has received almost no attention.
In the second instance, Golden Frog shows that a wireless broadband Internet access provider is interfering with its users’ ability to encrypt their SMTP email traffic. This broadband provider is overwriting the content of users’ communications and actively blocking STARTTLS encryption. This is a man-in-the-middle attack that prevents customers from using the applications of their choosing and directly prevents users from protecting their privacy.
They demonstrate this with the following graphic:
This is scary. If ISPs are actively trying to block the use of encryption, it shows how they might seek to block the use of VPNs and other important security protection measures, leaving all of us less safe. Golden Frog provides more details of what's happening in this case:
Golden Frog performed tests using one mobile wireless company’s data service, by manually typing the SMTP commands and requests, and monitoring the responses from the email server in issue. It appears that this particular mobile wireless provider is intercepting the server’s banner message and modifying it in-transit from something like “220 [servername] ESMTP Postfix” to “200 ********************.” The mobile wireless provider is further modifying the server’s response to a client command that lists the extended features supported by the server. The mobile wireless provider modifies the server’s “250-STARTTLS” response (which informs the client of the server’s capacity to enable encryption). The Internet access provider changes it to “250-XXXXXXXA.” Since the client does not receive the proper acknowledgement that STARTTLS is supported by the server, it does not attempt to turn on encryption. If the client nonetheless attempts to use the STARTTLS command, the mobile wireless provider intercepts the client’s commands to the server and changes it too. When it detects the STARTTLS command being sent from the client to the server, the mobile wireless provider modifies the command to “XXXXXXXX.” The server does not understand this command and therefore sends an error message to the client.
As Golden Frog points out, this is "conceptually similar" to the way in which Comcast was throttling BitTorrent back in 2007 via packet reset headers, which kicked off much of the last round of net neutrality concerns. The differences here are that this isn't about blocking BitTorrent, but encryption, and it's a mobile internet access provider, rather than a wired one. This last point is important, since even the last net neutrality rules did not apply to wireless broadband, and the FCC is still debating if it should apply any new rules to wireless.

After reading the Golden Frog filing, the answer should be that it is absolutely necessary to apply the rules to wireless, because practices like these put us all at risk by undermining the encryption that keeps us all safe. As Golden Frog notes:
Absent enforceable Commission rules, broadband providers can (and at least one already does) block and discriminate against entirely acceptable Internet uses. In this case, users are not just losing their right to use the applications and services of their choosing, but also their privacy. It is not at clear that this type of encryption blocking would be forbidden for fixed broadband Internet access, under the proposed rules’ exception for reasonable network management. This example involves mobile wireless broadband, however, and it is clear that the proposed rules would not prohibit the activity. STARTLLS encryption does not constitute “a lawful website” or “an application[] that compete[s] with the provider’s voice or video telephony services[.]”11 The proposed rules on their face do not prohibit mobile broadband Internet access providers from blocking user efforts to maintain privacy through encryption.
Furthermore, Golden Frog concludes:
The claim that rules banning blocking and unreasonable discrimination are solutions in search of a problem is flatly wrong. There have been problems in the past and there are problems now. The proposed rules do not resolve all of the problems identified in the NPRM. Further broadband Internet access providers are still interfering with beneficial and privacy-enhancing applications users want to employ.
This is incredibly important -- just at a time when we need stronger encryption and privacy online, the FCC may undermine it with weak net neutrality rules that allow this type of behavior to continue.

A few months ago, I got into a conversation with a well-known internet entrepreneur/investor, who asked about possible "compromise" rules on net neutrality, suggesting that maybe it's okay to throttle Netflix traffic because there's so much of it. He argued that, perhaps there could be some threshold, and if your traffic was above that threshold it's okay to throttle it. After some back and forth, I asked the hypothetical about encryption: what if, at a time when more and more encryption is important, such a rule was in place, and overall encrypted traffic passed that threshold, then suddenly access providers could throttle all encrypted traffic, doing tremendous damage to security and privacy. What I didn't realize was that some access providers are effectively already attacking privacy and encryption in this manner.

Reader Comments (rss)

(Flattened / Threaded)

  1.  
    icon
    Josh in CharlotteNC (profile), Oct 13th, 2014 @ 10:42am

    I hope this is stupidity over malice. I wonder if someone's spam prevention measures are ill-thought-out. It's not quite clear what's happening from the graphic or description, and I can't see the filing (at work) if it's got additional details.

     

    reply to this | link to this | view in thread ]

  2. This comment has been flagged by the community. Click here to show it
     
    icon
    kenichi tanaka (profile), Oct 13th, 2014 @ 10:43am

    Here's a novel idea, if you don't want your internet connection tHrottled, STOP SHARING MOVIES, MUSIC. SOFTWARE, ANIME AND TELEVISION SHOWS.

    ISPs throttled a user's internet connection when they are downloading pirated content that they have not paid for. VPNs always get targeted because 99% of people who are using VPNs are doing so to watch streaming content that is not licensed for their region or they are sharing copyrighted content.

     

    reply to this | link to this | view in thread ]

  3.  
    identicon
    Anonymous Coward, Oct 13th, 2014 @ 10:55am

    Re:

    Except they aren't just throttling - they're actually changing the messages sent from the server. And it was for email, not streaming content. And the Netflix case wasn't even about piracy. Did you even read the article before you started trolling?

     

    reply to this | link to this | view in thread ]

  4.  
    identicon
    the threat to peace is the USA, Oct 13th, 2014 @ 10:55am

    fuck paying them then

    get singleplayer video games and hand bomb your mp3s and videos

    its time we stick it up there god damn cop window smashing thieving assholes

     

    reply to this | link to this | view in thread ]

  5.  
    icon
    Rikuo (profile), Oct 13th, 2014 @ 10:58am

    Re:

    Your comment might have made the slightest bit of sense if you had read the article and come to the comprehension that this article is talking about disabling the use of encryption via EMAIL.
    So in your insane world, if loads of people use torrent programs to get at unauthorized content, then their ISP should just disable encryption for those same users on their email? Last I checked, people aren't using email to distribute content.
    Not only that, but this is done without any sort of due process. You are in essence casting a wide net, saying that because a subset of the population infringes copyright, then it's perfectly all right for an ISP to punish all of the population by disabling encryption.
    And no, VPNs aren't mostly used to infringe copyright. I myself used a VPN - mostly so I could access Crunchyroll's USA catalog. Businesses and industries use VPNs all the time - ATMs use VPNs for one example.

     

    reply to this | link to this | view in thread ]

  6.  
    identicon
    the threat to peace is the USA, Oct 13th, 2014 @ 10:58am

    @#2

    and if i am doing a game purchase online i do it unencrypted right you fuckwit

    if i play the game online i do it unencrypted right

    NO YOU STUPID FUCKING TWIT

    this is the stupidest person period , no justification for it.

    THESE LAZY DO NOTHING GOOD FOR NOTHING ACTORS MUSICIANS NEED TO FUCK THE HELL OFF.

    and the usa and it spuppet hollywood govt can kiss my fuckign ass

    i dont download movies games or tv
    YET i use encryption and i cant play the online game well cause its crashing due to your throttling all the fucking time.

    FUCK FUCK FUCK YOU.

     

    reply to this | link to this | view in thread ]

  7.  
    icon
    Uriel-238 (profile), Oct 13th, 2014 @ 11:03am

    Company enforced tragedy-of-the-commons.

    What if we like sharing movies, music, software, anime and television shows? Doing so shouldn't be illegal and often isn't. Frankly, regional licensing is way inappropriate.

    Incidentally, VPNs are primarily used for commerce and the protection of companies accessing internet resources, not end-users pirating. So please stop tossing out bullshit statistics.

     

    reply to this | link to this | view in thread ]

  8.  
    icon
    That Anonymous Coward (profile), Oct 13th, 2014 @ 11:07am

    Re:

    a majority of black people commit crimes, so stop being black and the cops will stop bothering you.

    Please to learn to read the story before making your uninformed pronouncements, they make you look even more silly.

     

    reply to this | link to this | view in thread ]

  9.  
    identicon
    Anonymous Coward, Oct 13th, 2014 @ 11:13am

    They are keeping track and reporting it

    They do not like the encryption because they have been secretly commanded to record and give access to these records. They are dragging their feet on sending the data because of the added stresses it puts on the system and because no court will convict them of doing it.

     

    reply to this | link to this | view in thread ]

  10.  
    identicon
    Anonymous Coward, Oct 13th, 2014 @ 11:13am

    Re:

    If I could guess, they're disabling encryption on mobile because it uses more bandwidth and doesn't allow caching. That's not a good enough reason to disable encryption if the user is specifically asking for encryption, of course.

    One thing that seemed weird to me is that they didn't name the guilty party anywhere in the filing. That pretty much makes it impossible to verify the claims.

     

    reply to this | link to this | view in thread ]

  11.  
    identicon
    Anonymous Coward, Oct 13th, 2014 @ 11:14am

    Re:

    Really? No one under the guise of stupidity alters packets in transit in attempts to disable security.

    Its intentional and damn sure malevolent!

     

    reply to this | link to this | view in thread ]

  12.  
    icon
    anticrawl (profile), Oct 13th, 2014 @ 11:14am

    Re: Categorically false

    Your comment is categorically false. The primary use of VPN is business. Everyone who has to do work from home or off site has to login to VPN.

    There really is nothing else to say other tahn you might be the dumbest person to comment on Techdirt in the history of the site's existance.

     

    reply to this | link to this | view in thread ]

  13.  
    icon
    ChurchHatesTucker (profile), Oct 13th, 2014 @ 11:16am

    Re:

    Here's a novel idea, if you don't want your internet connection tHrottled, STOP SHARING MOVIES, MUSIC. SOFTWARE, ANIME AND TELEVISION SHOWS.

    Or working from home, or banking, or...

     

    reply to this | link to this | view in thread ]

  14.  
    identicon
    Just Another Anonymous Troll, Oct 13th, 2014 @ 11:32am

    Re:

    Get a clue, kenichi.
    They aren't making YOU slower, they're making Netflix slower. Also, they're disabling encryption, which is bad.
    Also, there are many legit reasons for using VPNs.
    Please return to your bridge and enjoy watching Netflix on your throttled connection.
    Oh, and if you'd like to make another inane post in response to mine, use the reply button indicated by the arrow.
    ||
    \/

     

    reply to this | link to this | view in thread ]

  15.  
    icon
    cerda (profile), Oct 13th, 2014 @ 11:33am

    this is MITM done wrong.

    Look no further than offerings to monitor HTTPS -- the easy way to to is to intercept the user's request, replace the web host certificate by your own (and the roots for it), and then act as a MITM between the user and the target server.

    You end up with a user monitor, and a monitor target web server. Easy. Privacy? Gone. Security? Gone as well.

    And yes, available from many of the "security" vendors.

    you can minimise the risk with certificate pinning, I guess. Easily done if the user is running a program. But very difficult to do for web browsers.

     

    reply to this | link to this | view in thread ]

  16.  
    identicon
    JP, Oct 13th, 2014 @ 11:38am

    Tyranny of the default

    My guess is this is simply an issue with the ISP in question using a default configuration on a Cisco ASA. Similar to this:

    https://stomp.colorado.edu/blog/blog/2012/12/31/on-smtp-starttls-and-the-cisco-asa/

    What's the saying? Never ascribe to malice that which can be adequately explained by stupidity.....

     

    reply to this | link to this | view in thread ]

  17.  
    identicon
    Anonymous Coward, Oct 13th, 2014 @ 11:48am

    Re:

    This is pretty much the scorched earth stance on copyright enforcement. Screw collateral damage, as long as people stop doing things I don't like. If it gets people to stop sharing my (insert copyright material) then I don't care if the rest of the legitimate world burns.

    And that's why people hate copyright so much.

     

    reply to this | link to this | view in thread ]

  18.  
    icon
    Josh in CharlotteNC (profile), Oct 13th, 2014 @ 11:52am

    Re: Re:

    No one under the guise of stupidity alters packets in transit

    Happens all the time. Misconfigured mail servers or aggressive spam rules could both explain the responses given.

    in attempts to disable security.

    It remains to be seen if that was the intent. There's just not enough data given to lead to a conclusion here. We don't know the methodology of the tests. We don't know what service this result was from. All we know for sure based on the graphic and description is that they did a telnet session over port 25 to apps.[redacted].com and got some admittedly odd results. Is the redacted server under Golden Frog's control, or is it the ISP's or someone else's?

    My point isn't that there isn't malice here, but before jumping to the 'this mobile wireless service is disabling encryption on my email' you should examine other possibilities and give more evidence than a single screenshot. If you think packets are being altered, then proof I would expect would be packet capture logs from both sides showing that. That's not a new concept - its how the P2P forced reset packets Comcast was injecting were discovered.

     

    reply to this | link to this | view in thread ]

  19.  
    icon
    John Fenderson (profile), Oct 13th, 2014 @ 11:54am

    Re:

    "ISPs throttled a user's internet connection when they are downloading pirated content that they have not paid for"

    Incorrect. ISPs can't possibly know what downloads constitute a copyright infringement and what don't.

    "VPNs always get targeted because 99% of people who are using VPNs are doing so to watch streaming content that is not licensed for their region or they are sharing copyrighted content."

    Even more incorrect. The primary use of VPNs is business-related. The amount of VPN use that is devoted to piracy is rather difficult to determine, but in my experience about 99% of VPN use is totally unrelated to piracy.

    But even if that number is lower, it still doesn't justify ISPs blocking this traffic.

     

    reply to this | link to this | view in thread ]

  20.  
    identicon
    PRMan, Oct 13th, 2014 @ 12:00pm

    Re: Re:

    It also seems like it makes it likely that the example company doesn't fall under FCC jurisdiction. It may be in some extreme state such as Iran or Syria.

     

    reply to this | link to this | view in thread ]

  21.  
    identicon
    Anonymous Coward, Oct 13th, 2014 @ 12:02pm

    Throttling

    There was ZERO evidence that throttling occurred against Netflix. All claims of throttling have been debunked and Netflix has repeatedly confirmed Verizon did not throttle Netflix.

    Service degradation due to congested peering ports is not the same as throttling.

    Although at times controversially biased, Techdirt usually sticks to facts. Apparently not anymore.

     

    reply to this | link to this | view in thread ]

  22.  
    identicon
    PRMan, Oct 13th, 2014 @ 12:05pm

    Re: Tyranny of the default

    Wow, excellent work. Nice link.

     

    reply to this | link to this | view in thread ]

  23.  
    identicon
    Anonymous Coward, Oct 13th, 2014 @ 12:20pm

    Re: Re: Throttling

    Saying... what? That a direct connection to Netflix improved performance compared to using 3rd party providers with congested peering ports?

    Nothing to do with throttling. The word throttle wasn't even mentioned as there wasn't any.

     

    reply to this | link to this | view in thread ]

  24.  
    identicon
    Anonymous Coward, Oct 13th, 2014 @ 12:24pm

    Re: Tyranny of the default

    An ASA with SMTP fixup is usually for the end user. As you can see in the telnet test, it's showing the banner so fixup isn't activated between the user and the mail server. My guess would be a proxy server incorrectly intercepting SMTP traffic or purposely depending on the ISP, or a DPI box that is actually intercepting and mangling packets such that STARTTLS is given incorrectly to the MTA.

    My guess would be a full transparent proxy server (IE User -> Proxy -> Mailserver), but I would need to wireshark a computer on the network and an mail server to see what's sent/received on both ends to really drill down.

     

    reply to this | link to this | view in thread ]

  25.  
    identicon
    Anonymous Coward, Oct 13th, 2014 @ 12:25pm

    Re: Re: Tyranny of the default

    Scratch that, he does have the ********* banner for an ASA.

     

    reply to this | link to this | view in thread ]

  26.  
    identicon
    Anonymous Coward, Oct 13th, 2014 @ 12:25pm

    Re: Throttling

    That's like saying goodwill donations to the local mob affiliate isn't the same thing as protection money.

     

    reply to this | link to this | view in thread ]

  27.  
    icon
    andrew_duane (profile), Oct 13th, 2014 @ 12:27pm

    Re: Throttling

    Technically Verizon wasn't throttling Netflix, they were throttling Level 3, which carries Netflix traffic. And congested peering ports aren't throttling, unless they are deliberately under-provisioned to the point where they cannot carry their designated load.

    Link capacity of peering connections (as well as all other Network-with-a-capital-N connections) aren't just pulled out of a hat. Extremely large amounts of throughput analysis is done to understand the requirements of the link. It is extremely important to ISPs to be able to handle the required bandwidth.

    This issue would have been spotted in 5 minutes by a junior engineer. Add the fact that it was fixed only after weeks of press shaming and the "payments" from Netflix means it was deliberately designed to extort such payments.

     

    reply to this | link to this | view in thread ]

  28.  
    identicon
    Anonymous Coward, Oct 13th, 2014 @ 12:34pm

    Re: Re: Re: Throttling

    It's not throttling. It just looks like throttling and quacks like throttling.

     

    reply to this | link to this | view in thread ]

  29.  
    icon
    Mike Masnick (profile), Oct 13th, 2014 @ 12:38pm

    Re: Throttling

    There was ZERO evidence that throttling occurred against Netflix. All claims of throttling have been debunked and Netflix has repeatedly confirmed Verizon did not throttle Netflix.

    Verizon purposely let its connection clog and didn't do the most minor thing to connect another port to clear it up. To me, that's throttling. End result is identical.

     

    reply to this | link to this | view in thread ]

  30.  
    icon
    Josh in CharlotteNC (profile), Oct 13th, 2014 @ 12:47pm

    Re: Re: Re:

    Or firewall rules. Thanks to JP below, looks like he found the likely cause.
    https://stomp.colorado.edu/blog/blog/2012/12/31/on-smtp-starttls-and-the-cisco-asa/

     

    reply to this | link to this | view in thread ]

  31.  
    identicon
    Anonymous Coward, Oct 13th, 2014 @ 12:48pm

    Improper interception happening on Time Warner too

    Though not quite as bad as shown here, Time Warner Cable recently began actively disrupting customers too. They decided they wanted to tell customers about their great new faster modems. Rather than do something sane like send an e-mail to the primary account holder e-mail address or include an insert in the account statement, their solution was to deploy a packet interceptor that dropped all traffic on tcp/443 (https) and intercepted all traffic on tcp/80 (http). All http traffic not pointed to the server advertising their new modems was answered with a redirect to that server. I was using only https when they turned it on, so everything mysteriously broke. When I happened to use http and saw what they'd done, it was not too hard to acknowledge their ad, but they caused a huge mess in the meantime, including losing all my work on the https site I had been using.

    To top it all off, upon acknowledging the ad, they showed a page confirming the acknowledgement, then remotely reset the cable modem without telling me, which broke what few connections I had on protocols they had not broken with the idiotic deep packet inspection.

     

    reply to this | link to this | view in thread ]

  32.  
    identicon
    Vladlagg, Oct 13th, 2014 @ 12:51pm

    VPN and Comcast

    My internet download speeds are 5-6 times faster through Comcast using PIA VPN routing through their largest exit gateway (Toronto, CA). I pay for about 30 megs from Comcast and get over 150 mb download on average through a new docsis 3 modem. for some reason Comcast prioritizes encrypted traffic (at least on my network). I am not about to complain, but if i should be pissed off if they throttled it, should I instead praise their boosted speeds? Them giving me the maximum speed capable for a fraction of the price is very nice indeed. It always leaves me feeling conflicted.

     

    reply to this | link to this | view in thread ]

  33.  
    identicon
    Anonymous Coward, Oct 13th, 2014 @ 12:56pm

    Re: this is MITM done wrong.

    Any chance that changing my DNS server broke this sort of man in the middle attack by my provider?

    I switched my DNS server to Google's 8.8.8.8 and 8.8.4.4 and Netflix through Verizon FIOS which had been frequently stalling stopped having problems.

     

    reply to this | link to this | view in thread ]

  34.  
    identicon
    Anonymous Coward, Oct 13th, 2014 @ 1:00pm

    Re: Re: Re: Re:

    Dear person you ARE STUPID!!!!

    Modifying an in-transit packet as per article...
    "It appears that this particular mobile wireless provider is intercepting the server’s banner message and modifying it in-transit from something like “220 [servername] ESMTP Postfix” to “200 ********************.”"

    This IS NOT ACCIDENTAL CONFIGURATION!

    I work in IT, I have done this before, and based on your comment you do not work in IT and if you do you need to be FIRED so damn fast that your head spins!

    Misconfiguration on email SERVER would be allowing unencrypted sessions to take place when it should be encrypted (which appears to be the case here as well) but DOES NOT MODIFY THE DAMN PACKET!!!! That is something else entirely!

     

    reply to this | link to this | view in thread ]

  35.  
    identicon
    Anonymous Coward, Oct 13th, 2014 @ 1:31pm

    Re: Re: Throttling

    Some like to call that flimsy "plausible deny-ability".

    I like to call it treating everyone like they are stupid.

     

    reply to this | link to this | view in thread ]

  36.  
    identicon
    Anonymous Coward, Oct 13th, 2014 @ 1:34pm

    If this is an American ISP sending falsified telecommunication signals. I'm pretty sure that violates the Computer Fraud and Abuse Act.

    In fact, I'm pretty sure Comcast is also guilty of violating the CFAA by injecting falsified reset signals into their customer's private communications.

     

    reply to this | link to this | view in thread ]

  37.  
    identicon
    Anonymous Coward, Oct 13th, 2014 @ 1:45pm

    Re: Throttling

    I'm sure to a junior, inexperienced, naive network engineer without much operational savvy it may look that way. But those of us who have been doing this work for the last 30-odd years know deliberate underprovisioning when we see it.

     

    reply to this | link to this | view in thread ]

  38.  
    identicon
    Anonymous Coward, Oct 13th, 2014 @ 1:47pm

    Re:

    When you are a big business then you get special protections... and besides... if you read the contract you signed for service it tells you that you have given all of your packets to them for whatever they want to do with them.

    Either way this can be handled by having both client and server refuse anything other than encrypted connections, but that is not likely, even though it should be the default.

     

    reply to this | link to this | view in thread ]

  39.  
    icon
    Josh in CharlotteNC (profile), Oct 13th, 2014 @ 1:49pm

    Re: Re: Re: Re: Re:

    Calm down and read what I actually posted and not what you misinterpreted through that fog of anger.

    I never said a misconfigured mail server would modify a packet in transit. I said that a misconfigured mail server or spam rules could explain the screenshot. I also said jumping to the conclusion of modified packets without evidence from both ends of the connection wasn't warranted based on the extremely brief information presented in the article. I'll stand by that judgement. Any claim requires evidence, and their claim of an ISP deliberatly breaking email encryption was a heavy one. And now since we have a plausible explanation of a default configuration on a router that could cause exactly what is shown, I'll stand by my first comment: Stupidity, not Malice.

    And I'll be sure to let my boss know I should be fired because some anonymous person on the internet said so.

     

    reply to this | link to this | view in thread ]

  40.  
    icon
    Shawn (profile), Oct 13th, 2014 @ 1:51pm

    Re: Re: Re: Re: Re:

    "I work in IT, I have done this before, and based on your comment you do not work in IT and if you do you need to be FIRED so damn fast that your head spins!"

    You really out to understand what you are talking about before you make idiotic statements like that. While you may "work in IT" you obviously have never seen how a Cisco ASA (going all the way back to when they were Pix's) treats SMTP by default. While arguably some of the things the default SMTP security server (SMTP fixup) protects against are nice it will absolutely clobber the starttls command and it will look just like the screenshot provided.

     

    reply to this | link to this | view in thread ]

  41.  
    icon
    Kenpachi (profile), Oct 13th, 2014 @ 1:52pm

    In need of some basic clarification...

    Could anyone explain, in laymen terms, the following aspect of this issue? (Maybe it's already explained but I'm unable to see it, so I apologize in advance, I don't have the technical background to understand most of this)

    The fact that packets are being modified in transit to purposely disable encryption, that happens with no warning or error message to the end user, unless of course the user knows what he/she's doing, and the connection takes place nonetheless, IN THE CLEAR, even tough the user wanted to use encryption technology?

    So, the message (or wahtever) is sent, and the whole time the user thought was using an encrypted means of communication and never noticed that that wasn't the case?!

    Am I missing something? Please someone tell me that the user gets an error message and the connection is NOT established, or something, that the user does not send anything.

    (Even if that's the case, what about machine to machine communications? Big data sent back and forth automatically, without user intervention?)

    If I'm correct, (and I hope I'm not), this should be a crime, it's encryption disabling technology without the user's consent and knowledge that it's happening in the background, in real time. What about corporate secrets, R&D, scientific/academic research, medical records, financial records, etc, etc.

    This is flat out malicious interference, wtf...

    I really thank anyone and everyone who can comment on this and put my suspicion (or my worst fears) to rest.

     

    reply to this | link to this | view in thread ]

  42.  
    identicon
    Anonymous Coward, Oct 13th, 2014 @ 1:56pm

    However, OpenVPN, if you are technically inclined enough to set it up, cannot be detected.

    Even though Iran outlawed VPNs, I still see a lot of connections on my SoftEther/VPNGate VPN from Iran. I periodically change the ports that it operates on and use non-standard ports so that it cannot be detected.

     

    reply to this | link to this | view in thread ]

  43.  
    identicon
    Anonymous Coward, Oct 13th, 2014 @ 1:58pm

    Re: Re: Re: Re: Re: Re:

    Prove it... you show me where any mis-configured EMAIL configuration (not a network device) will lead to a packet being intercepted and changed...

    Get on Mr. Brilliant! Anyone else?

     

    reply to this | link to this | view in thread ]

  44.  
    identicon
    Anonymous Coward, Oct 13th, 2014 @ 2:01pm

    Re: Re: Re: Re: Re: Re:

    Part of your post...
    "Happens all the time. Misconfigured mail servers or aggressive spam rules could both explain the responses given."

    This indicates to me that you are saying a misconfigurated mail server or spam rules could do this. which is not the case in either situation.

    The server can respond to an encryption request badly due to misconfiguration, but it cannot alter the packet. SPAM filtering does not alter any packets either, but it does study them to determine if the message will be delivered or placed in quarantine or even deleted.

     

    reply to this | link to this | view in thread ]

  45.  
    identicon
    Anonymous Coward, Oct 13th, 2014 @ 2:05pm

    Re: In need of some basic clarification...

    Looking at the logs of the transaction, the server is supposed to be advertizing STARTTLS, which is opportunistic encryption upon request and it's been disabled.

    JP probably has the best guess of what's taking place, the problem is we don't know where the ASA resides.

    This could in fact be on GoldenFrog's network and they simply forgot to turn off SMTP fixup, or it could be in bridge mode on the wireless carrier. Either way a simple call and one line in the config will fix the issue. I'd chalk this up to stupidity more than malice.

     

    reply to this | link to this | view in thread ]

  46.  
    identicon
    Anonymous Hero, Oct 13th, 2014 @ 2:09pm

    ?

    I'd like more info about this.

    First, there's some cognitive dissonance going on here. On one hand, ISPs are blocking encryption, on the other hand, people are seeing better throughput if they proxy their network traffic through an encrypted channel (i.e., VPN). This would suggest that the ISPs are not targetting encryption as a whole, but rather are targetting ESMTP or STARTTLS specifically.

    Golden Frog's write-up, is confusing to me. I think there is a serious problem with downgrading encryption at the ISP level, but the Golden Frog report leaves me with many more questions than it answers. Is it SMTP over port 25 specifically? What other services are being downgraded? Etc.

     

    reply to this | link to this | view in thread ]

  47.  
    icon
    Shawn (profile), Oct 13th, 2014 @ 2:11pm

    Re: Re: Re: Re: Re: Re: Re:

    Mr "I work in IT" https://stomp.colorado.edu/blog/blog/2012/12/31/on-smtp-starttls-and-the-cisco-asa/
    I do not even think anyone is arguing this is the exact cause as what is shown in the screen shot but it is not only possible it is the default behaviour for firewalls that are all over the freaking place and those of us who 'Work in IT' and have troubleshot SMTP and ESMTP have seen screens like the one posted for years.
    Not going to try and prove anything to you. Your statements prove to the rest of us that you are talking out of your rectum.

     

    reply to this | link to this | view in thread ]

  48.  
    icon
    John Fenderson (profile), Oct 13th, 2014 @ 2:25pm

    Re: In need of some basic clarification...

    If you've configured your mail reader to require an encrypted connection and that connection cannot be set up, then the mail should not be delivered and you should get an error message. If this is not happening, you're using a terrible mail reader and should use a different one.

     

    reply to this | link to this | view in thread ]

  49.  
    icon
    Shawn (profile), Oct 13th, 2014 @ 2:51pm

    Re: Re: In need of some basic clarification...

    SMTP is as we all know by know a clear text protocol used for email for the last 32 years give or take. About 12 years ago the SMTP protocol was extended (ESMTP) to support among other things TLS (Transport Layer Security) which is basically SMTP over SSL. Since 'plain old' SMTP was (and probably still is) the most popular method of delivering email the Extended commands are optional. STARTTLS is called opportunistic because if it can use TLS it will, otherwise it will fall back to plain old SMTP and deliver the email in clear text. You can configure your server to only use TLS and it will refuse to send email 'in the clear' but if you do this on a border email gateway (one that sends and receives to the general Internet) You will quickly see that many many many email servers are not configured to accept TLS connections and you will have a lot of dead mail on your server that could not be delivered.

     

    reply to this | link to this | view in thread ]

  50.  
    icon
    Kenpachi (profile), Oct 13th, 2014 @ 2:51pm

    Re: Re: In need of some basic clarification...

    Thanks a lot. On the token of "stupidity trumps malice", call me a tin foil, but it's undeniable at this point, when every day the entire world is waking up on what has been and is happening behind the curtains, that malice is winning by a landslide (as far as government and corporate overreach goes).

    Of course, that does not take anything away from stupidity, which is rampant. But if even with all the facts and evidence that are a click away, we still attribute to stupidity things like this, we are really making it exponentially easier for this things to keep on going.

    In any case, ultimately it does not matter the nature of the beast, what matters is how to avoid it. And always be vigilant of things like this.

     

    reply to this | link to this | view in thread ]

  51.  
    icon
    Kenpachi (profile), Oct 13th, 2014 @ 3:11pm

    Re: Re: In need of some basic clarification...

    Thanks man. Now, what really bothering me is what's happening at the other side of the pond. That the user adopts good practices when configuring software is a must. Been there, done that.

    And then there's this little tiny thing, where at the other side, a 3rd party is actively disabling that way of communication. Whether an encrypted communication is possible or not, that's not an issue, the fact is that they are, arbitrarily and without user intervention, making it impossible to use a secure method.

    If that's not a dead giveaway of the intentions behind it, I frankly don't know what is...

    Taking into account the fact (well known to ISPs and the like), that 99.9 % of internet users do not have the first clue as to how to properly configure an email client.)

    I don't know man... I hope it's all just a perfect blend of my level of paranoia and my lack of understanding of the subject matter.

     

    reply to this | link to this | view in thread ]

  52.  
    identicon
    Anonymous Coward, Oct 13th, 2014 @ 3:54pm

    Re: Re: Re: Re: Re:

    Josh wouldn't be the one I would fire in this group even if he was wrong.

     

    reply to this | link to this | view in thread ]

  53.  
    icon
    JMT (profile), Oct 13th, 2014 @ 4:35pm

    Re: Throttling

    "Service degradation due to congested peering ports is not the same as throttling."

    Deliberately allowing those ports to become and remain congested is exactly the same as throttling, particularly to the person at the end of the line.

    "Techdirt usually sticks to facts."

    That they do, unlike the 'truthiness' from shills like yourself.

     

    reply to this | link to this | view in thread ]

  54.  
    identicon
    Anonymous Coward, Oct 13th, 2014 @ 4:56pm

    Re: Re: Re: Throttling

    When most people think of throttling, I don't think they care about the technical details. If you have the load spread across 5 cables, and you rip one out of the wall, it has the same effect as leaving them all in but dropping every fifth packet.

     

    reply to this | link to this | view in thread ]

  55.  
    icon
    Christopher (profile), Oct 13th, 2014 @ 5:43pm

    Re: Re: Re: Re: Re: Re: Re:

    Exactly.... this is not simply the server in question sending back "We do not support encryption!"
    This is something actively changing the packet while in transit falsifying the "We do not support encryption!" when the server in question is saying "Yeah, we support encryption, turn it on!"

     

    reply to this | link to this | view in thread ]

  56.  
    identicon
    Anonymous Coward, Oct 13th, 2014 @ 5:56pm

    legitimate use?

    So, as referenced in the above links, Cisco's "ASA Series Firewall" apparently has a feature where it rewrites the mail server's response to the EHLO command to hide that the server supports STARTTLS. It also intercepts and fails any client attempt to run a starttls command.

    Why would it do this? Seriously - what is the supposedly legitimate use of this feature? This stinks badly of a some corporate-internal-spying trick similar to how they put their own "trusted CA" cert in PKI settings for company laptops, so they can run a Man-in-the-Middle attack on all HTTPS usage by routing traffic through a transparent proxy.

    The last time I saw legitimate editing of in-band content like this was during IP Masquerading (NAT) to fixup old, pre-IPMasq protocols such as FTP that sent the local IP address in-band. If there is such a legitimate use for stripping encryption while still forwarding to the real SMTP server, I'd love to hear about it. For now, while this story may be "legitimately" be a case of someone using a Cisco ASA firewall, why they are using a firewall with such a compromising feature?

    As a side note, Cisco's (voluntary or involuntary) involvement with the NSA and the resulting "implanted" hardware has not gone unnoticed. How this relates (or not) to the issues mentioned in the story is an open question that is worth considering.

     

    reply to this | link to this | view in thread ]

  57.  
    identicon
    Anonymous Coward, Oct 13th, 2014 @ 6:02pm

    I need to find someone to remove my tinfoil hat, but

    This would make things a lot easier for the NSA, wouldn't it? And, given the power they have, anyone here think they might be connected to this?

     

    reply to this | link to this | view in thread ]

  58.  
    identicon
    Matt, Oct 13th, 2014 @ 6:06pm

    Re:

    Oh dear. If you really think 99% of VPN users are using them to 'illegally' access content, you really have no clue how the business world works.

     

    reply to this | link to this | view in thread ]

  59.  
    identicon
    Anonymous Coward, Oct 13th, 2014 @ 6:07pm

    Re: I need to find someone to remove my tinfoil hat, but

    Of course it makes it easier for the NSA. Getting people to not use encryption - by corrupting standards, bribing/convincing people that run infrastructure, and many other methods - is probably their primary tool. You don't break encryption by brute force when you can go after a side channel.

    I strongly recommend watching PHK's talk that explores these ideas called "NSA Operation Orchestra".

     

    reply to this | link to this | view in thread ]

  60.  
    icon
    tqk (profile), Oct 13th, 2014 @ 6:16pm

    Re:

    VPNs always get targeted because 99% of people who are using VPNs are doing so to watch streaming content that is not licensed for their region or they are sharing copyrighted content.

    You're an ignorant fool, or an idiot.

    How's about a gun analogy? You can use a gun to shoot bottles on a fence, or targets on a shooting range, or rabbits (you beast!) or lots of other things. They can also be used to shoot attacking bears or pumas, and humans.

    My last big client (I'm a geek) provided me with a laptop which was so locked down, all it could do was communicate with their servers via VPN. Well, until you booted from a LiveCD Linux disk, that is (but I digress).

    Guns don't shoot people. People do.

    Tech is just complicated hammers (tools). The intent of those using it is what matters. VPN's just a tool.

     

    reply to this | link to this | view in thread ]

  61.  
    identicon
    Central Services, Oct 13th, 2014 @ 6:24pm

    Service XOR Provider

    (Pick one.)

     

    reply to this | link to this | view in thread ]

  62.  
    icon
    tqk (profile), Oct 13th, 2014 @ 6:27pm

    Re: Re: this is MITM done wrong.

    Any chance that changing my DNS server broke this sort of man in the middle attack by my provider?

    Or, your provider just off-loaded tracking you onto google (your ISP: "hey thanks!"), as G. is certainly an NSA puppet?

    No offense, G. I know you have no choice in the matter. Carry on.

     

    reply to this | link to this | view in thread ]

  63.  
    icon
    Brett Glass (profile), Oct 13th, 2014 @ 6:43pm

    That looks like an anti-spam proxy.

    That behavior does not look like the ISP's equipment is altering messages from the remote server, but rather as if it is redirecting the session to a proxy server -- most likely for the purpose of stopping outbound spam.

    Such a proxy can send the mail onward in encrypted form once it's checked for spam. If this is the case, and there is encryption or even just physical security on the inbound connection, the privacy of the message isn't compromised.

    It's sad that such measures are necessary, but they are -- not only in public hotspots but in situations where a new user might sign up for new service from an ISP and then send boatloads of spam until the account is turned off. (This is happening a great deal nowadays.)

     

    reply to this | link to this | view in thread ]

  64.  
    icon
    Kenpachi (profile), Oct 13th, 2014 @ 7:06pm

    Re: legitimate use?

    Of course it has everything to do with the NSA's takeover of the internet. Now that the rest of the world has a clue as to what to look for, these things are popping up left and right.

    The problem with a lot of technical people having trouble in differentiating this from misconfigurations or simple stupidity, is that they are attacking the problem from an engineering point of view.

    It is to be expected, after all, it's really hard to reconcile the reality of this madness and not feel overwhelmed and/or powerless.

    The first reaction is to tackle the issue with an "IT mentality", to speculate on the nature of what's being discovered, and to try to come up with a solution/workaround.

    Anyway, the most important thing is, in my view, that good people around the globe, who work for a free and open internet, keep looking for things like this and bring them to light.

    Nobody asked or wanted to be dragged into this, but we need to come together and defend our privacy, our freedom of communication/association, without monitoring or outside interference.

     

    reply to this | link to this | view in thread ]

  65.  
    identicon
    Michelle Sullivan, Oct 13th, 2014 @ 7:08pm

    Re: Re:

    VPNs are generally GRE or IPSec which use different protocols (like 50 or 51 not TCP).. this is not the same. SSL VPNs are done over port 80 using STARTTLS or 443 (SSL) - this would not block TCP port 443.

    Personally I'd call b***shit on this, normally when blocking commands you block the full command or part of it... IE the response should be XXXXXXXX or XXXXXXXS *not* XXXXXXXA.


    Michelle

     

    reply to this | link to this | view in thread ]

  66.  
    identicon
    Anonymous Coward, Oct 13th, 2014 @ 7:34pm

    Re: Re: legitimate use?

    I generally agree with your points - especially with regard to the blindness common in engineering (I say this as an engineer myself). I just like to sanity-check that I'm not missing some obvious explanation that I might have missed. Compatibility workaround for legacy software (like the FTP+IPMasq/NAT issue I mentioned) would be a good example. Until I hear such an explanation, I tend to believe that this is a feature created with poor intentions.

    What many engineers don't seem to understand is that while the technical is certainly important, if you ignore the political side you surrender that front to those that are not ignoring politics.

    While it is from a work of fiction, I think this quote from Susan Ivanova in "Sleeping In Light" (the last episode of Babylon 5) said it nicely: (emphasis mine)
    It taught us that we had to create the future, or others will do it for us.
    It showed us that we have to care for each other – because if we don't, who will?
    And that strength sometimes comes from the most unlikely of places.

    Mostly, though, I think it gave us hope that there can always be new beginnings, even for people like us.

     

    reply to this | link to this | view in thread ]

  67.  
    identicon
    Anonymous Coward, Oct 13th, 2014 @ 7:38pm

    Re: Re: Throttling

    No it isn't. The end result may be the same but that doesn't make it throttling.

     

    reply to this | link to this | view in thread ]

  68.  
    identicon
    Anonymous Coward, Oct 13th, 2014 @ 7:51pm

    TLS traffic...

    Actually... I see this occasionally myself a lot... I work in a tech support group that works with many other email servers, and usually this is due to some router on a network that's misconfigured to inspect SMTP traffic, which usually follows strict SMTP spec, as opposed to ESMTP which includes the usage of TLS.

     

    reply to this | link to this | view in thread ]

  69.  
    icon
    Shawn (profile), Oct 13th, 2014 @ 7:52pm

    Re: legitimate use?

    The use at the time it was introduced (and ESMTP was less common) was to enforce the RFC 821 https://www.ietf.org/rfc/rfc821.txt This stance limits it to only accepting only the origional SMTP commands and blocking anything else. As to why it is still defaulting to that mode (you can configure it to allow ESMTP) I couldnt hazzard a guess. I have never been a fan of dealing with SMTP security on a firewall vs a good SMTP gateway but some people want the firewall to be the be-all end all in their border security. I am still leaning towards laziness vs malice

     

    reply to this | link to this | view in thread ]

  70.  
    identicon
    Anonymous Coward, Oct 13th, 2014 @ 8:28pm

    Re: Re: Re: Re: Re:

    You. Are. An. Idiot.

    Misconfigured/misdesigned/misdeployed devices in the data path do things like this all the time. The Cisco PIX firewalls were notorious for their non-compliance with standards (RFCs) and they're not the only ones -- just one of the more notable examples from a very long list.

    Moreover, mail servers do EXACTLY what you say they don't do. Oh, they're not supposed to do it, but then again they're also not supposed to do accept-then-bounce processing or retry when presented with a 5XX SMTP response, but some of them do those things too.

    Anyone who has actually worked in the field, has read and understood the RFCs, actually reads their mail system logs, and periodically looks a traffic with tcpdump, knows all these things. They're just part of the brokenness that crops up all day, every day. Nobody is saying that these devices and these servers should be doing this stuff, but due to broken code or administrator error or unanticipated interactions or some combination of those (or other things), they really do.

    Of course, ignorant newbies like you don't know any of this stuff. You might learn one day, if you STFU long enough to listen. But it seems more likely you'll just rant and eventually turn out to be one of the incompetent assholes running sites that do wacky things like this, causing problems for the rest of us and triggered another set of rants from another generation of ignorant newbies.

     

    reply to this | link to this | view in thread ]

  71.  
    identicon
    Anonymous Coward, Oct 13th, 2014 @ 8:40pm

    Re: legitimate use?

    "why they are using a firewall with such a compromising feature?"

    Because they don't know any better.
    Because they're not aware of this problem (and others).
    Because nobody gets fired for buying Cisco.
    Because they only know about Cisco and don't want to learn anything else.
    Because they think Cisco is the be-all end-all.
    Because they don't care if this is a problem.
    Because it's not a problem for them.
    Because they have to justify their bullshit Cisco certifications. (They really are, you know.)
    Because it's what they've always done.
    Because they don't want to learn.

    There are more excuses, of course; there will always be more. But this little list explains much of what's wrong with the current Internet environment.

     

    reply to this | link to this | view in thread ]

  72.  
    identicon
    Anonymous Coward, Oct 13th, 2014 @ 8:46pm

    Re: Re: Re: this is MITM done wrong.

    I guess you can create your own local DNS server (at least for more common requests) and have it update periodically.

     

    reply to this | link to this | view in thread ]

  73.  
    identicon
    tommy, Oct 13th, 2014 @ 9:32pm

    this could be him having a cisco firewall with SMTP fixup enabled

    The *************** banner is normally seen when you are using a cisco ASA with SMTP fixup enabled, it messes with ESMTP commands such as STARTTLS.

    This may not be the ISP

     

    reply to this | link to this | view in thread ]

  74.  
    icon
    gojomo (profile), Oct 13th, 2014 @ 9:58pm

    Confused filing & neturality definition

    It's very suspicious that Golden Frog is not naming this "one mobile wireless company's data service". Other than the context that this is a filing to the FCC, can we even be sure it's in the US?

    Such in-flight data-tampering is a very different animal than traffic prioritization, and shouldn't be lumped together other. As a rare and egregious practice, it could be solved via name-and-shame in the market, or even basic laws against fraud. It doesn't need a federal 'neutrality' bureacracy.

    (It'd also be relatively easy to work-around via self-help... such as VPN-like tunnelling.)

     

    reply to this | link to this | view in thread ]

  75.  
    identicon
    Anonymous Coward, Oct 14th, 2014 @ 12:40am

    Re: @#2

    NSA psyops tactics are getting weird...

     

    reply to this | link to this | view in thread ]

  76.  
    identicon
    Anonymous Coward, Oct 14th, 2014 @ 1:21am

    Re: Re: Re: Throttling

    And just because the NSA stores tons of communications data, that doesn't mean they're collecting it.

     

    reply to this | link to this | view in thread ]

  77.  
    identicon
    Anonymous Coward, Oct 14th, 2014 @ 3:00am

    Re: Re: Re: In need of some basic clarification...

    "aking into account the fact (well known to ISPs and the like), that 99.9 % of internet users do not have the first clue as to how to properly configure an email client.)"

    This is correct. 99.9% of Internet users also do not have the first clue how to even select an email client, which is why -- in 2014 -- spam, phishing and malware distribution via email are still trivially easy.

    But wait: it gets worse. A large fraction of those Internet users have their email client forced on them by organizations run by ignorant newbies...who specify things like Outlook, despite a continuous history of Fail dating back to its initial release.

    So yes, ISPs and others are doing stupid dangerous unethical unprincipled things at all layers of the IP protocol stack, but a hell of a lot of users are undercutting themselves in far worse ways.

     

    reply to this | link to this | view in thread ]

  78.  
    identicon
    Anonymous Coward, Oct 14th, 2014 @ 3:15am

    Re: Re: legitimate use?

    The question isn't why somebody would buy a Cisco firewall that happened to have this feature. I'm sure "Nobody ever got fired for buying Cisco" is a common first and last thought when choosing vendors.

    The issue is why a security-disabling MitM-attack feature exists at all.

    It's one thing if Cisco created it to work around, for example, some legacy compatibility issue. It is a very different thing if it is just some idiotic "pop corporate design" that is totally unnecessary and just happens to allow that corporation to log all the emails that went through their firewall.

     

    reply to this | link to this | view in thread ]

  79.  
    icon
    PaulT (profile), Oct 14th, 2014 @ 4:38am

    Re: Re: Re: Re: Re: Re: Re: Re:

    I worked an end user helpdesk for a hosting company for a couple of years. The biggest, most reliable indication that someone didn't really know what they were talking about was if they started the conversation with their credentials. Especially if their "credentials" were "I work with computers" rather than "I'm a CCIE qualified engineer" when trying to diagnose a Cisco fault, for example.

    Yeah, AC, you "work in IT". So does the guy who reads from a script in a helpdesk call centre for a living and the guy who came in to service our printers this week. That doesn't mean they know anything about routing and security at the ISP level. By the look of things, neither do you. Unless you want to quote actual knowledge and credentials, in which case I think we're all ears.

     

    reply to this | link to this | view in thread ]

  80.  
    identicon
    Anonymous Coward, Oct 14th, 2014 @ 7:13am

    Re: Re: Re: legitimate use?

    "The issue is why a security-disabling MitM-attack feature exists at all."

    Because security is not -- to them -- an important consideration. Minimizing the number of hours spent on support calls (and thus the cost) is an important consideration, so they've made a conscious decision to include this and enable it by default in order to save money.

    That shouldn't surprise anyone: Cisco, Oracle, Microsoft, Apple, all the major vendors have done this for years. Oh, it's all (almost all) documented and anyone that knows what they're doing can change/fix it, but they'll likely find that if they need support post-change, the first thing that the help desk personnel will them is to revert that change or reset to factory defaults, because "it's not a supported configuration" or "it's not a recommended configuration" or whatever.

    This is yet another of the many reasons why so many security breaches and dataloss incidents happen. Even moderately clueful mid-level engineers who are trying to do things right and who want to do things right are consistently undercut, e.g. CTO: "Just set it up like Cisco says and we'll live with it". Engineer: "But it's wrong." CTO: "Do it anyway."

    Until the people at the C-level are held personally accountable for their decisions, including purchasing, deployment, and maintenance, this won't change. Whoever's responsible for the fiasco at JP Morgan will keep their job, or at worst, be given a golden parachute and a reference that will let them land an equivalent one across the street. Meanwhile the poor schleps who have been working 90-hour weeks trying to do damage control will be saddle with their next boss, who will also be on the way out from somewhere else and will repeat the exercise.

    And Cisco will happily sell them more product any time they want to write a purchase order. Thus the symbiotic relationship and race to the bottom continues.

     

    reply to this | link to this | view in thread ]

  81.  
    icon
    SMTPSupportEngi1 (profile), Oct 14th, 2014 @ 9:22am

    SMTP Fixup

    Guys Just google SMTP fix up, its a badly configured firewall nothing more.

     

    reply to this | link to this | view in thread ]

  82.  
    identicon
    Brian, Oct 14th, 2014 @ 10:24am

    Re:

    Actually I think you'll find that most VPN users are connecting to work for telecommuting. There's a very small percentage that are using VPN for nefarious purposes.

     

    reply to this | link to this | view in thread ]

  83.  
    identicon
    Anonymous Coward, Oct 14th, 2014 @ 10:32am

    Re: Re:

    "Last I checked, people aren't using email to distribute content."

    Oh they are and have been since the days of AOL vs Prodigy. However it took weeks to download some things back then...

     

    reply to this | link to this | view in thread ]

  84.  
    identicon
    Actually has a clue about ISPs, Oct 14th, 2014 @ 10:33am

    Re: Re: Throttling

    Changing the route to a non-congested peering point can make traffic flow faster, yes.

     

    reply to this | link to this | view in thread ]

  85.  
    icon
    rkhalloran (profile), Oct 14th, 2014 @ 10:50am

    Re:

    On those days I work from home (for A Large US Bank), I connect via a VPN client. Are you telling me I'm breaking the law doing that? Or are you simply an uninformed git?

     

    reply to this | link to this | view in thread ]

  86.  
    identicon
    nobody, Oct 14th, 2014 @ 11:21am

    Re: Re:

    And Srpint, among others, also intercepts DNS, replacing answers to queries with their own, usually to redirect NXDOMAINqueries to a search-vertising site. And if you query some domains, such as ietf.org, Sprint will not return anything and the query times out.

    This is not net neutrality. It is a clear and obvious effort to deny privacy and security to their users, to monitor, track, intercept, interdict and control their activity.

     

    reply to this | link to this | view in thread ]

  87.  
    identicon
    nobody, Oct 14th, 2014 @ 11:29am

    Re: Re: Re:

    ps,

    Sprint's interception of DNS makes DNSSEC impossible. They clear the dnssec bit bothg ways. There is no way to authenticate the server or the answer. Sometimes the answer is inavlid, pointing to a spammer website. It's all screwed. Sprint users are all screwed.

     

    reply to this | link to this | view in thread ]

  88.  
    identicon
    Christopher Smith, Oct 14th, 2014 @ 11:43am

    This is spam prevention, not something malicous

    Ummm... the specific case they are referring to doesn't sound like anything nefarious.

    The specific example is using STARTTLS with SMTP at port 25, which is a *very* important and specific case. Many, many ISP's block or filter direct SMTP traffic that doesn't go through their own SMTP gateways. This is because by talking at port 25 you can effectively impersonate a mail server, thereby creating.... SPAM! Particularly for consumer facing ISP's, this is a very common problem because spammers exploit botnets on their customers to deliver their SPAM.

    The work around has been to declare a different port, 587, as a "submission" port. It speaks the same protocol, but the server knows that it should only accept messages that it is a reasonable SOURCE for (basically they know it is one of their customers so they can require authorization).

    Lots of ISP's outright block port 25. These guys are being nice and leaving it open but filtering for spam activity. Once it switches to STARTTLS though, the ISP can't see anything that is going on, and spammers exploit the heck out of this.

    So in fact this ISP is getting ripped to shreds for being *more permissive* than a lot of other ISP's.

    If an ISP gets labeled as a source of SPAM, the consequences are dire, so this kind of intervention isn't considered evil or deceitful by anyone I know.

    If this actually causes a problem for a customer, they can report it to tech support. The usual protocol is that someone will either inform them of the 587 option, or in a lot of cases just open up the access between that machine and whatever servers they are trying to talk to (it might let some spam through, but if it is actually causing a problem, that's worth allowing).

    Techdirt should know better.

     

    reply to this | link to this | view in thread ]

  89.  
    identicon
    Anonymous Coward, Oct 14th, 2014 @ 12:03pm

    Re:

    This is the default behavior of a Cisco ASA with inspect ESMTP enabled.

     

    reply to this | link to this | view in thread ]

  90.  
    identicon
    Dylan, Oct 14th, 2014 @ 1:58pm

    Re: In need of some basic clarification...

    Many servers (and clients) will attempt to relay a message over SSL/TLS, if possible. If SSL/TLS is not available, many servers/clients will relay the message in plain text.

    Apparently encrypted email is a relatively new thing. Many servers are/were not equipped with SSL capability. Expecting encryption for all of your email was unrealistic and was treated as a "nice to have."

    By breaking the STARTTLS negotiation, but servers/clients will silently switch to plain text transmission.

    I've personally set my server to REQUIRE secure connections (it's no longer a "nice to have.") I hope everyone else has seen enough and started to do the same.

     

    reply to this | link to this | view in thread ]

  91.  
    identicon
    GEMont, Oct 14th, 2014 @ 3:20pm

    Business as usual

    Sadly, this mess is probably just another "sheep to the sheering pen" scam, where the players - Verizon and their ilk, and the FCC - bat the ball - Net Neutrality - back and forth, until the public demands a set of rules/laws be created to protect their interest.

    Then the players sit down and manufacture another "for the children" law where the wording is just loose enough to allow the whole wish list of the players to be interpreted from the text, after the laws are in place.

    The government can then claim that the Public "made'em do it", so its not their fault that the rules turned out to be exactly what the Players wanted and nothing that the public wanted.

    Methinks you will get your new Net Neutrality Rules.
    But I don't think anyone but the Players are gonna like'em.

    ---

     

    reply to this | link to this | view in thread ]

  92.  
    identicon
    Zehd, Oct 14th, 2014 @ 3:35pm

    Um, guys.... calm down a moment

    This can very well be simply an ANTI-SPAM measure implemented for the express purpose of blocking botnets from sending spam using these wireless connections.

    End users should not be using port 25 to connect to their SMTP hosts to begin with--they should be using either 587 or 465. Port 25 is often blocked altogether from end users connecting to any mail server but the one hosted by the ISP--but it's quite possible that the ISP may provide a firewalled mail service that is MITM to block spam from being relayed--but the middleware cannot work if the connection is encrypted--they cannot detect that the connection is from a botnet trying to encrypt the session or a regular computer (since the login request/response is encrypted). The way they are doing this means that they on the one hand can be assured that an infected machine cannot simply connect directly to thousands of mail servers to send spam--but on the other hand don't have to field all the customer support complaints about not being able to send emails at all and having to walk them through changing from port 25 to a different port number (port 587 at least is usually configured to require authentication even if the recipient is local to that server).

     

    reply to this | link to this | view in thread ]

  93.  
    identicon
    Anonymous Coward, Oct 14th, 2014 @ 5:41pm

    In Mother Russia...

    the blacklists wear YOU!

     

    reply to this | link to this | view in thread ]

  94.  
    identicon
    Spins Meats, Oct 14th, 2014 @ 6:22pm

    Re: Re:

    There's an awful lot of "security" devices out there that have protocol handlers for an awful lot of different applications. The fact that these ones are allowing ESMTP _at all_ is an improvement on 10 years ago.

    That said, these sorts of devices shouldn't be used on customer-facing networks. They're designed for corporates and other managed networks, for the most part.

    Hopefully this is idiocy rather than malice, but you wouldn't know either way.

     

    reply to this | link to this | view in thread ]

  95.  
    identicon
    me, Oct 14th, 2014 @ 6:44pm

    Mobile provider in Iran/Syria?

    Would anyone there even hope for encryption in any event?

     

    reply to this | link to this | view in thread ]

  96.  
    identicon
    Ross Presser, Oct 14th, 2014 @ 8:45pm

    Verizon FiOS does not permit port 25 to anything but its own servers.

     

    reply to this | link to this | view in thread ]

  97.  
    identicon
    Anonymous Coward, Oct 14th, 2014 @ 9:53pm

    Re:

    fucking idiot. That is all

     

    reply to this | link to this | view in thread ]

  98.  
    icon
    Kenpachi (profile), Oct 14th, 2014 @ 9:56pm

    Re: Re: Re: legitimate use?

    Amen to all, brother

     

    reply to this | link to this | view in thread ]

  99.  
    identicon
    Anonymous Coward, Oct 14th, 2014 @ 10:46pm

    So many NSA operatives commenting today. Never mind the man behind the curtains. Nothing to see here, move along.

     

    reply to this | link to this | view in thread ]

  100.  
    identicon
    Anonymous Coward, Oct 15th, 2014 @ 12:01am

    Re:

    "ISPs throttled a user's internet connection when they are downloading pirated content that they have not paid for."

    What if you pay for the pirated content, will they still throttle the connection?

     

    reply to this | link to this | view in thread ]

  101.  
    icon
    PaulT (profile), Oct 15th, 2014 @ 12:43am

    Re: Re:

    More to the point, how does the ISP know the copyright status of encrypted content, let alone whether the user has legally paid for it?

    Given that he's not bothered replying, and he's the sort of blinkered fool who can't imagine a non-entertainment usage for a VPN, I think we can safely ignore him now.

     

    reply to this | link to this | view in thread ]

  102.  
    icon
    Aethera (profile), Oct 15th, 2014 @ 1:13am

    Thank you

    I remember years ago as a sysadmin being stupefied at a mail server that was not functioning correctly. Everything checked out yet mail was not getting through. Only after breaking out the telnet mail.host.com 25 as above did I query the network admin and found out that the Cisco firewall's ESMTP security feature was messing with the application layer.

    It does seem kind of unusual that a wireless company's equipment would have such an option enabled or even that such an option would be present on their type of equipment.

     

    reply to this | link to this | view in thread ]

  103.  
    identicon
    J-P, Oct 15th, 2014 @ 1:26am

    throttling == connection speed

    He argued that, perhaps there could be some threshold, and if your traffic was above that threshold it's okay to throttle it.


    That sounds quite fair to me.

    Actually that sounds EXACTLY like it already IS currently.

    If you get 100Mbps connection, you get throttled at 100Mbps. With 10, the limit is 10 etc.

    If you get any less, you don't get what you are paying for (or the problem is at the other end).

     

    reply to this | link to this | view in thread ]

  104.  
    identicon
    Anonymous Coward, Oct 15th, 2014 @ 4:42am

    Re: Re: Categorically false

    Not really. antidirt and his gaggle of cocksucking fanboys are regular contenders for that role.

    That said, when even spambots post smarter things than you, it's pretty clear how low you've gone.

     

    reply to this | link to this | view in thread ]

  105.  
    icon
    Paul L (profile), Oct 15th, 2014 @ 5:10am

    Re: Re: Re: Re: Re: Re:

    I don't really like commenting here any more. I think TechDirt has gone off the deep end on a variety of topics. But it's good to see your response (as well as others) putting some actual real info out there containing research that SHOULD have been done before this article was ever published.

    But you're dead on. I've been using Cisco ASA firewalls all the way back to the old PIX models as well, and this is something that I've had to deal with myself a few times over the years. SMTP fixup can be a bitch, and the moment I saw the screenshot with the telnet session, I knew what was up.

    I've also seen configurations where they expect ANY TLS traffic to be submitted over 587, even though STARTTLS is just fine to use over port 25. But that doesn't seem to be what's going on here.

    Where I live, my ISP forces me to use THEIR mail server for all outgoing mail. They will relay anything, from any source address, as long as it's in their IP space. If I try to connect to port 25 outside of my ISP's network, the connection is always blocked. They implemented this as an anti-spam measure a few years ago.

     

    reply to this | link to this | view in thread ]

  106.  
    icon
    Josh in CharlotteNC (profile), Oct 15th, 2014 @ 7:18am

    Re: Re: Re: legitimate use?

    The issue is why a security-disabling MitM-attack feature exists at all.

    Because everyone's needs are different, and that feature is just a tool that has legitimate uses.

    and just happens to allow that corporation to log all the emails that went through their firewall.

    That is a perfectly legitimate use - if you've never worked in a highly regulated industry - like banking and financial services - then you might not be aware that there are regulations that require just that in some situations.

     

    reply to this | link to this | view in thread ]

  107.  
    identicon
    stoat, Oct 15th, 2014 @ 10:55am

    The real error

    Is that they allow endusers access to world:25 AT ALL.

    Most ISPs nullroute such traffic and with good reason - it's the #1 vector for spam traffic.

    Endusers (you and me) use MUAs, not MTAs. MUAs talk to MTAs on ports 993 (imaps) or 465/587 (smtp auth).

    Allowing endusers to directly connect to an external MTA allows spammers to do their thing, which in turn means the IPs involved get blacklisted. If you have a few clues to rub together and can demonstrate you can run a MTA without annoying the world then most ISPs will punch a hole for you.

    The real question is "Why on earth do people such as Golden Frog think they can run a MTA from a consumer connection, without proving they're competent to do so?"

    The fact that they can't interpret a PIX firewall's responses show that they lack that competence.

     

    reply to this | link to this | view in thread ]

  108.  
    identicon
    Christopher Bosak, Oct 15th, 2014 @ 1:36pm

    If they win, then boycott these ISP's out of business.

    If they truly want to charge us more then I say screw them. Let them be boycotted so bad that they go out of business. F-Them and their desire for complete control over the Internet.

     

    reply to this | link to this | view in thread ]

  109.  
    identicon
    chilling farts, Oct 15th, 2014 @ 1:50pm

    That's why FCC bribed peruvian regulator?

    Inside peru at year 2012 i explained something similar with ISP America Movil (aka Claro). While they tried to trick the old Glasnost test (used against them in a claim at 2011) some encrypted sites doesn't load correctly.

    Months later, they blocked the swedish and criticized service VPNTunnel via DNS. Affected service was mobile internet.

    The report was in spanish, essentially i discovered some differences on VPNTunnel launcher behavior filtering it's data via MS Network Monitor.

    https://es.scribd.com/doc/109732457/

    Oh, and Why i talked about FCC? Our regulator OSIPTEL published a neutrality bill last year to intensify infringements against net neutrality, but it got lost after a meeting with FCC in mid 2013. Months later, other bill opened some exceptions to net neutrality instead.

     

    reply to this | link to this | view in thread ]

  110.  
    identicon
    Cramer, Oct 15th, 2014 @ 5:50pm

    Re: Re:

    I guess none of the commenters have ever used a Cisco firewall (pix or asa) with SMTP inspection enabled. Because it does exactly what was just obfuscated in this "article". Until they name names and stop obscuring details, it's complete nonsense.

     

    reply to this | link to this | view in thread ]

  111.  
    identicon
    Cramer, Oct 15th, 2014 @ 5:59pm

    Re: Re: Throttling

    Verizon purposely let its connection clog and didn't do the most minor thing to connect another port to clear it up.
    And the fact that VPNs could "route around" congested peering links means Netflix doesn't know jack about traffic engineering. In other words, Netflix has every opportunity to direct users to netblocks that are routed through different paths, thus avoiding the issue entirely. Instead they like prefer to whine to the media in a lame attempt to get free peering and/or free colo hosting of their "OpenConnect" servers.

    (Don't fall for the BS. OpenConnect isn't "open"; it carries only Netflix traffic.)

     

    reply to this | link to this | view in thread ]

  112.  
    identicon
    franz mair, Oct 16th, 2014 @ 1:13am

    overwerite the starttls cmd ... wtf?

    why did they overwrite the starttls command??

    vim ...master.cf

    and set

    smtpd_tls_security_level=encrypt"

    to

    "smtpd_tls_security_level=may

    noobs...

     

    reply to this | link to this | view in thread ]

  113.  
    identicon
    Don Bowman, Oct 16th, 2014 @ 2:10pm

    Most ISP block port 25

    Most consumer ISP block port 25. It's a best practice. If you don't. If you don't, outbound spam comes from your network and you get blacklisted.

    This ISP appears to be running a spam scrubber instead, which is much better.

    I would rather have a port 25 enabled ISP w/o STARTTLS than a no-port-25 ISP.

    I would guess if you sign in to the SMTP server you will see it not do this modification.

     

    reply to this | link to this | view in thread ]

  114.  
    identicon
    John Thacker, Oct 17th, 2014 @ 7:31am

    Re: throttling == connection speed

    "Other end" is a bit of a simplification. The Internet isn't a bunch of point to point connections. The problem can be "in the middle," along any number of interconnection points (or points internal to one of many networks over which the packets travel.)

     

    reply to this | link to this | view in thread ]

  115.  
    identicon
    John Thacker, Oct 17th, 2014 @ 7:36am

    At this point, some news outlets would breathlessly reprint an article claiming that blocking spam from an open mail relay was a violation of net neutrality-- and I suppose by some definitions it technically is. I suppose John Gilmore supports his open mail relay for that very reason.

    Network neutrality seems like a simple concept, but if you don't take the strong Gilmore position that things like open mail relays should be allowed, actually writing regulations is pretty tricky (and having them be flexible enough to ban new bad things but also allow new good things.)

     

    reply to this | link to this | view in thread ]

  116.  
    identicon
    GEMont, Oct 17th, 2014 @ 6:40pm

    Criminals don't break law when law protects criminals

    Making monopolies illegal again might be the best way to slow this whole thing down long enough to get a good handle on it.

    Monopolies used to be illegal, for a very good reason.
    Does anyone remember what that reason is, or was?

    Monopolies are now a standard business model.
    Does anyone know why monopolies are now legal?
    Did the reason for their illegality go away?
    ---

     

    reply to this | link to this | view in thread ]

  117.  
    identicon
    joe, Oct 19th, 2014 @ 4:52pm

    Verizon blocking VPN

    they are definitely blocking VPN access to PrivateInternetAccess

     

    reply to this | link to this | view in thread ]

  118.  
    icon
    rtechie (profile), Oct 20th, 2014 @ 10:20am

    The article is 100% wrong

    I had to create an account here because this article is so misleading.

    I did some research and this is actually just a bug in that particular Cisco ASA:

    "Yes, if you upgrade to the newest firmware (version 8, my ASA is running 8.0(4)) then it support TLS in the esmtp inspection policy."

    It also may be that they're using the latest firmware, but they haven't explicitly enabled STARTTLS, which is turned off by default. https://stomp.colorado.edu/blog/blog/2012/12/31/on-smtp-starttls-and-the-cisco-asa/[1]

    So Golden Frog could have just contacted the ISP and asked them to fix their ASA.

    In any event, this is definitely not malicious on the part of the ISP. It is normal for consumer connections to interfere with SMTP server traffic from consumer hosts. This is to prevent spam.

    By having this appliance in place this ISP shows that they were being good to customers. The alternative is just to block all SMTP traffic, and many ISPs do that.

     

    reply to this | link to this | view in thread ]


Add Your Comment

Have a Techdirt Account? Sign in now. Want one? Register here
Get Techdirt’s Daily Email
Save me a cookie
  • Note: A CRLF will be replaced by a break tag (<br>), all other allowable HTML will remain intact
  • Allowed HTML Tags: <b> <i> <a> <em> <br> <strong> <blockquote> <hr> <tt>
Follow Techdirt
Advertisement
Essential Reading
Techdirt Reading List
Techdirt Insider Chat
Advertisement
Recent Stories
Advertisement

Close

Email This