Malware? We don't need no stinking malware!

Friday, October 24, 2008 2:25 PM



"This site may harm your computer"
You may have seen those words in Google search results — but what do they mean? If you click the search result link you get another warning page instead of the website you were expecting. But if the web page was your grandmother's baking blog, you're still confused. Surely your grandmother hasn't been secretly honing her l33t computer hacking skills at night school. Google must have made a mistake and your grandmother's web page is just fine...



I work with the team that helps put the warning in Google's search results, so let me try to explain. The good news is that your grandmother is still kind and loves turtles. She isn't trying to start a botnet or steal credit card numbers. The bad news is that her website or the server that it runs on probably has a security vulnerability, most likely from some out-of-date software. That vulnerability has been exploited and malicious code has been added to your grandmother's website. It's most likely an invisible script or iframe that pulls content from another website that tries to attack any computer that views the page. If the attack succeeds, then viruses, spyware, key loggers, botnets, and other nasty stuff will get installed.

If you see the warning on a site in Google's search results, it's a good idea to pay attention to it. Google has automatic scanners that are constantly looking for these sorts of web pages. I help build the scanners and continue to be surprised by how accurate they are. There is almost certainly something wrong with the website even if it is run by someone you trust. The automatic scanners make unbiased decisions based on the malicious content of the pages, not the reputation of the webmaster.

Servers are just like your home computer and need constant updating. There are lots of tools that make building a website easy, but each one adds some risk of being exploited. Even if you're diligent and keep all your website components updated, your web host may not be. They control your website's server and may not have installed the most recent OS patches. And it's not just innocent grandmothers that this happens to. There have been warnings on the websites of banks, sports teams, and corporate and government websites.

Uh-oh... I need help!
Now that we understand what the malware label means in search results, what do you do if you're a webmaster and Google's scanners have found malware on your site?

There are some resources to help clean things up. The Google Webmaster Central blog has some tips and a quick security checklist for webmasters. Stopbadware.org has great information, and their forums have a number of helpful and knowledgeable volunteers who may be able to help (sometimes I'm one of them). You can also use the Google SafeBrowsing diagnostics page for your site (http://www.google.com/safebrowsing/diagnostic?site=<site-name-here>) to see specific information about what Google's automatic scanners have found. If your site has been flagged, Google's Webmaster Tools lists some of the URLs that were scanned and found to be infected.

Once you've cleaned up your website, use Google's Webmaster Tools to request a malware review. The automatic systems will rescan your website and the warning will be removed if the malware is gone.

Advance warning
I often hear webmasters asking Google for advance warning before a malware label is put on their website. When the label is applied, Google usually emails the website owners and then posts a warning in Google's Webmaster Tools. But no warning is given ahead of time - before the label is applied - so a webmaster can't quickly clean up the site before a warning is applied.

But, look at the situation from the user's point of view. As a user, I'd be pretty annoyed if Google sent me to a site it knew was dangerous. Even a short delay would expose some users to that risk, and it doesn't seem justified. I know it's frustrating for a webmaster to see a malware label on their website. But, ultimately, protecting users against malware makes the internet a safer place and everyone benefits, both webmasters and users.

Google's Webmaster Tools has started a test to provide warnings to webmasters that their server software may be vulnerable. Responding to that warning and updating server software can prevent your website from being compromised with malware. The best way to avoid a malware label is to never have any malware on the site!

Reviews
You can request a review via Google's Webmaster Tools and you can see the status of the review there. If you think the review is taking too long, make sure to check the status. Finding all the malware on a site is difficult and the automated scanners are far more accurate than humans. The scanners may have found something you've missed and the review may have failed. If your site has a malware label, Google's Webmaster Tools will also list some sample URLs that have problems. This is not a full list of all of the problem URLs (because that's often very, very long), but it should get you started.

Finally, don't confuse a malware review with a request for reconsideration. If Google's automated scanners find malware on your website, the site will usually not be removed from search results. There is also a different process that removes spammy websites from Google search results. If that's happened and you disagree with Google, you should submit a reconsideration request. But if your site has a malware label, a reconsideration request won't do any good — for malware you need to file a malware review from the Overview page.



How long will a review take?
Webmasters are eager to have a Google malware label removed from their site and often ask how long a review of the site will take. Both the original scanning and the review process are fully automated. The systems analyze large portions of the internet, which is big place, so the review may not happen immediately. Ideally, the label will be removed within a few hours. At its longest, the process should take a day or so.

New spam and virus trends from Enterprise

Tuesday, August 12, 2008 2:01 PM



The Google Apps Security & Compliance team, which provides email and web security for more than 40,000 companies, regularly tracks trends in spam, viruses, and other threats. Check out some of our latest findings over on the Enterprise blog. Also, on Friday, August 15, at 10:00 am PT, we'll be hosting a webinar on keeping your business safe from web and email threats -- tune in if you'd like to learn more.

Keyczar: Safe and Simple Cryptography

Monday, August 11, 2008 11:06 AM



Cryptography is notoriously hard to get right and if improperly used, can create serious security holes. Common mistakes include using the wrong cipher modes or obsolete algorithms, composing primitives in an unsafe manner, hard-coding keys in source code, or failing to anticipate the need for future key rotation. With these risks in mind, we're pleased to announce the open-source release of Keyczar.

Keyczar is a cryptographic toolkit that supports encryption and authentication for both symmetric and public-key algorithms. It addresses some of the aforementioned issues by choosing safe defaults, tagging outputs with key version information, and providing a simple application programming interface. Keyczar's key versioning system makes it easy to rotate and revoke keys, without worrying about backward compatibility or making any changes to source code.

We look forward to working with the open source community and continuing to make cryptography safer and easier to use. To download Keyczar or for more information, please visit our Google Code project and discussion group.

Are you using the latest web browser?

Wednesday, July 16, 2008 1:24 PM



In view of mass defacements of hundreds of thousand of web pages - with the intent to misuse them to launch drive-by download attacks - security researchers from ETH Zurich, Google, and IBM Internet Security Systems were interested in looking at the other side of the attack: the web browser. By analyzing the web browser versions seen in visits to Google websites, they have shown that more than 600 million Internet users don't use the latest version of their browser.

Slow migration to latest browser version
The researchers' paper, entitled "Understanding the Web Browser Threat", shows that as of June 2008, only 59.1% percent of Internet users worldwide use the latest major version of their preferred web browser. Firefox users are the most attentive: 92.2% of them surfed with Firefox 2, the latest major version before the recently released 3.0. Only 52.5% of Microsoft Internet Explorer users have updated to version 7, which is the most secure according to multiple publicly-cited Microsoft experts (among them Sandi Hardmeier). The study revealed that 637 million Internet users worldwide who use web browsers are either not running the latest version of their preferred browser or have not installed the latest patches. These users are vulnerable to exploitation due to their web browser's "built-in" vulnerabilities and the lack of more recent security mechanisms such as improved phishing protection.

Neglected security patches
Over the past 18 months, the study also shows, a maximum of 83.3% of Firefox users were using the latest major version of the web browser and also had all current patches installed (i.e. latest minor version). Only 56.1% and 47.6% of Opera and Internet Explorer users, respectively, were similarly utilizing fully-patched web browsers. Apple users are no better: since the public release of Safari 3, only 65.3% of users operate the latest Safari version.


Maximum measured share of users surfing the web with the most secure versions of Firefox, Safari, Opera and Internet Explorer in June 2008 as seen on Google websites.


Obsolete browser warning
The study's most important finding is that technical measures now in place do not sufficiently guarantee browser security, and that users' security awareness must be further developed. The problem is that most users are unaware that they are not using their browser's latest version. It must be made clear to web browser users that outdated software is associated with significantly higher risk. The researchers therefore suggest that, as a critical component of web software, a visible warning be instituted that warns the user of missing security patches in a way analogous to the 'best before' date in the perishable food industry. Software updates must also be made easier to find. The resulting transparency would go far in contributing to end user awareness of software weaknesses, and allow users to better evaluate risks.


Example "best before" implementation on a Web browser


As a side effect, having users migrate faster to the latest browser version would not only increase security but also make the lives of webmasters easier, as they would need to test and optimize websites for fewer older versions of web browsers.

Meet ratproxy, our passive web security assessment tool

Tuesday, July 1, 2008 4:49 PM



We're happy to announce that we've just open-sourced ratproxy, a passive web application security assessment tool that we've been using internally at Google. This utility, developed by our information security engineering team, is designed to transparently analyze legitimate, browser-driven interactions with a tested web property and automatically pinpoint, annotate, and prioritize potential flaws or areas of concern.

The proxy analyzes problems such as cross-site script inclusion threats, insufficient cross-site request forgery defenses, caching issues, cross-site scripting candidates, potentially unsafe cross-domain code inclusion schemes and information leakage scenarios, and much more. (A more-detailed discussion of these features and information on securing vulnerable applications is provided here.) Compared with more-traditional active crawlers, or with fully manual request inspection and modification frameworks, this approach offers several significant advantages in terms of minimized overhead; marginalized risk of site disruptions; high coverage of complex, client-driven application states in web 2.0 solutions; and insight into dynamic cross-domain trust models.

We decided to make this tool freely available as open source because we feel it will be a valuable contribution to the information security community, helping advance the community's understanding of security challenges associated with contemporary web technologies. We believe that responsible security research brings a net overall benefit to the safety of the Web as a whole, and have released this tool explicitly to support that kind of research.

To download the proxy, please visit this page. Also, please keep in mind that the proxy is designed solely to highlight interesting patterns in web applications, and a further analysis by a security professional is often required to interpret the results and their significance for the tested platform.

Safe Browsing Diagnostic To The Rescue

Thursday, May 15, 2008 1:49 PM



We've been protecting Google users from malicious web pages since 2006 by showing warning labels in Google's search results and by publishing the data via the Safe Browsing API to client programs such as Firefox and Google Desktop Search. To create our data, we've built a large-scale infrastructure to automatically determine if web pages pose a risk to users. This system has proven to be highly accurate, but we've noted that it can sometimes be difficult for webmasters and users to verify our results, as attackers often use sophisticated obfuscation techniques or inject malicious payloads only under certain conditions. With that in mind, we've developed a Safe Browsing diagnostic page that will provide detailed information about our automatic investigations and findings.

The Safe Browsing diagnostic page of a site is structured into four different categories:

  1. What is the current listing status for [the site in question]?

    We display the current listing status of a site and also information on how often a site or parts of it were listed in the past.

  2. What happened when Google visited this site?

    This section includes information on when we analyzed the page, when it was last malicious, what kind of malware we encountered and so fourth.   To help web masters clean up their site, we also provide information about the sites that were serving malicious software to users and which sites might have served as intermediaries.

  3. Has this site acted as an intermediary resulting in further distribution of malware?

    Here we provide information if this site has facilitated the distribution of malicious software in the past. This could be an advertising network or statistics site that accidentally participated in the distribution of malicious software.

  4. Has this site hosted malware?

    Here we provide information if the the site has hosted malicious software in the past. We also provide information on the victim sites that initiated the distribution of malicious software.


All information we show is historical over the last ninety days but does not go further into the past.   Initially, we are making the Safe Browsing diagnostic page available in two ways.  We are adding a link on the interstitial page a user sees after clicking on a search result with a warning label, and also via an "additional information" link in Firefox 3's warning page. Of course, for anyone who wants to know more about how our detection system works, we also provide a detailed tech report [pdf] including an overview of the detection system and in-depth data analysis.

Contributing To Open Source Software Security

Monday, May 5, 2008 11:38 AM



From operating systems to web browsers, open source software plays a critical role in the operation of the Internet. The security of open source software is therefore quite important, as it often interacts with personal information -- ranging from credit card numbers to medical records -- that needs to be kept safe. There has been a long-lived discussion on whether open source software is inherently more secure than closed source software. While popular opinion has begun to tilt in favor of openness, there are still arguments for both sides. Instead of diving into those treacherous waters (or giving weight to the idea of "inherent security"), I'd like to focus on the fruits of this extensive discussion. In particular, David A. Wheeler laid out a "bottom line" in his Secure Programming for Linux and Unix HOWTO which applies to both open and closed source software. It predicates real security in software on three actions:

  1. people need to actually review the code

  2. developers/reviewers need to know how to write secure code

  3. once found, security problems need to be fixed quickly, and their fixes distributed quickly


While distilling anything down to three steps makes it seem easy, this isn't necessarily the case. Given how important open source software is to Google, we've attempted to contribute to this bottom line. As Chris said before, our engineers are encouraged to contribute both software and time to open source efforts. We regularly submit the results of our automated and manual security analysis of open source software back to the community, including related software engineering time. In addition, our engineering teams frequently release software under open source licenses. This software was written either with security in mind, such as with security testing tools, or by engineers well-versed in the security challenges of their project.

These efforts leave one area completely unaddressed -- getting security problems fixed quickly, and then getting those fixes distributed quickly. It has been unclear how to best resolve this issue. There is no centralized security authority for open source projects, and operating system distribution publishers are the best bet for getting updates to the highest number of users. Even if users can get updates in this manner, how should a security researcher contact a particular project's author? If there's a potential, security-related issue, who can help evaluate the risk for a project? What resources are there for projects that have been compromised, but have no operational security background?

I'm proud to announce that Google has sponsored participation in oCERT, the open source computer emergency response team. oCERT is a volunteer workforce of security professionals from the open source community with the goal of providing security vulnerability mediation and incident response services to open source projects. It will strive to contact software authors with all security reports and aid in debugging and patching, especially in cases where the author, or the reporter, doesn't have a background in security. Reliable contacts for projects, publishers, and vendors will be maintained where possible and used for notification when issues arise and fixes are available for mediated issues. Additionally, oCERT will aid projects of any size with responses to security incidents, such as server compromises.

It is my hope that this initiative will not only aid in remediating security issues in a timely fashion, but also provide a means for additional security contributions to the open source community.

All Your iFrame Are Point to Us

Monday, February 11, 2008 1:57 PM



It has been over a year and a half since we started to identify web pages that infect vulnerable hosts via drive-by downloads, i.e. web pages that attempt to exploit their visitors by installing and running malware automatically. During that time we have investigated billions of URLs and found more than three million unique URLs on over 180,000 web sites automatically installing malware. During the course of our research, we have investigated not only the prevalence of drive-by downloads but also how users are being exposed to malware and how it is being distributed. Our research paper is currently under peer review, but we are making a technical report [PDF] available now. Although our technical report contains a lot more detail, we present some high-level findings here:

Search Results Containing a URL Labeled as Harmful


The above graph shows the percentage of daily queries that contain at least one search result labeled as harmful. In the past few months, more than 1% of all search results contained at least one result that we believe to point to malicious content and the trend seems to be increasing.

Browsing Habits

Good computer hygiene, such as running automatic updates for the operating system and third-party applications, as well as installing anti-virus products goes a long way in protecting your home computer. However, we have been wondering if users' browsing habits impact the likelihood of encountering malicious web pages. To study this aspect, we took a sample of ~7 million URLs and mapped them to DMOZ categories. Although we found that adult web pages may increase the risk of exploitation, each DMOZ category was affected.

Malicious Content Injection

To understand if malicious content on a web server is due to poor web server security, we analyzed the version numbers reported by web servers on which we found malicious pages. Specifically, we looked at the Apache and the PHP versions exported as part of a server's response. We found that over 38% of both Apache and PHP versions were outdated increasing the risk of remote content injection to these servers.

Our "Ghost In the Browser [PDF]" paper highlighted third-party content as one potential vector of malicious content. Today, a lot of third-party content is due to advertising. To assess the extent to which advertising contributes to drive-by downloads, we analyze the distribution chain of malware, i.e. all the intermediary URLs a browser downloads before reaching a malware payload. We inspected each distribution chain for membership in about 2,000 known advertising networks. If any URL in the distribution chain corresponds to a known advertising network, we count the whole page as being infectious due to Ads. In our analysis, we found that on average 2% of malicious web sites were delivering malware via advertising. The underlying problem is that advertising space is often syndicated to other parties who are not known to the web site owner. Although non-syndicated advertising networks such as Google Adwords are not affected, any advertising networks practicing syndication needs to carefully study this problem. Our technical report [PDF] contains more detail including an analysis based on the popularity of web sites.

Structural Properties of Malware Distribution


Finally, we also investigated the structural properties of malware distribution sites. Some malware distribution sites had as many as 21,000 regular web sites pointing to them. We also found that the majority of malware was hosted on web servers located in China. Interestingly, Chinese malware distribution sites are mostly pointed to by Chinese web servers.

We hope that an analysis such as this will help us to better understand the malware problem in the future and allow us to protect users all over the Internet from malicious web sites as best as we can. One thing is clear - we have a lot of work ahead of us.

Help us fill in the gaps!

Thursday, November 29, 2007 2:28 PM



We've been targeting malware for over a year and a half, and these efforts are paying off. We are now able to display warnings in search results when a site is known to be malicious, which can help you avoid drive-by downloads and other computer compromises. We are already distributing this data through the Safe Browsing API, and we are working on bringing this protection to more users by integrating with more Google products. While these are great steps, we need your help going forward!

Currently, we know of hundreds of thousands of websites that attempt to infect people's computers with malware. Unfortunately, we also know that there are more malware sites out there. This is where we need your help in filling in the gaps. If you come across a site that is hosting malware, we now have an easy way for you to let us know about it. If you come across a site that is hosting malware, please fill out this short form. Help us keep the internet safe, and report sites that distribute malware.

Auditing open source software

Monday, October 8, 2007 4:13 PM



Google encourages its employees to contribute back to the open source community, and there is no exception in Google's Security Team. Let's look at some interesting open source vulnerabilities that were located and fixed by members of Google's Security team. It is interesting to classify and aggregate the code flaws leading to the vulnerabilities, to see if any particular type of flaw is more prevalent.
  1. JDK. In May 2007, I released details on an interesting bug in the ICC profile parser in Sun's JDK. The bug is particularly interesting because it could be exploited by an evil image. Most previous JDK bugs involve a user having to run a whole evil applet. The key parts of code which demonstrate the bug are as follows:

    TagOffset = SpGetUInt32 (&Ptr);
    if (ProfileSize < TagOffset)
      return SpStatBadProfileDir;
    ...
    TagSize = SpGetUInt32 (&Ptr);
    if (ProfileSize < TagOffset + TagSize)
      return SpStatBadProfileDir;
    ...
    Ptr = (KpInt32_t *) malloc ((unsigned int)numBytes+HEADER);

    Both TagSize and TagOffset are untrusted unsigned 32-bit values pulled out of images being parsed. They are added together, causing a classic integer overflow condition and the bypass of the size check. A subsequent additional integer overflow in the allocation of a buffer leads to a heap-based buffer overflow.

  2. gunzip. In September 2006, my colleague Tavis Ormandy reported some interesting vulnerabilities in the gunzip decompressor. They were triggered when an evil compressed archive is decompressed. A lot of programs will automatically pass compressed data through gunzip, making it an interesting attack. The key parts of the code which demonstrate one of the bugs are as follows:

    ush count[17], weight[17], start[18], *p;
    ...
    for (i = 0; i < (unsigned)nchar; i++) count[bitlen[i]]++;

    Here, the stack-based array "count" is indexed by values in the "bitlen" array. These values are under the control of data in the incoming untrusted compressed data, and were not checked for being within the bounds of the "count" array. This led to corruption of data on the stack.


  3. libtiff. In August 2006, Tavis reported a range of security vulnerabilities in the libtiff image parsing library. A lot of image manipulation programs and services will be using libtiff if they handle TIFF format files. So, an evil TIFF file could compromise a lot of desktops or even servers. The key parts of the code which demonstrate one of the bugs are as follows:

    if (sp->cinfo.d.image_width != segment_width ||
        sp->cinfo.d.image_height != segment_height) {
      TIFFWarningExt(tif->tif_clientdata, module,
        "Improper JPEG strip/tile size, expected %dx%d, got %dx%d",

    Here, a TIFF file containing a JPEG image is being processed. In this case, both the TIFF header and the embedded JPEG image contain their own copies of the width and height of the image in pixels. This check above notices when these values differ, issues a warning, and continues. The destination buffer for the pixels is allocated based on the TIFF header values, and it is filled based on the JPEG values. This leads to a buffer overflow if a malicious image file contains a JPEG with larger dimensions than those in the TIFF header. Presumably the intent here was to support broken files where the embedded JPEG had smaller dimensions than those in the TIFF header. However, the consequences of larger dimensions that those in the TIFF header had not been considered.

We can draw some interesting conclusions from these bugs. The specific vulnerabilities are integer overflows, out-of-bounds array accesses and buffer overflows. However, the general theme is using an integer from an untrusted source without adequately sanity checking it. Integer abuse issues are still very common in code, particular code which is decoding untrusted binary data or protocols. We recommend being careful using any such code until it has been vetted for security (by extensive code auditing, fuzz testing, or preferably both). It is also important to watch for security updates for any decoding software you use, and keep patching up to date.