Looks like you're enjoying the discussion, but you're not signed up for an account. When you create an account, we remember exactly what you've read, so you always come right back where you left off. You also get notifications, here and via email, whenever new posts are made. You can like posts to share the love. 
Join 
Hiding from Anti-Malware search bots
- http://krebsonsecuri...re-search-bots/
April 23, 2010 - "Malicious hackers spend quite a bit of time gaming the Internet search engines in a bid to have their malware-laden sites turn up on the first page of search results for hot, trending news topics. Increasingly, though, computer criminals also are taking steps to block search engines bots from indexing legitimate Web pages that have been hacked and booby-trapped with hostile code. Search giants Yahoo! and Google each have automated programs that crawl millions of Web sites each week in search of those hosting malicious code. When the search providers find these sites, they typically append a warning to the hacked Web site’s listing in search results, alerting the would-be visitor that the site could be dangerous. These warnings not only result in fewer people visiting infected sites, but they have a tendency to alert a listed site’s owners to a malware problem that needs attention...
“So basically the malware was checking if the user agent was from the Google or Yahoo bot and not returning the malware on that case,” wrote Dede, a security expert from Brazil [who maintains the blog http://blog.sucuri.n...rom-google.html ]. Meanwhile, regular visitors to the infected sites received malicious Javascript that tried to foist malware, Dede found.
Denis Sinegubko, a Russian researcher [with the blog http://UnmaskParasites.com ], recently has documented at least two examples of malware stitched into blogs that will modify the host site to hide malicious redirects from Google’s search bots. “And the fact that I can see many such blogs in Google search results without any warnings shows that this simple trick does its job,” Sinegubko wrote.
Google’s search experts say they’re aware of and constantly counteracting these types of obfuscation techniques. Niels Provos, principal software engineer at Google, said cyber crooks frequently try to play both sides, by attempting to block search bots from finding malware stitched into hacked sites, while simultaneously gaming the search engine bots. “This has been going on for some time. What happens is if a Web crawler comes along, [the attackers will configure the hacked site so that it] ends up showing [trending content] they get from news sites,” Provos said. “This is to game the ranking of search content. But then if the visitor comes to one of these sites via a search engine, he ends up getting exploit code.” Provos declined to discuss the specific steps Google takes to combat these tactics, noting that the fight with these Web site hackers is a constant arms race..."
