Good bots, bad bots: friend or foe?
Credit to Author: Christopher Boyd| Date: Wed, 20 Feb 2019 16:00:00 +0000
One of the most talked about technologies online today is the ubiquitous bot. Simultaneously elusive yet also responsible for all of civilisation’s woes, bots are a hot topic of contention. If we went purely by news reports, we’d assume all bots everywhere are evil, and out to get us (or just spreading memes). We’d also assume every single person we ever disagreed with online is a bot.
It might surprise you to learn that not all bots are bad. You may only hear about the negatives, but they can be a genuine form of assistance for both people at home and in the workplace.
First, let’s pin down exactly what a bot is (and isn’t).
What is a bot?
Good question.
Bots, as we understand them, perform basic or complex tasks at a speed much faster than we humans can. They’re often there to prop up the bits of a process that humans can’t get to, keeping the plates spinning on our behalf.
The ones you’ve probably heard doing good deeds are search engine web crawlers, chatbots in Skype, Slack, or various forms of instant messaging, and even front line support queries for businesses.
The rest? Those could be bad, but their mileage may vary. More areas of business depend upon bots than you may think, and they’re increasingly being used for all manner of tasks. Some benefit people at home, others simply benefit the organisation running them. However they stack up, we’re going to look at some of the more common ones and give you some things to think about. If you’re putting your own bot together, we hope this will help.
Crawlers
Crawlers do exactly what the name suggests: they crawl. They weave their merry way across the Internet, grabbing, analysing, and cataloguing unimaginable amounts of data daily. Without them going about their business, many things we take for granted simply wouldn’t function as well as they do.
For example, search engine crawlers help us to flesh out search engines. If they didn’t do their job and do it well, you might never actually find the thing you were looking for. Search engine crawler stagnation essentially equals the same for your website—stagnation, marooned on an island of “doesn’t live here.” There are some cases where website owners may not want to be crawled, and they can block bot access via the Robots Exclusion Standard.
Robots.txt is a file you can place in your website directory to prevent specific content from being scraped. Essentially, the Robots.txt is itself a form of (ro)bot, politely turning visitors away. Want a specific example? Many people don’t want old versions of their websites recorded for all time. As a result, they may include a line in Robots to exclude Internet Archive to come calling and scrape the content.
Where this method often goes wrong is that the polite turn away is exactly that—too polite.
Rules: meant to be broken
When the bad bots show up, they’re likely to ignore the “we’re full, sorry” notice and just throw a chair through the window. In fact, some security people will suggest not bothering with a Robots.txt at all. The theory is that some rogue entities will deliberately look for it, and then immediately go poking around all the site portions the owner wanted to hide.
“Wait, which bots are the bad bots creeping around the Internet?!” I hear you cry. Well, there’s a lot of them and poor old Robots.txt file probably won’t be much help here. One of the best ways to tell a bad bot from a good one is to examine its behavior. Bad bot behavior includes:
- Brute force login attempts
- Content scraping to steal or mirror content
- Probing for hidden areas
- Overloading the website with traffic
- Vulnerability hunting: looking to exploit outdated apps, plugins, or content management systems
Even if you think your website is up-to-date, the server it runs on may not be, which means the bot issue is likely out of your hands. There’s a lot to contend with for a website admin.
Not all is lost, however. You can make use of a variety of scanning tools to mimic bot behaviour and see which form of bad bottery you’re most susceptible to. At that point, you can apply the correct fix as required.
Good, bad, or somewhere in-between
For some people, lines may blur a little between good bots versus bad ones. The most basic of interactions can produce all manner of knock on effects. For example:
Imagine your site is attacked by a content scraper, and all your hard work ends up on a cut and paste merchant’s website. Not cool. You then sign up to a copyright detection bot service, which crawls the web in search of your pilfered text. The scammer running the site has a block in its robots.txt file explicitly requesting the copyright sniffer not to come knocking. At this point, the bot is fully justified in avoiding the polite request to go in, scan the text, then report back to base that someone’s been up to no good. Your bot is now breaking the rules, and you’re tainted with justified wrongdoing forever.
Beating the system
Additionally, search engines can be gamed. SEO poisoning, where rogue links are included in results, was a problem for a long time before major providers started clamping down (with variable success). Even so, there are variations on these attempts. And outside of those, you still have the threat of compromised sites giving bad portals a boost.
If your organisation intends to deploy a web-scraping bot of its own, you may want to keep some of these developments in mind. It’s a fine line between helpful and nuisance, and not all rival bots play nice. It only takes a few mishaps with another org’s service or website, and you’ve got a major PR issue to deal with.
Time for a chat?
Chatbots have been around for a long time. The first was ELIZA, created in 1966 by Joseph Weizenbaum. While he considered ELIZA to highlight the superficiality of human/computer interaction, he was surprised at people attributing human emotions to the dialogue. Wind forward a couple of decades, and you have Roman Mazurenko turned into a chatbot for friends and family to interact with after his tragic early death. Years later, the same questions are being asked in terms of where the line is drawn, and whether such interactions are even healthy.
Many people think of chatbots (at least the good ones) as a recent development. However, chatbots have been used for some time for nefarious purposes—the first thing that springs to mind is pornography spam bots asking for credit card details. Quite often, that association is accompanied by thoughts of of malware and other shenanigans. Spreading out from forums and old-style chatrooms/IRC to instant messaging platforms and social media, bots have improved in their ability to actually help, instead of pilfer data or infect machines.
Often sporting limited phrases and becoming the butt of endless “look at me fool this spam bot” jokes, many businesses didn’t bother to invest in bots because the technology wasn’t there. Nowadays, you’ll find decent bot assistance for everything from shopping portals and banking to utility service providers.
Healthy living
Even Microsoft are in on the action at this point, with their Microsoft Healthcare Bot. This allows providers to customise their own AI-driven bot solution and roll it out to customers and clients. Elsewhere, chat-centric health bots are clearly seen as the future of medical assistance, with everything from therapy to simple daily reminders to take your pills. This view may be a little optimistic, as the potential for incorrect diagnosis or faulty advice is there. Integration with household IoT devices known to occasionally glitch out could increase that possibility. However, this is a clear use-case for mostly maligned bot technology as a force for good.
Fun for all the family?
Chatbots for children/teens are also a big thing now. Many of them are integrated with Facebook messenger, and will allow them to talk some Hearthstone, Marvel, or (for the older bot fans) converse with an AI replica of a dead horror movie character.
Ad fraud
Ad fraud is something that seems to have been around as long as ads themselves. Bots automate the process of clicking ads to provide a bump in income for the person who placed the ad. The more clicks, the more revenue generated. This is most commonly accomplished by infecting as many PCs as possible, then using those PCs to click ads.
There’s been many ad fraud trends over the years. One of the biggest I can remember is the rush to profit from high pay-outs on the word “Mesothelioma,” a rare form of cancer related to asbestos. For this, websites hijacked IE users, infected their PCs, and used instant messaging to send bad links while opening the ads in the unaware user’s browser.
Quite sophisticated, and apart from scale and profit, nothing much has changed. Ad fraud is entirely harmful, and often goes hand-in-hand with malvertising and ransomware attacks. These bots were designed to do bad, and they are accomplishing what they were meant to do.
Snipers in commerce land
Let the bidding wars begin! Automated commerce tools are pretty cut and dry. Not everyone wants web pages crawling, but you aren’t really going to lose out to someone in direct competition. Company X may use chatbots and your business doesn’t, but some customers will prefer the human touch and vice-versa. It isn’t going to make or break anything, particularly.
Where sales are concerned though, it’s pretty black and white. Where cash is involved, anything can happen and usually does. It’s a long time since scammers used bots to “buy” from other bots and bump up fake reputations, and that was quickly replaced in popularity by sniper tools.
Sniping tools have been around for a long time, and are somewhat controversial in seller circles. The basic idea is to give the sniper tool access to your eBay account (or any other bidding service), and at the very last moment before a sale ends, it’ll throw in your bid. Rivals are unable to counter because there’s nothing they can do about an automated service working to nanoseconds instead of a human hammering at a keyboard. So is this bad? For the other users, yes. For eBay as a platform, absolutely. Overall? Remains to be seen.
Fending off the bad bidders
Fixed price sales are a bidding bot’s worst enemy, because there’s nothing to gamble. Take it or leave it at the listed price. Some sites will offer a time extension if a last minute bid comes in, which may or may not help ward off the snipers. One of the biggest drawbacks to sniping is you often must hand over login details to the sniping tool. Do you trust it? Is it safe? Can the people who operate the service see your credentials? All of this and more are natural drawbacks to sniping, and could keep your business on top of those grabbing all the best items.
In the digital space of non-tangible goods, bidding and trading also reigns supreme. Sadly, it comes with major risks. Steam, the video game platform juggernaut, offers its own marketplace. There, you can buy all manner of in-game items, cosmetics, game cards, and so on. Some of these items sell for pennies and cents, others fetch hundreds of pounds and dollars.
A short-lived victory
One enterprising individual made a trading bot for the Steam marketplace, and spent some time buying low and selling high across three separate Steam accounts. Ultimately, they amassed game items worth $10,000, which included 2,261 Team Fortress 2 keys.
Valve discovered the botting antics, and subsequently banned all accounts and deleted all the items. Yes, all ten thousand dollars’ worth. This is a clear case of gaming the system and would have also arguably impacted others. While this may have caused a few people to grab some items at a lower price, overall, it’s tough to call this one an example of a good bot (except maybe for the creator).
Bots by any other name
Most of our examples are essentially quite crude bots, living out their days simply sniffing the web or making the occasional product bid. There’s a big push for bots on your devices instead of scouring the web, mostly in the form of personal digital assistants. To a large degree, any regular mobile device does a lot of this anyway (Ahem, hi Siri!). Personalising said tasks and wrapping them up under a friendly interface is the name of the game.
As with other bot types, much of the information you’ll come across online is aimed at the bad stuff. That’s fine—it’s usually easier to spot things getting up to no good than invisible processes ticking along in the background harming nobody. Even so, plugging “mobile bots” into Google brings back nothing but bad bots, mobile game hijacks, scams, and more bad stuff. There are a few hints as to how this new realm of bot may play out as a force for good, including some outside of the mobile world, that illustrate the positive directions botting could move in.
While the word “bot” may never quite shake its negative associations, it’s absolutely worth revisiting and re-evaluating the next time your work colleagues mention a cool new bot program they’ve been assigned. Who knows, you may even give them some helpful suggestions to get the ball rolling.
The post Good bots, bad bots: friend or foe? appeared first on Malwarebytes Labs.