Help with suspicious activity?

[Spacetrucker]

I need some help. Here’s the problems. We have folks either intentionally or unintentionally scalping data off of our websites using automated routines they’ve written. They’re making an http request once a second or so. I’ve done things like find the ip address and block it only to have another ip address from the same subnet pick up the scalping routine. Can the version of Smart View Monitor or (SVM Lite) that comes standard with R60 help us solve this problem? One, I need to be alerted via email, pager, cell phone that the event is occurring and two, I need an automated response to the event. An idea I have is that if we get x number of hits per minute for 5 continuous minutes from a single ip address the requests from that ip address or throttled back to 1 answer per minute or dropped for 60 minutes.

[Thorpuse]

SmartDefense has a defense for successive multiple connections. This sounds like the right tool for the job, otherwise it's a job better suited to Eventia Analyser or a similar real-time log analysis tool.

[Spacetrucker]

Thanks Thorpuse, I'll have a look at the tools you've recommended.

[varera]

Agree, Eventia analyzer looks like a solution for your case. You can create some customized event policy triggered by certain amount of http opened from the same source during certain period of time.

[Spacetrucker]

All - I've found the successive connections category in Smartdefense. I've gotten the available doc's on configuring user defined alerts from Secure Knowledge. Most are locked down and say you need advanced access to view them. I have sk27180, sk27318, and sk26129. Suggestions on others that would be helpful would be appreciated.

And, would someone help me out with an example of a script that would send an alert to my pager and cell phone? We have an R60 SPLAT box.

[dsb.nepo]

I have no idea how to implement this with CP (i own no eventia license)

Is this a high loaded server? which OS?

I do something what you like to implement directly at the machine self, but it depends what OS and webserver is implemented.

It is also possible to implement this with a bridge in front of the webserver (but this means additional work)

If you can identify the user agent you can also extend smart defense and drop the requests with basic regular expressions!

[Spacetrucker]

Originally Posted by dsb.nepo

I have no idea how to implement this with CP (i own no eventia license)

Is this a high loaded server? which OS?

I do something what you like to implement directly at the machine self, but it depends what OS and webserver is implemented.

It is also possible to implement this with a bridge in front of the webserver (but this means additional work)

If you can identify the user agent you can also extend smart defense and drop the requests with basic regular expressions!

SPLAT R60 Kernal version 2.4.21-20cp. I don't know what you mean when you say high loaded server? The web servers are Windows 2003 IIS version 6 if that helps.

[dsb.nepo]

With high loaded server I mean a application server that serves more than 100+ pages/second

At the IIS site you can implement code that protects the server
Prevent Denial of Service (DOS) attacks in your web application - Omar AL Zabir blog on ASP.NET Ajax and .NET 3.5

With a bridge/additional machine in front you can do this for example with the following lines at a *BSD machine and pf.

table <bad_hosts> persist

block quick from <bad_hosts>                                         
pass in on $ext_if proto tcp to $webserver port www keep state \     
        (max-src-conn-rate 100/10, overload <bad_hosts> flush global)

If the user-agent is for example 'wget' or 'curl' you can block the request for example in R65 (don't know if this is possible with R60) in smartdefense.

[smartdefense]->[web intelligence]->[http protocol ...]->[Header rejection]->[edit]
define a regular expression for this user-agent here

[Spacetrucker]

Thanks for the link and explanation on high loaded server. I'm looking into your suggestions. I'm sending the link to Omar's ASP code over to our developers for review.

[Spacetrucker]

I got a tip from a guy over in EE.
Smartdefense -> Network Security -> IP and ICMP -> Network Quota

What do you guys think about it?