Register | Login
Views: 19364387
Main | Memberlist | Active users | ACS | Commons | Calendar | Online users
Ranks | FAQ | Color Chart | Photo album | IRC Chat
11-02-05 12:59 PM
0 user currently in Acmlmboard support?.
Acmlm's Board - I2 Archive - Acmlmboard support? - Search Bots - any way to block them? | |
Add to favorites | "RSS" Feed | Next newer thread | Next older thread
User Post
Gywall

Silver axe
Level: 30

Posts: 150/356
EXP: 164069
For next: 1800

Since: 03-15-04
From: In front of my moniter, where you're not!

Since last post: 104 days
Last activity: 14 hours
Posted on 11-09-04 09:19 PM Link | Quote
other than ip banning search bots, is there any other way to stop them from visiting?

I've had enough bots already. I just want to block them for being bandwith eating asses. (I've hit a record for bandwith this month, 50 MB in two days from a SMALL board)
Kitten Yiffer

Purple wand
Furry moderator
Vivent l'exp����¯�¿�½������©rience de signalisation d'amusement, ou bien !
Level: 135

Posts: 5945/11162
EXP: 28824106
For next: 510899

Since: 03-15-04
From: Sweden

Since last post: 3 hours
Last activity: 4 min.
Posted on 11-10-04 12:56 AM Link | Quote
Tell the robots what to do instead of blocking them.

Just restrict them to the index.php or something, and they hopefully won't drain bandwidth. :/
Sandy53215
Acmlm (10:55:31 PM): they're having fun for the first time in so long
Level: 47

Posts: 215/948
EXP: 713034
For next: 53169

Since: 03-15-04
From: Milwaukee, Wisconsin (U.S.A)

Since last post: 1 day
Last activity: 4 hours
Posted on 11-10-04 05:16 AM Link | Quote
I think I have a idea with that. In your link it goes band=... I bet people searching for band have that popup as a search and create a lot of bots. Maybe change that to something else. It might help.
ErkDog

Fuzz Ball
Level: 47

Posts: 896/982
EXP: 752190
For next: 14013

Since: 03-15-04
From: Richmond, VA

Since last post: 40 days
Last activity: 19 days
Posted on 11-10-04 07:50 AM Link | Quote
here's the obvious soloution.....

get a webhosting provider that it's not an issue...

like

http://fiftypounds.com

45 gigs of bandwidth... who cares about 50 megs ??

but yes as far as the technical version....

robots.txt

or IP ban them... cause there are onlya bout 3 subnets that actively scan...

google only polls websites like once in forever

MSN and Inktomi (Yahoo) are CONSTANTLY in my guests table.....
Gywall

Silver axe
Level: 30

Posts: 152/356
EXP: 164069
For next: 1800

Since: 03-15-04
From: In front of my moniter, where you're not!

Since last post: 104 days
Last activity: 14 hours
Posted on 11-10-04 07:04 PM Link | Quote
Originally posted by ErkDog
here's the obvious soloution.....

get a webhosting provider that it's not an issue...

like

http://fiftypounds.com

45 gigs of bandwidth... who cares about 50 megs ??


I would IF i had any way to pay (hell, I'm forced to save my cash up!)
Davideo7

Koopa
Level: 18

Posts: 79/112
EXP: 28817
For next: 1080

Since: 03-21-04

Since last post: 52 days
Last activity: 17 days
Posted on 11-10-04 07:13 PM Link | Quote
Why is it a bad thing that the bots are on your site? They don't drain that much bandwidth do they?
And don't they help your site get listed in search engines?
Gywall

Silver axe
Level: 30

Posts: 153/356
EXP: 164069
For next: 1800

Since: 03-15-04
From: In front of my moniter, where you're not!

Since last post: 104 days
Last activity: 14 hours
Posted on 11-10-04 07:17 PM Link | Quote
Originally posted by Davideo7
Why is it a bad thing that the bots are on your site? They don't drain that much bandwidth do they?
And don't they help your site get listed in search engines?


If they eat 50 MB of 750 MB limit (I didn't get warned on the decrease) in two days..
uhh, problem!
Davideo7

Koopa
Level: 18

Posts: 80/112
EXP: 28817
For next: 1080

Since: 03-21-04

Since last post: 52 days
Last activity: 17 days
Posted on 11-10-04 07:30 PM Link | Quote
They can take up that much? Wow, that's crazy
ErkDog

Fuzz Ball
Level: 47

Posts: 911/982
EXP: 752190
For next: 14013

Since: 03-15-04
From: Richmond, VA

Since last post: 40 days
Last activity: 19 days
Posted on 11-10-04 09:58 PM Link | Quote
Microsoft used 2 out of the 3.5 Gigs of bandwidth my forums used in October
MathOnNapkins

Math n' Hacks
Level: 67

Posts: 914/2189
EXP: 2495887
For next: 96985

Since: 03-18-04
From: Base Tourian

Since last post: 1 hour
Last activity: 32 min.
Posted on 11-11-04 12:58 PM Link | Quote
Originally posted by ErkDog
Microsoft used 2 out of the 3.5 Gigs of bandwidth my forums used in October


Do you consider that a good or a bad thing? O_o
ErkDog

Fuzz Ball
Level: 47

Posts: 917/982
EXP: 752190
For next: 14013

Since: 03-15-04
From: Richmond, VA

Since last post: 40 days
Last activity: 19 days
Posted on 11-11-04 05:07 PM Link | Quote
ohh definitely bad, I actually sent them an e-mail.... said....

2.5 gigs out of the 3.5 my website used in October was sent to IPs owned by your MSNBot.... something is wrong with that.....
Sokarhacd

Ball and Chain Trooper
Resistance is Futile
You Will Be Assimilated
Hab SoSlI' Quch
Level: 61

Posts: 621/1757
EXP: 1799888
For next: 76708

Since: 03-15-04

Since last post: 6 days
Last activity: 4 hours
Posted on 11-16-04 10:15 PM Link | Quote
ouch....and to think...they are supposed to help...not if you dont have alot of bandwidth, and the searchbots are sucking all down.
Modereb

Paragoomba
Level: 15

Posts: 75/75
EXP: 14749
For next: 1635

Since: 06-04-04

Since last post: 350 days
Last activity: 339 days
Posted on 11-17-04 03:24 PM Link | Quote
Originally posted by ErkDog
here's the obvious soloution.....

get a webhosting provider that it's not an issue...

like

http://fiftypounds.com

45 gigs of bandwidth... who cares about 50 megs ??

but yes as far as the technical version....

robots.txt

or IP ban them... cause there are onlya bout 3 subnets that actively scan...

google only polls websites like once in forever

MSN and Inktomi (Yahoo) are CONSTANTLY in my guests table.....
The server or webhosting provider has NOTHING to do with the behavior of search bots in a webpage.

Recommended is that you tell search robots to cache your index page, but don't go any further. That way your site will show up on the robot's search engine eventually, but it won't go through all pages, such as thread pages.

Between your <head> tags, put the following:
<meta name="robots" content="index,nofollow" />
Xkeeper
The required libraries have not been defined.
Level: NAN

Posts: -3068/-863
EXP: NAN
For next: 0

Since: 03-15-04

Since last post: 2 hours
Last activity: -753366 sec.
Posted on 11-19-04 11:04 PM Link | Quote
If you had a root folder (www.dumbpage.org/) you could also use a robots.txt file (Google it)

(kinda funny how long it took to get a decent response out of this thread)
CRZ
Newcomer
Level: 6

Posts: 3/9
EXP: 658
For next: 249

Since: 03-17-04
From: Twin Cities

Since last post: 127 days
Last activity: 6 days
Posted on 11-19-04 11:15 PM Link | Quote
Originally posted by Xkeeper
If you had a root folder (www.dumbpage.org/) you could also use a robots.txt file (Google it)

(kinda funny how long it took to get a decent response out of this thread)
I think that was covered in the second post in this thread, actually.
Add to favorites | "RSS" Feed | Next newer thread | Next older thread
Acmlm's Board - I2 Archive - Acmlmboard support? - Search Bots - any way to block them? | |


ABII


AcmlmBoard vl.ol (11-01-05)
© 2000-2005 Acmlm, Emuz, et al



Page rendered in 0.009 seconds.