Points of Required Attention™
Please chime in on a proposed restructuring of the ROM hacking sections.
Views: 88,498,865
Main | FAQ | Uploader | IRC chat | Radio | Memberlist | Active users | Latest posts | Calendar | Stats | Online users | Search 04-29-24 02:31 AM
Guest: Register | Login

0 users currently in Help/Suggestions/Bug Reports | 1 guest

Main - Help/Suggestions/Bug Reports - Microsoft URL Control New thread | New reply


Drag
Posted on 06-26-07 10:14 PM Link | Quote | ID: 49868


Spike
Dragon
Level: 57

Posts: 598/705
EXP: 1483286
Next: 2642

Since: 02-19-07
From:

Last post: 3930 days
Last view: 3930 days
I collected some interesting information about it here


Microsoft URL Control is not connected in any way to Microsoft or its search engine spiders, it is actually a "control" that allows programs running on Windows to access the web.

Microsoft URL Control is used by many different types of programs, some are nice and some are not. In many cases, it's an email address sniffing bot, so you would want to probably ban the IP address it's coming from, not necessarily the whole thing. I don't necessarily think banning it in the robots.txt file would work.

The Microsoft search engine robot is called MSNbot.


This site has some ways to trap bad robots. It's kinda funny to read.

That Microsoft URL Control thing is weird, there doesn't seem to be any official information on it.

____________________

Xkeeper
Posted on 06-26-07 10:16 PM Link | Quote | ID: 49869


Level: 105

Posts: 2344/2846
EXP: 12030781
Next: 231479

Since: 02-19-07

Last post: 6062 days
Last view: 2800 days
It's the User-Agent that Ninetales' Po100T Thread Checker leaves behind.

TinaSitter would probably do something similar too, if she left behind data in the board database.

____________________
I dealt with it.

Drag
Posted on 06-26-07 10:19 PM Link | Quote | ID: 49871


Spike
Dragon
Level: 57

Posts: 599/705
EXP: 1483286
Next: 2642

Since: 02-19-07
From:

Last post: 3930 days
Last view: 3930 days
That makes sense, actually, because a lot of sites that mention the user-agent, they hint that it's associated with VB in some way.

Still though, internet web crawlers are kinda scary. I mean, some make money off your site without giving anything back.

____________________

Xkeeper
Posted on 06-26-07 10:20 PM Link | Quote | ID: 49873


Level: 105

Posts: 2346/2846
EXP: 12030781
Next: 231479

Since: 02-19-07

Last post: 6062 days
Last view: 2800 days
Yes. But in the inverse, sometimes they can help you make money (by directing users there when they have a search your site hits high for).

Although, yes, having money being made off of you (searches coupled with ads) feels wrong at first, it's more of a symbiotic relationship, in that they give us hits and we give them an index they can stick ads next to.

____________________
I dealt with it.

Kawa
Posted on 07-01-07 08:26 PM Link | Quote | ID: 51259


CHIKKN NI A BAAZZKIT!!!
80's Cheerilee is best pony
Level: 138

Posts: 379/5344
EXP: 30951404
Next: 711577

Since: 02-20-07
From: The Netherlands

Last post: 4501 days
Last view: 2636 days
TinaSitter doesn't use Microsoft URL Control. It uses raw winsocks. UserAgent is "Tina".

____________________
Wife make lunch - Shampoo
Opera - give it a spin
Spare some of your free time?
<GreyMaria> I walked around the Lake so many goddamn times that my sex drive was brutally murdered
Kawa rocks — byuu

Xkeeper
Posted on 07-02-07 01:09 AM (rev. 2 of 07-02-07 01:09 AM) Link | Quote | ID: 51369


Level: 105

Posts: 2422/2846
EXP: 12030781
Next: 231479

Since: 02-19-07

Last post: 6062 days
Last view: 2800 days
Oh.

Well, the more you know...

____________________
I dealt with it.

Metal_Man88
Posted on 07-13-07 12:09 AM Link | Quote | ID: 54342


Level: 48

Posts: 389/458
EXP: 776661
Next: 46882

Since: 02-19-07
From: The Void

Last post: 3033 days
Last view: 3033 days
I just lay out a robots.txt file. I want that thar Alexa thing to archive my site a lot, but Google should stay away from my dynamically generated content.

OoBurns
Posted on 07-13-07 12:23 AM (rev. 2 of 07-13-07 12:24 AM) Link | Quote | ID: 54352


Boo
Level: 68

Posts: 827/1036
EXP: 2642126
Next: 86674

Since: 02-19-07
From: It is a mystery.

Last post: 5725 days
Last view: 5725 days
........Wrong thread.


I don't have a robots.txt file. Maybe I should make one.

____________________

Main - Help/Suggestions/Bug Reports - Microsoft URL Control New thread | New reply

Acmlmboard 2.1+4δ (2023-01-15)
© 2005-2023 Acmlm, blackhole89, Xkeeper et al.

Page rendered in 0.022 seconds. (321KB of memory used)
MySQL - queries: 42, rows: 58/59, time: 0.015 seconds.