[Date Prev][Date Next][Thread Prev][Thread Next][Minivend by date
][Minivend by thread
]
Re: Server load,bad robots and server config
****** message to minivend-users from Ryan Hertz <rhertz@gyb.baits.com> ******
At 08:11 AM 5/21/99 , mikeh@minivend.com wrote:
>****** message to minivend-users from mikeh@minivend.com ******
>
>Quoting Chris Daniel (krskrem@speedfactory.net):
>>
>> Hello all,
>> My minivend site is receving around 6000 to 7000 hits a day.
>> I had set the mv servers running at 15.
You might take a look at the minivend.cfg settings for 'housekeeping' and
'hammerlock' to see if MV is spending too much time _not_ serving pages.
>> When I checked the netstat command it sometimes is running all 15
>> servers at a time.
>> And in TOP, it is running several instances of minivend as well. zombie
>> and perl.
>> This demand slows the server down a little. The server is a single
>> processor P2/400 w/256 ram running redhat/apache and mv3.12
>> This just about uses all the memory and cpu. And server load at this
>> level is 9.0+, guess its time for more memory.
>> Other than placing more static pages in the store, is there a way to
>> increase performance and decrease server load?
>> Is there benchmark data on server setups and configs?
>> I limit the graphics a bit, ie: using same graphics on all pages,they
>> are optimised.
>> As the site is adult it will recieve more "just looking" customers than
>> mainstream sites.
>> So in order to be profitable I must show the store to lots of folks.
>>
>> And in error.log I see bad_robot warnings up to 600+ calls..
>> I imagine I can fix that,if they are spiders with <meta index no follow>
Sometimes, but not always (Or at least try using a robots.txt file in your
root dir)... and consider activating that lockout command.
>>
>> But I have limited the bad_robot stuff to 50. Does this not block them?
>> Does each element of a page count as a robot access? As in a chopped up
>> graphic or table?
No, it counts request to minivend - not images. Check out the ipfwadm
command (*nix) - or if the same 'bot is to blame, consider making a
'.htaccess' file (Apache) that will block that 'bot explicitly.
>>
>> Well I am interested to hear what kind of configs are out there,server
>> loads,traffic etc..
>> And ideas about squeezing as much as I can outta mv and my server.
>> If you can help ,thanks.
>> Chris Daniel
>
>Hi,
>
>My MiniVend server gets over 100,000 parsed page hits per day (about
>1.4 million overall requests) and I don't see anything like this. The
>gasp<
>server is equivalent in capacity, although I have a WWW frontend to
>a dedicated catalog server. Load average on both runs between 0.2 and
>0.9 depending on time of day. The speed is consistent, too; check out
>the MiniVend demos which run on that machine pair and you will see them
>running very fast even at peak hours.
>
>15 is too high for MaxServers unless you have at least 512MB
>of memory. I would back off to about 7, 9 at most.
>
>There is something else going on; you need to optimize your database
>performance somehow and I would double-check the FAQ items. If you haven't
>run expireall -r then your session database is probably getting big.
>
>But there are lots of other potential problems, one of which is "hotspots"
>on the session database when all your hits are in one catalog. This
>is particularly a problem with DB_File. There I recommend running with
>MINIVEND_NODBM and using the file-based sessions. With a little trouble,
>it is even possible to do that and still have access to GDBM or DB_File
>databases. Just put "require Vend::Table::GDBM;" somewhere at the top
>of the Data.pm module and run with MINIVEND_NODBM set in your environment.
>Then you can explicitly specify in catalog.cfg
>
>Database products products.asc TAB
>Database products GDBM 1
I've found a huge speed increase by using this in my catalog.cfg:
Database products products.asc TAB
Database products INDEX description
>
>and it should use GDBM. (This works for MV3.12 only.)
>
>As I said, it could be many things. And if you think MV is alone in
>that regard, take a look at the many articles in technical publications
>that deal with optimizing database performance. It isn't always easy. 8-)
>
>--
>Mike Heins http://www.minivend.com/ ___
> Internet Robotics |_ _|____
> 131 Willow Lane, Floor 2 | || _ \
>It's a little-known fact Oxford, OH 45056 | || |_) |
>that the Y1K problem caused <mikeh@minivend.com> |___| _ <
>the Dark Ages. -- unknown 513.523.7621 FAX 7501 |_| \_\
>-
>To unsubscribe from the list, DO NOT REPLY to this message. Instead, send
>email with 'UNSUBSCRIBE minivend-users' in the body to
Majordomo@minivend.com.
>Archive of past messages: http://www.minivend.com/minivend/minivend-list
>
Ryan Hertz tel 520-645-3812
Webmaster tel 800-645-BAIT
Advertising Director fax 520-645-2588
Gary Yamamoto Custom Baits, Inc. http://www.yamamoto.baits.com
-
To unsubscribe from the list, DO NOT REPLY to this message. Instead, send
email with 'UNSUBSCRIBE minivend-users' in the body to Majordomo@minivend.com.
Archive of past messages: http://www.minivend.com/minivend/minivend-list
