Re: [mv] Optimizing a catalog

["Bill" <bill@webteam.net>]

Thanks for the pointers Mike, I've just implemented a couple of your ideas
and I've already seen an incredible increase in performance.  I see many
other improvements that can be made as well!

Bill


> > > Hi,
> > >
> > > I have a single catalog that is experiencing about 20-40,000 page
views a
> > > day.  I'm running MV3.12 on a Digital Alpha with 512 MB of RAM.  I use
mySQL
> > > and file based sessions.
> > >
> > > I was wondering if anyone out there has any recommendations for
optimizing
> > > my catalog, my server has a load averaging 4-5 at peak hours and many
> > > customers are receiving Error Messages because the Minivend server can
> > > handle the number of requests (I'm not blaming Minivend, I know I need
to do
> > > some work to make things run more efficiently).  I currently have
MaxServers
> > > set to 15, Hammerlock to 120, Housekeeping to 15, and PID check to
300.
> >
> >   A few suggestions...
> > 1) the obvious... another server with a transparent switchover.. I can't
think
> > what they call it...
> > 2) take your catalog out of minivend and only use the order process. If
> > you switched to C or mod_perl you'd get much better perfomance, and I'm
> > sure the catalog gets hit more than the order section.
> > 3) are you doing static pages at all? Make your process as static as
possible.
> > 4) I believe mv can cache pages... find some that don't change and cache
them.
> >
>
> This type of thing might be necessary for a huge load, but 20,000-40,000
> page views a day is not that much traffic. Even if 10% of the high range
> of that amount happens in the peak hour, that is only 4000 requests
> per hour or slightly more than one per second. Just proper page building
> should handle that just fine. We routinely serve 200,000 *Minivend-parsed*
> pages a day on a single Pentium system. It should handle a couple of
> hundred requests per minute without difficulty, providing some sensible
> approaches are taken.
>
> The usual problem is one of:
>
> - No index in the database table
> - Many embedded [perl] or [calc] or [if] tags in an iterating
>   list
>
> Unrolling your loops or placing your complex testing in the individual
> product page is the easiest way to avoid this stuff. MiniVend cannot
> repeal the laws of physics; if you perform 50 database accesses and
> conditionals for every item in a repeating list, then your performance
> will suffer greatly.
>
> We had a situation just last week on Minivend.com -- a user changed their
> catalog and started bringing the system to its knees with load averages
> of 5 or more. (They were doing brute-force line-by-line searches of a
> 30,000 item catalog right on the home page.) We changed a few searches,
> did some indexing, and unrolled one loop. The system now runs at a load
> average of less than one and serves 100,000 parsed pages per day (though
> it is not the web server as well).
>
> Perhaps I know more about this stuff and can optimize it better, but
> the technicques I use are the exact same ones I have repeatedly posted
> to this list:
>
> -- Usertags where appropriate to reduce the number of [if] things
> -- Placing search results inside a Perl tag and building the
>    result list with Perl
>
> >   I'm sure there are more ways... but probably moving your catalog out
of
> > MV is the easier one/cheaper one that will get results. Depends on how
> > much mv code you really use in the catalog. We use very little, and have
> > been toying with the idea ourselves. We already have a way of storing
the
> > number of items ordered in a cookie so we can use a complete order tag
in
> > a non minivend page. using C or fast cgi or some mod_perl piece would
> > probably bring your load down a lot and serve pages out faster.
>
> I am surprised you guys still participate here.... 8-)
>
> --
> Mike Heins                          http://www.minivend.com/  ___
>                                     Internet Robotics        |_ _|____
> It's a little-known fact            131 Willow Lane, Floor 2  | ||  _ \
> that the Y1K problem caused         Oxford, OH  45056         | || |_) |
> the Dark Ages. -- unknown           <mikeh@minivend.com>     |___|  _ <
>                                     513.523.7621 FAX 7501        |_| \_\
> -
> To unsubscribe from the list, DO NOT REPLY to this message.  Instead, send
> email with 'UNSUBSCRIBE minivend-users' in the body to
Majordomo@minivend.com.
> Archive of past messages: http://www.minivend.com/minivend/minivend-list