"Order Entry" Stress Test: Measuring Enterprise Class Performance

One complaint that we've historically received regarding our Forums database test was that it isn't strenuous enough for some of the Enterprise customers to make a good decision based on the results.

In our infinite desire to please everyone, we worked very closely with a company that could provide us with a truly Enterprise Class SQL stress application. We cannot reveal the identity of the Corporation that provided us with the application because of non-disclosure agreements in place. As a result, we will not go into specifics of the application, but rather provide an overview of its database interaction so that you can grasp the profile of this application, and understand the results of the tests better (and how they relate to your database environment).

We will use an Order Entry system as an analogy for how this test interacts with the database. All interaction with the database is via stored procedures. The main stored procedures used during the test are:

sp_AddOrder - inserts an Order
sp_AddLineItem - inserts a Line Item for an Order
sp_UpdateOrderShippingStatus - updates a status to "Shipped"
sp_AssignOrderToLoadingDock - inserts a record to indicate from which Loading Dock the Order should be shipped
sp_AddLoadingDock - inserts a new record to define an available Loading Dock
sp_GetOrderAndLineItems - selects all information related to an Order and its Line Items

The above is only intended as an overview of the stored procedure functionality; obviously, the stored procedures perform other validation, and audit operations.

Each Order had a random number of Line Items, ranging from one to three. Also randomized was the Line Items chosen for an order, from a pool of approximately 1500 line items.

Each test was run for 10 minutes and was repeated three times. The average between the three tests was used. The number of Reads to Writes was maintained at 10 reads for every write. We debated for a long while about which ratio of reads to writes would best serve the benchmark, and we decided that there was no correct answer. So, we went with 10.

The application was developed using C#, and all database connectivity was accomplished using ADO.NET and 20 threads - 10 for reading and 10 for inserting.

So, to ensure that IO was not the bottleneck, each test was started with an empty database and expanded to ensure that auto-grow activity did not occur during the test. Additionally, a gigabit switch was used between the client and the server. During the execution of the tests, there were no applications running on the server or monitoring software. Task Manager, Profiler, and Performance Monitor were used when establishing the baseline for the test, but never during execution of the tests.

At the beginning of each platform, both the server and client workstation were rebooted to ensure a clean and consistent environment. The database was always copied to the 8-disk RAID 0 array with no other files present to ensure that file placement and fragmentation was consistent between runs. In between each of the three tests, the database was deleted, and the empty one was copied again to the clean array. SQL Server was not restarted.

SQL Stress Results Order Entry Results
Comments Locked

144 Comments

View All Comments

  • Nighteye2 - Thursday, April 21, 2005 - link

    Small observation: in the article DDR2 is mention as a solution to AMD memory bandwidth, while in the news posts there's a bit about AMD skipping DDR2...

    Anyway, that being said, it's looking good...I wonder how long until there'll be sub-300 A64 X2's available? ^_^

    Also, about the hardware problems...isn't running a dual-socketed MB with a single CPU asking for trouble?
  • Spacecomber - Thursday, April 21, 2005 - link

    I really enjoyed taking a look at what you could bring us about these upcoming Athlon dual core processors. It looks like dual core will be the future for all of us, at least at some point.

    Just a quick comment on the price comparisons that you provided between the dual core opterons and their single core predecessors, I found it interesting to compare prices on the basis of the number of cores.

    So,

    Opteron 248: 2x$455=$910
    Opteron 174: $999

    Opteron 848: 4x$873=$3492
    Opteron 275: 2x1299=$2598

    Assuming the performance scales simply based on the number of cores involved, the pricing of the new dual core opterons looks more attractive.

    Space
  • Quanticles - Thursday, April 21, 2005 - link

    pwnt
  • fishbits - Thursday, April 21, 2005 - link

    It's odd that some picture game developers immediately supporting the PhysX chip as soon as it's available, but think they'll drag their feet to take advantage of another whole CPU core at their disposal.

    Maybe that will be the reality though, as MT programming is supposed to be a lot harder. Still, to be able to get a game out the door that blows away any of the competition, it might happen sooner than we think. And I could see how Intel would want to push this along to help their sales, and might contribute resources towards making it happen. "OMG, that new game is great, but it totally rules on a new dual core rig! Saw it at my friend's house the other day!"

    Who knows, maybe games'll gobble up that second core so fast, it won't be long before we complain about how sluggish the system is when multi-tasking, and that we're shutting down background processes, anti-virus, etc all over again. "We need quad core!" :P
  • Aenslead - Thursday, April 21, 2005 - link

    Ah, well... most of you are forgetting something: sure, the chip's cost is almost 50% higher than the cheapest Intel offer, however, to use a Pentium D, you require a new motherboard (i955x @ 180USD, probably... nF4 IE @ 200USD), AND DDR2 memory... plus, if you have an AGP card, the PCIe video card as well. That's about 650USD for the whole Intel upgrade. AMD, on the other side, is just the processor, which ends up being FAR cheaper.

    And that is all I have to say about that.
  • Jep4444 - Thursday, April 21, 2005 - link

    the X2 4400+ does not parralel to the 3500+, it parallels to the upcoming socket 939 3700+(which a 939 4000+ could be underclocked to compare)

    the 3500+ would be best compared to a X2 4200+

    the X2 4600+ is dual 3800+ and the 4800+ are dual 4000+
  • xtknight - Thursday, April 21, 2005 - link

    It's too bad that gamers or people that don't multitask are basically left in the dark (extra-performance-wise) by dual-core. I'm not going to break the bank for something that's going to give me less performance than I already have.

    There's multitasking and then there's multitasking. One kind is having a main program up that gets most of the CPU's attention and another BitTorrent or whatever that's taking up <5% CPU. Usually I'm not trying to encode a video while I play a game, which would be the other type of multitasking. Only the second kind would greatly benefit from these new CPUs, which is a shame. In the first multitasking type I talked about, dual core will improve responsiveness but not raw processing performance.

    Does this mark the end for single-threaded performance? Programmers will have a hell of a time creating dual core-beneficial applications, unless by nature the program would benefit from it (i.e. a game server browser, or an AI-heavy game). If the PhysX chip comes through, dual-core won't help too much with physics either. The only benefit that would ever see the light of day for me is the fact that the rest of my system isn't lagged while something else is taking up 100% CPU. For example I could still move my mouse and use Windows Explorer while I'm compressing some files with WinRAR. Even then these scanarios don't come up too often for me personally.

    When it comes to raw number crunching performance, the dual-core CPUs don't show any improvement over single-core ones. Sadly enough I think it's going to take forever for programmers to multi-thread their applications. That being said, any program I make from now on will be multithreaded as much as possible.
  • AtaStrumf - Thursday, April 21, 2005 - link

    Nice review as usual :-)

    ...but I do have one complaint. I would like to have seen a top of the line Intel single core CPU (there were no single core P4s in your tests) compared to "X2 4400+" and a 3500+ 939 Athlon 64 which runs at the same 2,2 GHz as the "X2 4400+" to see a direct effect of the second core instead of the 2,4 GHz 3800+.

    Some multitasking tests were a bit weird, dare I say unrealistic, but OK for starters. The way I multitask is usually a bunch of IE windows (12 ATM), one folding at home client, 3 - 6 bittornado clients, 1 or 2 (sometimes more) word documents, 1 or 2 (sometimes more) excel documents, outlook express, possibly Photoshop CS, a bunch of Windows Explorer windows, few notepads, some winzip/winrars every now and then, windows media player playing MP3s, Kaspersky antivirus, a dictionary, ACDsee from time to time, Opera with a few open tabs if IE isn't right for the job, ... and I rarely play any games anymore. OK I think that's it. This is not at all uncommon for me, so I'm really looking forward to dual cores, I'm just very sorry that AMD can't offer anything at a competitive price, so instead of going for a Socket 939 from a socket 754, I might go for an Intel platform. I don't know jet, a lot depends on how hard those PD are too cool. No word on that yet from you. I wonder why?
  • morcegovermelho - Thursday, April 21, 2005 - link

    At AMD site:

    http://www.amd.com/us-en/Processors/ProductInforma...

    Hypertransport Links:

    152 - (3 links - 0 coherent links)
    252 - (3 links - 1 coherent link)
    852 - (3 links - 3 coherent links)

    that means they are different. Time to change page 3? ;)
  • SLIM - Thursday, April 21, 2005 - link

    Nice review AT. One small suggestion would be to compare one x75 to two 248s to see the effects of memory bandwidth as well as having two cores communicating directly with each other.

    BTW, that's pretty damn funny that you have to install new circuit breakers and outlets just to power up the zeons.

Log in

Don't have an account? Sign up now