ok, so you’re using sqlserver2008 r2, but you can’t afford, or haven’t invested in the enterprise edition (which by the way, i think is an awesome product). so in other words, you can’t take advantage of compression, and you can’t take advantage of partitioning. if you are struggling for sqlserver performance, what do you do?
there are probably hundreds of articles out there that give you hints and suggestions, and they are good – no doubt, but to find them and get them all in place takes a bit of doing. so i’ve gone ahead and done a little homework for you but first: my configuration:
- sqlserver2008 r2 – enterprise edition (no partitioning, and no compression turned on) 64 bit
- windows 7 enterprise – 64 bit
- quad core i7-2630qm x 2 (at 2.00 ghz) = 8 cores
- 16gb ram
- 7500 rpm sata ii internal disk, no raid
after quite a bit of tweaking, i can get a bulk-insert (insert into … select from) to run between 45,000 and 60,000 rows per second. the row sizes vary but are around 500 bytes in width (without the headers, and all the other items).
so how do you do it?
- tune the temp area – which i cannot do because it is all existing in a single environment. to tune your temp area, follow these suggestions
- move the temp dbf and temp mdf files to raid zero+1 configuration disks, or move them to internal disk (if it has higher performance ratings than your san/nasd)
- set the “growth rate” of the log file and the data file to be larger than expected.
- pre-size / pre-extend the temp dbf and mdf files, to be large enough to accomodate huge sorts, group by’s, order by’s and large index joins
- setup multiple dbf and multiple mdf files on your disk device, spread them across different san disks for maximized performance, put them all on raid zero + 1
- defragment your standard database
- use the following commands in your staging area, and your datawarehouse database:
sp_dboption <database name>,’select into/bulkcopy’,true;
sp_dboption <database name>,’trunc. log on chkpt.’,true;
alter database <database name> set recovery bulk_logged;
- move your staging dbf and mdf files to a different disk array on a separate i/o channel away from your tempdb and away from your target warehouse db.
these are just some suggestions i have. there are more, but for now, this should help. at the end of the day, compression and partitioning in sqlserver are key to performance – as is raid 0+1 in a data warehousing environment. i hope that somehow, you are able (in the future) to upgrade to the enterprise edition (no, i am not being paid for this, and i receive no compensation from microsoft).
there are a number of great products out there that you might want to use to help you including:
quest: spotlight on sqlserver
idera monitoring tools
a book: “sql performance tuning” by peter gulutzan and trudy pelzer, the isbn is: 0-201-79169-2