Applied dimensionality » Microsoft http://ykud.com/blog Mon, 30 Apr 2012 06:10:53 +0000 en hourly 1 http://wordpress.org/?v=3.3.2 Microsoft SQL Server reporting database configuration http://ykud.com/blog/bicpm/microsoft/microsoft-sql-server-reporting-database-configuration-practices http://ykud.com/blog/bicpm/microsoft/microsoft-sql-server-reporting-database-configuration-practices#comments Thu, 23 Sep 2010 12:18:07 +0000 ykud http://ykud.com/blog/?p=429 A check-list for using Microsoft SQL Server as a reporting database. This is a ‘be aware’ list for myself in first place, since I forget some details as time passes.

Table of contents

Overall database settings

Use simple recovery mode

Set up simple recovery mode for database to minimize performance issues with transaction log records. Usually you can use daily backups with DWH, since data is loaded daily. In more complex situations you can use ETL-based ‘checkpoints’, but in general, daily backups are more than enough.

Separate log and data io channels

Use decent I\O systems and separate data and log I\O channels. This is obvious, just reiterating.
RAID 1+0 seems to be preferable choice of disk configuration for DWH loads. See this entry, this one or test it yourself )

2008 r2

Use SQL Server 2008r2 Enterprise if you have a choice. 2008 adds table\index compression (which is great) and get’s partitioning to work way better in queries. And I’ve  20% performance speed-up against 2005 on the same query set (with same execution plans and that surprised me).

Auto shrink Off

Set Auto shrink off, it kicks unpredictable I\O strain. Shrink database daily, just after backup in the same maintenance task.

Table configuration

identity columns as primary keys

Use identity columns as table primary keys. And using this column as clustered index will help I\O in most cases.

Set up foreign keys

Again, obvious, but I’ve seen this violated more than once. Use foreign keys for fact-dimension table reference. This will help optimizer a lot.
You can disable these keys to speed up inserts, but in 90% cases it’s not needed.

Table partitioning

Always use partitioned tables (even fake, with 1 partition) to:
- achieve partition elimination during query execution. Especially in Ms SQL 2008
- use partition switching for data load (check this article as well)

Table compression in 2008

SQL Server 2008 introduces table compression which helps to reduce I\O (but increases CPU load), which is very good for reporting tasks. Use it for fact tables.

ETL

partition switching as default load strategy

More benefits of partition switching for loading:
- while you’re loading data in ‘new_data’ partition, base table is accessible and consistent
- after you’ve prepared ‘new_data’ partition you can check it’s data for quality (planning to write about later)
You can implement switching by creating another temporary partition, which I like to call ‘last load’. So you switch:
- current -> last_load
- new_data -> current
in one transaction
After that you have ‘last_load’ to:
- switch back in emergency
- check data from current\previous load for some more quality checks

Filtered indexes for calculations

Another 2008-only feature is called filtered indexes. These indexes can be essentially smaller than common ones if you know for sure what region of data you’ll be accessing. And this turns out to be very efficient for SQL code you’re using for different calculations. Since I mostly work with Oracle Data Integrator as an ETL tool and it’s an SQL-generating machine: I get up to10 times calculation speed up by building small, calculation-specific filtered indexes.

Tablock hint

All inserts should be done with TABLOCK hint, which speeds up insert by acquiring a full table lock during the whole transaction. Since in DWH main inserts are performed by ETL there should be no conflicts. TABLOCK also speeds up logging during bcp.

Query tuning

Indexes

Really a vast topic, but to sum it up ) :
- columns in indexes should be ordered by selectivity
Be very-very careful, ‘your mileage may vary’:
- creating a huge number of large indexes may actually slow system down due to I\O overload
- you should try to create indexes that can help a number of reports\query instead of tuning a single request
See Books session below, skip through at least one of them before starting it out wild.

Filtered indexes

As with ETL calculations — filtered indexes may be used for tuning specific queries.

Index compression

Indexes can be compressed in the same way as tables (which is obvious given that a table is a clustered index) and this speeds queries by reducing IO

Books to read

I’d recommend following
And if you have time and will
]]> http://ykud.com/blog/bicpm/microsoft/microsoft-sql-server-reporting-database-configuration-practices/feed 1 Mcirosoft PerformancePoint is out http://ykud.com/blog/bicpm/mcirosoft-performancepoint-is-out http://ykud.com/blog/bicpm/mcirosoft-performancepoint-is-out#comments Fri, 21 Sep 2007 06:35:52 +0000 ykud http://ykud.com/blog/?p=180 Official release

Launch site.

Lets fasten our belts, BI market will never be the same )

]]> http://ykud.com/blog/bicpm/mcirosoft-performancepoint-is-out/feed 0 Signed &downloaded PerformancePoint CTP http://ykud.com/blog/bicpm/signed-downloaded-performancepoint-ctp http://ykud.com/blog/bicpm/signed-downloaded-performancepoint-ctp#comments Sun, 26 Nov 2006 16:18:44 +0000 ykud http://ykud.com/blog/2006/11/26/signed-downloaded-performancepoint-ctp/ Going to install/toy with it in free time.
And just before I’ll be more informed, the current view is:
I think the strong points of Ms BI(CPM\EPM, call it whether you like) are:
1 None (or little) integration problems. All niche-BI vendors spend a lot of effort integrating acquired  products (for ex, Cognos with Adaytum&Frango, BO with SRC). And it’s still a long way to go. And see Ms AX and Nav as an example of hard integration road.
2 Enterprise scale db & olap engine.
3 Special workflow engine
4 User-friendly interface (excel)
5 Strong brand\cheap prices
6 The possibility to develop add-ons and increase functionality by programming.
And the weak points:
1 The “integration” of SQL server, AS, BizTalk, Proclarity. It’s easier, due to all .Net stuff, but on initial phase it just won’t work (remember EPB).
2 No experience in CPM area, nor in development, nor in selling such solutions. 
3 The lack of concentration. Ms is so huge and wide-aimed, it just lost it’s focus. See Joel’s excellent post on this theme. And the update.
4 The possibility to develop add-ons and increase functionality by programming.

It’s interesting to see how these statements will change after some product experience.

]]> http://ykud.com/blog/bicpm/signed-downloaded-performancepoint-ctp/feed 0 Firefox http://ykud.com/blog/bicpm/firefox http://ykud.com/blog/bicpm/firefox#comments Mon, 02 Oct 2006 19:07:07 +0000 ykud http://ykud.com/blog/2006/10/02/firefox/ I like FireFox a lot, and was allways disappointed when changing to IE to view Contributor applications.

Recent versions of IETab addon allow to open tabs, using IE as renderer, and, moreover, start contrib apps in FireFox tabs.

So I don’t have to switch browser anymore. Hoooray?

]]> http://ykud.com/blog/bicpm/firefox/feed 0 Windows Utils http://ykud.com/blog/bicpm/windows-utils http://ykud.com/blog/bicpm/windows-utils#comments Sun, 10 Sep 2006 08:41:58 +0000 ykud http://ykud.com/blog/2006/09/10/windows-utils/ While ASP permission-fighting, discovered a couple of usefull utils from SysInternals.

Filemon –shows File System activity

RegMon — shows registry activity

This is win vs lin to me. In linux it’s all built-in, already there, you can ask kernel, monitor everything, w/o installing anything.

]]> http://ykud.com/blog/bicpm/windows-utils/feed 0