How to do it:

1) go to server, open command line, type systeminfo (processor info will be in first ten lines). Or if you don’t like it cool, open Control Panel, CPUs are described on second pane )

2) check whether your CPU’s are in this list

3) ask you server admin, or manually reboot server, look into BIOS and turn HyperThreading off

4) Test your TM1 processes, they may speed up considerably

What’s behind the scenes.

Hyper-threading, in a nutshell, is a technique showing each physical CPU core as 2 logical ones. Since during normal work a lot of time is spent in thread switching and related registry load/unload and yada yada, physical cores actually are underloaded in common multithreaded system. Adding a duplicate registry set and a ‘virtual’ core allows to utilise physical core up to 20-40% more, earning throughput benefits.

Key words here are “threading” and “throughput”, meaning that HT benefit multithreaded applications where lots of small request are processed at the same time. OLTP databases are a prominent example.

But most of the systems I work with, almost all OLAP engines and even DWH-tuned DBMSes, actually suffer in this scenario.

For example, TM1 calculations are executed in a single thread, so if this thread is assigned half an CPU core, speed drops significantly. The same logic goes for Essbase metadata update process, for example, or for Cognos Enterprise Planning Job processing.

I’ve seen significant performance degradation (30%) in Cognos EP job processing due to HT context switching and have advised turning HT off on all Cognos EP Job servers.

Encountering the same issue with TM1 recently left me puzzled, since there was plenty of empty CPUs on server and there shouldn’t be any thread switching. But we’ve got 2+ time speedup after flickering the switch anyhow )

I promised a bonus and here it goes:

When buying a TM1 server with Intel CPUs, buy way more cores than you’ll license. Physical cores, not logical.

This will turn on Intel Turbo-Boost and it might give you another 30-40% speed boost. I haven’t tried that yet and there’s a bit of scepticism of current boost-detection approach in Wikipedia, but it’s definitely worth testing in a lab before you buy hardware. Cores are cheap compared to licence costs, so you may save yourself a ton of money by having a faster system. As in wikipedia example, for Core i7-2920XM you can boost up to 3,5 Ghz per core with 2,5 Ghz base speed. That’s a hefty speed up for a good old one-threaded TM1.

Overall approach excites coder part of me: modular development, agile, best practice source code — all this is actually very helpful. Having so much Turbo Integrator ‘how-to’s solves a lot of ‘re-inventing the bicycle’ cases.

As far as I understand bulk of this project’s code is developed by CubeWise. Hats of to them for being so open and willing to push all TM1 community further.

It’s interesting to see whether this site will gain momentum — whether there’ll be non-CubeWise commiters, how will adoption go and etc.

I would be proud to contribute some code there someday, if I’ll be doing TM1 related stuff. Maybe they will accept my small utilities for start….

And there’s an absolute must-read article about new TM1 9.5.2 Parallel Interaction feature.
I’m actually asonished to see an article of such depth in support knowledge base. It contains detailed explanation:
- of how things worked before, common problems and workarounds
- what has changed in server engine
- how will this change affect current systems
It’s actually terrific, I’d kill to get more of such articles on tools I work with, not just ‘try reinstalling ones’.

On an unrelated note: I’ve uploaded updated versions of tm1 stress testing tool (with more input parameters) and tm1 model comparison tool (just some bug fixing).

And on another unrelated note: I was nominated an IBM Information Champion again. Thank you all for reading this blog )

I’ve made random changes to Planning Sample — here’s the resulting change report

No need to install anything, I packed it into .exe file:
1) Download application archive, unzip it to any folder
2) run

tm1cmp.exe -o Path_to_Old_Model -c Path_to_Changed_Model

You can also specify -r report_file_name for name of generated report
3) Open report.html or the filename you specified and enjoy

How it works:

• basic file comparison
• for changed files I use difflib to detect changes + very basic logic to ignore simple edit_datestamp changes

Watch TM1Forums threads for more comments as people start trying it out.
9.5.2 – New Features and Issues
Migrating to TM1 v9.5.1 or TM1 v9.5.2?

Several points before we start:

• You can load a rule file in TI process by using RuleLoadFromFile function
• It’s pretty easy to generate need rule file using ASCIIOutPut to write the need rule.
• But ASCIIOutPut opens the file on procedure (EPILOG, PROLOGUE, META) start and closes connection only when procedure ends.

I was thinking about using one TI process for generating a number of rule files (simple [Dim.Element1]=[Dim.Element2] format, where Dimension and elements are parameters) and attaching them to a list of cubes, so the closing file restriction was a problem. I just couldn’t generate the files and attach them in the same procedure.

An easy workaround is to use a very simple bat file, that will echo your generated rules into txt files that you will load in cycle.

TI Prolog Code then looks like this

# Iterating through your given list of cubes 
# see Planning Sample language change process for example
# For given cube -- currentCube holds it's name
# Generating rules file
SQ =  '''';
lineToWrite = '"[' | SQ | Dimension | SQ | ':' | SQ | elementFrom | SQ | '] = [' | SQ | Dimension | SQ | ':' | SQ | elementTo | SQ |  '] ;"';
ExecuteCommand('echo.bat ' | lineToWrite,1);
# backing up current rules file
commandToExecute = 'copy.bat ' | currentCube | '.RUX '  | currentCube | '.RUX_BACKUP';
ExecuteCommand(commandToExecute,1);
 
# attaching generated rule, saving data
RuleLoadFromFile(currentCube,'test.txt');

and a simple echo.bat

echo %~1 > test.txt

and copy.bat

copy %1 %2 /y

• I vizualize model data flow (way bigger than PlanSample in this picture, but node filtering helps).



• Looking at such graph it’s easy to see model inputs (to left side), hardest calculations (these are the gordian knots) and reporting cubes.
• So I just start pouring data input and reads in suspiciously looking model parts. And if they suffocate, it’s up to optimizing rules or turning rules into TI processes.
• Redraw the picture and repeat until happy )

Very much like unjamming the creek. Spring )

Never trust sales slogans like above. Any kind of stress testing system should be approached with a huge amount of doubt, as well as this one. But it’s a simple and useful tool for testing your calculations for scalability (IMHO).

We needed to stress-test one of our TM1 models and looked around for solutions. Most of them are discussed in a very interesting thread on olapforums, but going via Excel + VB way seemed a bit too much of ‘Excel + F9′ to emulate the load. Turned out that it’s rather easy to roll-out a simple customizable Java application for this task, so here it is.

What the tool does

This tool tests your model for scalability and potential locking problems while performing calculations.
It doesn’t test you client connectivity (Excel, Contributor, Websheets) because it works via Java API. It doesn’t test your environment (network latency etc) as well.
It allows you to find and optimize the long-running and potentially blocking calculations in the model.
It allows you to simulate different types of users (like “What will happen when brand managers are inputting sales plan and analysts are working with investment portfolio and CashFlow at the same time?”).

Attention: Tool writes data to your model, don’t use it in production.

How does it work

You fill a user profile, containing:

• logins&passwords list to be used
• cubes for reading
• cubes for writing
• a list of dimension and their elements for writing — this is used for various purposes like simulating users from different regions (by specifying region in the list). Data will be written only to this elements in all the cubes containing dimension.
• number of user session to be started with this profile

When you run the tool with filled in profile (you can run different profiles at the same time), it creates a required number of user sessions and in each session:

• flips a coin to decide whether to read or write
• (if read) creates a random view in one of the cubes for writing and reading and reads it contents
• (if write) writes a random number (less than 100) of randomly picked cells into one of cubes for writing
• waits 3 seconds and goes on again. Maybe this wait interval should be random as well )

After starting the tool you fire up TM1Top and enjoy the results.

You’re interested in locking problems and long running calculations that are nicely depicted there (an example is underlined in the picture above).
You can run TM1perfmon to see the hardware side of the problem at the same time

After you’ve got enough, just hit CTRL + C in command window and it will stop.
As a side bonus you’ll get a test log in the file test_results_profile_name.csv which will contain something like this:

How to install&use it

1. You will need a only Java machine (jre), which you most likely already have, because of number of web-plugins developed in Java. This the only requirement, there’s no need to have TM1 installed or anything.
2. Download the tool, unpack to some directory (c:\tm1stress, for example)
3. Cryptic step of adding security certificate to your Java.
1. Find jre\bin\keytool.exe utility on your computer
2. Open the command line and navigate to jre\bin
3. Run command line keytool -keystore ..\lib\security\cacerts -alias Company -import -file c:\tm1stress\applixca.der
4. Enter ‘changeit’ as password
5. Type yes
4. Create a new user profile xml file, use the planning sample one as an example (if don’t need some sections — just leave them blank)
5. Open command line, navigate to c:\tm1stress, run java tm1_stress_test your_profile_filename.xml
6. You should get some informational messages (logins, operations, errors) in command output

There’s application source code in the download archive, it may serve as Java API usage sample.