So question is – when we would see any of this?

Accordingly to Blair Wheadon,  Product Manager in the Volume Business Unit at SAP, our cries may be answered relatively soon.

Post is a nice summary and I hope features discussed would be in the final product.

Few highlights I pick for myself and with comments:

• Crystal Reports Basic will no longer be included in Visual Studio 2010
  Finally! Every time I have to set-up Visual Studio environment, I have to make sure I do not bring CR Basic along since I have a full version instead.
  Do not worry “Crystal Reports for Visual Studio 2010 will be provided by SAP as a free download, no registration required“.
  We are committed to have a beta version at the time that Visual Studio 2010 goes GA. A production release will be no later than Q3 2010
• There are slight changes in Licensing… as usual, read it through.
• 64-bit run-time support.
  Good! Platform independent code for .Net applications.
• New WPF Viewer
  Even better! No more .Net 2.0 assemblies (I hope)
• New XLSX export to take advantage of the big grid for data-only Excel exports
  Finally! 64K page limit is inconvenient, to say the least.

You can read about other more or less important improvements in Blair’s post, but these are answers for questions I was looking for.

Let’s look at the following scenario:

-- Create Table_1
CREATE TABLE dbo.Table_1 (Col_A int NULL, Col_B numeric(18, 0) NULL, Col_C nvarchar(50) NULL) ON [PRIMARY]
GO;
--
-- Add data
INSERT dbo.Table_1 (Col_A, Col_B, Col_C) VALUES (10, 10.5, 'ABC');
INSERT dbo.Table_1 (Col_A, Col_B, Col_C) VALUES (20, 0.5, 'DEFG');
--
-- Create View
CREATE view [dbo].[vw_MyView] AS
SELECT TBL.*, (TBL.Col_A * TBL.Col_B) AS ColMult FROM dbo.Table_1 TBL
GO;

Attempt to use the view would give the following result:

Col_A, Col_B, Col_C, ColMult
=================================
10, 10.5, ABC, 105
20, 0.5, DEFG, 10

So far we are OK. But “fun” starts when a new column is added to the table

Case #1 – bad:

ALTER TABLE dbo.Table_1 ADD Col_D nvarchar(10) NOT NULL DEFAULT 'A'

Result of the view:

Col_A, Col_B, Col_C, ColMult
=================================
10, 10.5, ABC, A
20, 0.5, DEFG, A

Well… OK, it is not as bad as it seems since we would find the problem almost immediately in application since types of expected and retured data are incompatible. Bad situation, but manageable.

Case #2 – catastrophic:

ALTER TABLE dbo.Table_1 ADD Col_D int NOT NULL DEFAULT 0

Let’s look at the result now:

Col_A, Col_B, Col_C, ColMult
=================================
10, 10.5, ABC, 0
20, 0.5, DEFG, 0

ColMult now contains 0 (zero)!!! If we have significant logic which relays on that, then big troubles are coming our way.
Now this is a catastrophic situation. And not easily visible. And off course work just fine on Developer’s machine…
Testing… Testing… Testing…

What just happened here?!!!

This is when knowing non-schema-bound-views definition is important. And let’s remember – this is default state of any new view created.

First, what a difference between schema bound views and not bound.
In order to create schema bound view special view attribute need to be specified. Let’s adjust original view definition as following:

-- Alter View
ALTER VIEW [dbo].[vw_MyView] WITH SCHEMABINDING
AS
SELECT TBL.*, (TBL.Col_A*TBL.Col_B) AS ColMult FROM Table_1 TBL

SCHEMABINDING
Binds the view to the schema of the underlying table or tables. When SCHEMABINDING is specified, the base table or tables cannot be modified in a way that would affect the view definition. The view definition itself must first be modified or dropped to remove dependencies on the table that is to be modified. When you use SCHEMABINDING, the select_statement must include the two-part names (schema.object) of tables, views, or user-defined functions that are referenced. All referenced objects must be in the same database.

If we try executing script above to add the column error would occur: “Syntax ‘*’ is not allowed in schema-bound objects.”

As you can see from definition of the attribute, view need to be defined explicitly with all columns listed. So if we were to have columns specified explicitly, then no problem would occur in the first place and is considered a good practice.

In our scenario above this is not an intent. Our view is designed to adjust itself to a table structure changes.
Inconvenience… or advantage… I let you decide.

Finding a cure

Is there a way to fix the problem?

Dirty way of forcing the view metadata to be refreshed would be simply recreate the view using original script, but imagine you have 30+ of them – Not good, time consuming and error-prone.

There is a simple, universal and “human-error protected” way – after changes are applied to tables, simply refresh metadata of every view defined:

DECLARE @Table_Name varchar(120)
DECLARE Refresh_Views CURSOR FOR
SELECT Table_Name FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_TYPE = 'VIEW' AND TABLE_SCHEMA = 'dbo'
OPEN Refresh_Views
FETCH NEXT FROM Refresh_Views INTO @Table_Name
WHILE @@FETCH_STATUS = 0
BEGIN
EXEC sp_refreshview @Table_Name
print 'refreshed View for: ' + @Table_Name
FETCH NEXT FROM Refresh_Views INTO @Table_Name
END
CLOSE Refresh_Views
DEALLOCATE Refresh_Views

In the script sp_refreshview would tell SQL Server to update the metadata for the all non-schema-bound views in current database.

Note: Alternative: We can also use more generic command – sp_refreshsqlmodule - which “updates the metadata for the specified non-schema-bound stored procedure, user-defined function, view, DML trigger, database-level DDL trigger, or server-level DDL trigger in the current database. Persistent metadata for these objects, such as data types of parameters, can become outdated because of changes to their underlying objects.” This may be even better since it would allow to address any possible schema issues with any objects and not just views.

I have a folder with arbitrary set of SQL scripts and want to apply them all automatically. How do I do that?

In other words:

• Having a folder – ex: C:\AutoSQLScripts – with some unknown set of SQL scripts
• And a SQL Instance where scripts need to be applied
• Run a scheduled task to perform the operation automatically.

Solution:

First, lets figure out single operation - run SQL Script from command line in unattended mode using osql utility:

::
osql -S MYSERVER -d MYDATABASE -E -i MySqlScript.sql  -o report.txt
::

Where:

-S indicates server name
-d is an alternative to USE db_name inside the script
-E indicates that trusted connection would be used
-i identifies the file that contains a batch of SQL statements
-o identifies the file that receives output from osql

Simple so far…

Fun starts when we need to go through files in the folder. Using “magic” of Power Shell, it is also easy:

::
FOR /f "TOKENS=*" %%a IN ('dir /b "%1*.sql"') do ECHO %%a

::or

SET _PATH=%1
IF (NOT %_PATH%=="") SET _PATH = '-p "%_PATH%"'
FORFILES %_PATH% -s -m *.* -c "CMD /C ECHO @FILE"
::

Let’s ignore the end for now and concentrate on the syntax of the loop itself.

FOR /f indicates that we need to perform a loop against a set of files specified in IN. To produce the set we are using “raw” dir of the folder, where %1 is, if specified, an input param of the batch file call.

FORFILES is a little more complex and given here for comparison, yet produces the same result.

Note: commands above work only with “local” (drive: based) locations. If you need apply scripts from Network folder, map it first as a local drive then apply scripts.

Now let’s combine SQL call with the batch processing and we would get the following ApplyScript.cmd which could be used to apply any scripts from current or specific folder to specified SQL Server/Database instance:

::
@ECHO OFF
SET _SERVER=MYSERVER
SET _DB=MYDATABASE

FOR /f "TOKENS=*" %%a IN ('dir /b %1*.sql') DO CALL osql -S %_SERVER% -d %_DB% -E -i %%a  -o report.txt
::

Enjoy.

Some of it is expected since on initial start up, a number of Crystal assemblies and objects need to be created and this does take time.
I want to emphasize that discussion below is related to the first run of any reports. We have to actually restart/reset IIS to see problem again.

Note: There was a question about IIS’ Application Pools recycling which could cause similar effect. Make sure your application pool is set properly.

Setting up a playground

Let’s assume ASP.Net pages were already migrated to use .Net 3.x and look at what happen behind the scene.

Remember that after restart IIS starts from ground up. This means several things in respect to our page content:

• .Net assemblies need to be preloaded and validated if necessary.
  Since Crystal Reports 2008 comes as .Net 2.0-based core and we may already moved to .Net 3.x, CR2008 Engine and some additional files need to be loaded and validated aside from one which already used by any previously loaded pages.
• For each session engine would create a cached version of RPT file in temp folder.
• Some code would have to be brought to the client machine for Viewer to operate properly.
• Database connection established, data retrieved, processed, paged and sent to the client Viewer.

That about it, now we ready to work with reports. Please notice that after that initial load, any other reports would come up faster. We could even close the browser or use another browser (ex. go from IE to FF) and it would be still faster than the first time, so there is something important about the first step above…

Combing the sand

Let’s try to break the IIS initialization process to get more detailed view.

Since we already had some ASP.Net pages loaded before coming to CR related page, we could disregard .Net core initialization procedure. It is there on IIS side: used and ready.

Ok, next thing is CR engine related stuff. There is some 5Mb of files in crystalreportviewers12 folder to support CR Viewer in ASP.Net and we need to send some of them over the network. In addition to that there is Crystal Reports Engine assemblies which need to be loaded by IIS at the time of the first use. And this is our spot to dig.

If we try and trace what exactly going on we would notice that aside from IIS loading a few dozen assemblies, there is also process associated with trying connect to CRL.VERISIGN.NET.

What is it?

“Problem” is that assemblies are Authenticode signed and therefore need to be verified or it technical terms they need to be checked against Certificate Revocation List (CRL) by Publisher for Code Access Security (CAS).

Default behavior is that they need to be verified by the certificate authority. If certificate is not present on the same machine (I have my doubts that SAP doing anything about that, but I could be wrong), validation need to be done via central repository mentioned above, or if machine does not have network/internet access the .NET thread might timeout waiting to connect.

Yes, by performing strong name signing of assemblies or placing the CA certificate on the same machine issue would be avoided, but it seems not being a case.

Building the castle

Since assemblies are provided by SAP, we cannot remove digital signature and it is a hassle to keep certificates current by obtaining them from CA every time they expire. Let’s concentrate on the Publisher mentioned above and turn it off.

It is all-or-nothing solution since it would require turning off CRC for the entire IIS.

When working with regular .Net apps, it can be done on the app level (assuming we already have fix for .Net 2.0) by adding the following section in <Application>.exe.config

<configuration>
  <runtime>
    <generatePublisherEvidence enabled="false"/>
  </runtime>
</configuration>

This new element described in this MSDN article. Interesting note there (why not to turn it then by default? Oh, security concerns… UAC anyone?):

We recommend that services use the <generatePublisherEvidence> element to improve startup performance. Using this element can also help avoid delays that can cause a time-out and the cancellation of the service startup.

Since there is no such config file for our ASP.Net app (web.config would not work here, because it defines settings that are only AppDomain wide and we need process wide for aspnet_isapi.dll being the hosting environment for the runtime), we would have to turn code access security (CAS) publisher policy off for the entire IIS.
There are two options:

• Create a file called w3wp.exe.config (for IIS6, or aspnet_wp.exe.config for IIS5). This will affect all .NET based web applications on the system.
• Or to specify this in machine.config, but then this affects every .NET application on the machine and is not available for override in individual Apps.

Adding the summer cabin

There is one more step which could be taken to improve performance by preloading some of the core assemblies while site visitor is doing something else.
I wouldn’t go into much details here, since it is implementation/application environment specific, but just give a hint:

• some other place in application, create a background process which would create CR document object, load some not essential report file, retrieve some data and then disapear without the trace. This would allow Crystal Reports Engine being initialized in the background offsetting time needed for the actual CR related page load. Don’t force garbage collection though, this may cancel desired effect.

Results

In some situation I observed 50% to 70% drop in start-up time…
Have fun!

We can use homegrown document handling solutions, use commercial 3rd party solutions or… utilize power provided by Google Docs.

When storing articles on Google Docs you are provided with the option to share them with some people only or with everyone. I like to have option to link my articles from anywhere but I also like to have not just plain page, but a page embedded into my web-site pages.

What it means is that instead of external page, I want to put it in my own box.

There was time when it wasn’t as easy because of the additional code Google was injecting into articles, but time changed and now it is fairly easy and strait forward.

Goals

• Embed Google Docs document into existing page
• PHP. This is not really a requirement per se, it is just code below is PHP.
• Retain copyright information

Solution

The following code snippet allows to get a desired result

// id parameter is used to pass Google Docs arcticle id
// Example: mydocview.php?id=nnnnnnnnnnnnnnnn

$id    = $_REQUEST['id'];
if ($id) {
  $started = "0";
  echo('<div class="googledocs" >Stored and managed via <a title="Learn more about Google Docs" href="http://docs.google.com" target="_blank">Google Docs &copy; -- Web word processing, presentations and spreadsheets.</a></div>');

  // Read document content, parse to normalize and output
  $file = fopen ("http://docs.google.com/View?id=" . $id, "r");
  while (!feof ($file)) {
    $line = fgets ($file, 4096);

    // Parse header info out
    if ($started == "0") {
      if (substr($line, 0, 5) == '<body') { $started = "1"; }
      else { continue; }
    }

    // Parse body content until document content block is detected
    if ($started == "1") {
      if (strstr($line, '<div id="doc-contents">')) { $started = "2"; }
      else { continue; }
    }

    // Output document content
    if ($started == "2") {
      // Stop processing, end of content reached
      if (strstr($line, '<div id="google-view-footer">')) { break; }

      // Normalize image links to keep link back to google docs
      $line = str_replace('src="File?id', 'src="http://docs.google.com/File?id', $line);

      // Normalize links to other articles
      $line = str_replace('href="View?docid=', 'href="/mydocview.php?id=', $line);
      $line = str_replace('href="View?id=', 'href="/mydocview.php?id=', $line);
      $line = str_replace('href="View?docID=', 'href="/mydocview.php?id=', $line);
      $line = str_replace('href="View?', 'href="/mydocview.php?', $line);

      // Output result
      echo($line);
    }
  }
  fclose($file);
}

Sample page:

<html>
<head></head>
<body>
<? // Include code above here
?>
</body>
</html>

I continue my journey with Crystal Reports Viewer (CRV) inside ASP.Net pages.
Topic of this post is to discuss problems and solutions related to long running queries in CRV.

Before continue I would like to thank Daniel Paulsen from SAP team for his help resolving the issue discussed below.

What are we talking about here?

As it usually happens at some point there is to much data/not optimal query/broken indexes/etc and it is takes significant time to return a result for CR report. As a result ASP.Net page with CRV start to throw errors, acts erratically, and overall strange.

Since we know that it is ASP.net and CRV uses page state and session information then we have usual suspects - session timeout and execution timeout.

Easy? Not so fast… Lets time our query – if it is more then 10 but less then 20 minutes then reason is not an ASP.Net timeout, but CR Engine timeouts.

Crystal Reports Engine Timeouts

Aside from timeouts imposed by .net infrastructure, there are two others which are introduced by Crystal Report itself.

Symptoms:

• Error similar to ‘Request timed out because there has been no reply from the server in N ms’
• First Page of the report is displayed but attempt to go to the next page gives JS error related to invalid object ID
• If report require parameter entry, they are requested again

Cause:

Crystal Reports by default has a timeout of 10 minutes (600,000 ms) before deciding that no data is returned by the query.
Depend on how code behind is implemented for the report page, timeout error may cause different problems and be hidden.

Solution:

Per Daniel’s suggestion I have changed the following settings to a bigger number and it did solve the problem for me

HKEY_LOCAL_MACHINE\SOFTWARE\Business Objects\Suite 12.0\Report Application Server\Client SDK\CorbaAdapter\WaitReplyTimeout = 600000

and

HKEY_LOCAL_MACHINE\SOFTWARE\Business Objects\Suite 12.0\Report Application Server\InprocServer\EnterpriseRequestTimeout = 600000

Final Note:

There are other settings which are allowed to be set for the CR engine as described in Developer’s Guide for CR 2008 under “Configuring your Web.Config file“. It worth to know about some of them.

Accordingly to a new addition to the Mac-v-PC adv. war – Linux – “WE are Linux” – yes.

Personalization in the look at the computers promoted for last few years by Apple, Microsoft, and then followed by Dell and others with “personalize your computer” campaigns now suggested to be replaced with big “communitification“  a-la Linux.

What is a freedom? Ability to be you through the lens of the Big Brothers or ability to be you as part of the Open (Source) World of WE?

Is it a freedom when someone tells you that “you are free to be me”? Does Linux community need to go into war with others, even if it is just advertisements, to prove itself?

I did not like it when Apple said “you are the sad PC and I am the coolest Mac”…

Microsoft had managed not to mention Apple…

What now? Now we have a new addition – a Linux’s freedom – The big “O”

Did it worth it? What is in it for Linux?

And I have expressed some of my concerns already, but now I have actually had a chance to try it last weekend… And there are my thoughts and ramblings.

To say I do like my Honda (Accord) and I was looking to replace my Nissan (Altima) with a new car.

With prices on gas, on cars, new models and personal preferences, a new Honda Insight was my target.

I have tried this EX model and I am afraid that I would have to look elsewhere or enjoy my old car a little longer and there is why…

• As I said before, 40/43mpg (48/45 mpg for Prius) is not an impressive statement from Honda for the comeback of the Insight. Not at all. Yes, driving in ECO mode and trying to stay in the “green” zone may help improve the mileage, but… There are some reports of 60+ on the highway, but I want to see it first and test-drive is probably not a good test for it and I do not want to turn everything off and drive at cruising speed just to get good mileage as reported here.
• Car feels very fragile when it comes to driving and acceleration. I am somewhat aggressive driver and it was very hard for me to be “green” and accelerate at the same time. There goes mileage…
• To be fair, on the cruise control, car stayed steady in the “green” zone.
• In contrast to some reviews, it was fairly quiet inside a car in all driving conditions and engine noise does not stand out like for ex. in Ford cars I have driven in the past.
• And engine kick-in was not affected much by the AC running at 20% of airflow power.
• No mpg estimation while driving. I read somewhere that it is optional and only available in some models, but I think it should be standard to “prove the point”.
• Small… very. Yes, after Accord and Altima, my expectations are a little bit high, but having 6′ one hits the celling sitting in the back and has claustrophobic feeling sitting in front. Plus foot space was on the verge of rejection/inability to fit.
  In addition it is not as wide as Accord and a rear view was a little bit “compressed”. Otherwise, with bigger side windows, it gives better view to the passing cars.
  Just to compare actual numbers, there is a table showing side-by-side Interior dimensions for Insight, Prius and Accord.
• I did not like light gray interior plastic though, kind of cheap, but it is personal feeling off course.
• I have driven in-city and on highway, and aside from the need to stay “green” car did provide enough power to stay in the crowd and not behind most of the time (yes, acceleration may be a problem )
• Price probably could be cheaper all consider, but 20K is few thousands cheaper then Prius. Question remains if it covers the difference between two.

Bottom line: if you are tall person, this car may not be for you. Add to this unimpressive mileage and some “cheap” feeling of EX model and expectations are ruined. I have had higher expectations, and may be I shouldn’t. Will see if anything would change with next year model. I guess it is time to visit Toyota or wait for Chevy Volt.

Environment:

• Crystal Reports 2008 engine
• IIS 7
• ASP.Net 3.0 page with CrystalReportViewer (CRV) being used to show report passed by parameter

When trying export as PDF (just an example), CRV defaults the name of the file to CrystalReportViewer1.pdf.
Not very nice since associated RPT name is MyVeryOwnReport.rpt and I have many other which I would expect export file at least inherit.

One would expect that Viewer would have a property allowing specify default export name. Not so fast… there is no such thing today exists.

After some head scratching and research, it just happen that Default Export File Name is actually based of the ID property value of the CRV which is still named as CrystalReportViewer1.

Options?

1. Rename ID value of the control as something else. Still, if we use just one page and load reports dynamically, then it would not help much.

   // ReportPreview.aspx file
   <CR:CrystalReportViewer ID="MyVeryOwnExportFile" />

2. Another option is to populate Default Export file name in the code

   // ReportPreview.aspx.cs file
   protected void Page_Init(object sender, EventArgs e)
   {
        ...
       // Load report here
       ...
       CrystalReportViewer1.ID = "MyVeryOwnExportFile";
   }

   It is important to have it set before any code would use it to manipulate a page view state and create references.

3. Trying manually set export options for the Report instance, would not help much unfortunately since viewer would disregard it.
4. Off course there is always an option of coding export functionality yourself and bypass built-in logic…

This concludes another Crystal Reports dance session.

Talk rumored in 1998 and surfaced once again in 2002 will continue - IBM was in final talks to acquire Sun… It was suspected that some news would be announced by Monday 04/06/2009

http://www.betanews.com/article/IBMs-acquisition-of-Sun-Down-to-the-details/1238698365
http://www.nytimes.com/2009/04/03/technology/business-computing/03blue.html

and they did – it fell apart… offer has been rejected

http://www.bloomberg.com/apps/news?pid=20601103&sid=aijKk7Ur2M0c&refer=news

Will we see negotiation saga continued or would be there another bidder? Or…

In rumors of Sun’s CEO could be replaced, so memorable statement was issued: “As a policy Sun does not comment on rumors or speculation. What we can say is that Sun is committed to its leadership team, growth strategy and building value for its shareholders”. Yahoo anyone?

It is not doom and gloom for Sun, but it all went against market expectations… and what is in there for IBM really?

• Cannot overrun a competitor, buy it
• No more patent wars over Java -> IBM JDK to consume final parts of Sun JDK
• Java push via existing customer base
• Solaris to replace AIX? Is there market left for Solaris? IBM will keep the goodies and Solaris to go away?
• NetBeans or Eclipse? Rational or Sun Studio? -> Eclipse+Rational with NetBeans flavor?
• + IBM JavaFX
• + IBM X86 virtualization

All good and nice, but remember the Lotus story? Died silently… what would follow? Java? mySQL? New Google DB engine to emerge built by ex Sun employees? Another era of mainframe computers with Java only present on enterprise level?

I am not pro-Java person, but this is the milestone not to miss!