Serge's Technology View

Talk about Technologies, Software Architecture and Management

Archive for the ‘Fun stuff with SQL Server’ Category

SQL tasks automation: apply scripts from folder

You will be amazed how common this question is:

I have a folder with arbitrary set of SQL scripts and want to apply them all automatically. How do I do that?

In other words:

  • Having a folder – ex: C:\AutoSQLScripts – with some unknown set of SQL scripts
  • And a SQL Instance where scripts need to be applied
  • Run a scheduled task to perform the operation automatically.

Solution:

First, lets figure out single operation - run SQL Script from command line in unattended mode using osql utility:

::
osql -S MYSERVER -d MYDATABASE -E -i MySqlScript.sql  -o report.txt
::

Where:

-S indicates server name
-d is an alternative to USE db_name inside the script
-E indicates that trusted connection would be used
-i identifies the file that contains a batch of SQL statements
-o identifies the file that receives output from osql

Simple so far…

Fun starts when we need to go through files in the folder. Using “magic” of Power Shell, it is also easy:

::
FOR /f "TOKENS=*" %%a IN ('dir /b "%1*.sql"') do ECHO %%a

::or

SET _PATH=%1
IF (NOT %_PATH%=="") SET _PATH = '-p "%_PATH%"'
FORFILES %_PATH% -s -m *.* -c "CMD /C ECHO @FILE"
::

Let’s ignore the end for now and concentrate on the syntax of the loop itself.

FOR /f indicates that we need to perform a loop against a set of files specified in IN. To produce the set we are using “raw” dir of the folder, where %1 is, if specified, an input param of the batch file call.

FORFILES is a little more complex and given here for comparison, yet produces the same result.

Note: commands above work only with “local” (drive: based) locations. If you need apply scripts from Network folder, map it first as a local drive then apply scripts.

Now let’s combine SQL call with the batch processing and we would get the following ApplyScript.cmd which could be used to apply any scripts from current or specific folder to specified SQL Server/Database instance:

::
@ECHO OFF
SET _SERVER=MYSERVER
SET _DB=MYDATABASE

FOR /f "TOKENS=*" %%a IN ('dir /b %1*.sql') DO CALL osql -S %_SERVER% -d %_DB% -E -i %%a  -o report.txt
::

Enjoy.

SQL Server 2008: moving forward

SQL Server 2008 has been available since Aug 6, 2008. This does not include CTP (Feb 2008) time-frame.
We are close to the year milestone now.

To my surprise there are still questions around if moving to a new version is necessary.

This is especially difficult question to discuss/suggest when SQL Server 2005 adoption cycle is just about being complete.

  • We all know about “have to wait until first SP is out” rule.
  • What is good about it to justify spending another chunk of IT budget?

To answer first question, there is a cumulative update 1 available since Nov, 2008. One could count it as SP1.

To answer second question could be more difficult.

There is Product Overview White Paper available along with Data sheet where main differences between 2005 and 2008 versions are highlighted.

As you could see many features are enterprise level, but many are to make life regular developers easier as well.

To summarise, while SQL Server 2008 could be considered just an extension to 2005, major enhancements are:

  1. Transparent Data Encryption – add protection to your data storage without custom and usually process extensive solutions
  2. External Key Management – integrate 3rd party security keys and hardware
  3. Enhanced Auditing – monitor data access and modification
  4. Advanced Database Mirroring – data corruption protection, improved performance, new performance counters
  5. Hot CPU management – change hardware configurations without downtime (for supported hardware platforms)
  6. Performance improvements, Service Broker Scalability, Integrated Full-Text Search, Sparse Columns (“zero storage”), no 8,000 byte limit - manage allocated resources and priorities, query plans optimizations, data compression, backup compression (finally :) ), extended performance monitoring tools, 64 bit platform optimization (RAM, hardware, etc)
  7. SQL Server Integration Services (SSIS) improvements
  8. Policy-Based Management
  9. Installation improvements
  10. ADO.Net Entity Framework and CLR Integration – Object-Oriented style support instead of Table/Field approach
  11. LINQ – even though some pieces (?temporary solutions?) were available before, now LINQ becomes first-class citizen for SQL Server. Forget your T-SQL and program using LINQ syntax.
  12. T-SQL improvements – even though LINQ may be a way to go, T-SQL still is not forgotten with added support for Table Value Parameters (TVP), Object Dependencies Views and Functions, new data types: DATE, TIME, DATETIMEOFFSET, DATETIME2 allowing conserve storage space and improve time precision, FILESTREAM, GEOGRAPHY and GEOMETRY  datatypes, and HIERARCHYID for better support of tree based data structures
    Compound operators – lovely “+=” syntax and in-line variable assignment DECLARE @myVar int = 5
    MERGE support – manage data inserts/updates in one SQL statement
  13. SQL Management Studio gets IntelliSense and other improvements and better Source Control integration
  14. SQL Management Studio T-SQL debugging – not as robust as it is in Visual Studio, but sufficient in many situations.
  15. Table Partitioning management
  16. Hot-swap data connectors – easier support for disconnected data applications
  17. Improved MS Reporting Services and report builder- MS Word/Excel/SharePoint integration improved
  18. OLAP support improvements

Even if you decided to stay with 2000/2005 version, keep in mind a few things which will make your future upgrades easier.

The list of features dropped/changed in SQL Server 2008.

  1. Deprecated SQL Server Features in SQL Server 2008
    • DTS support (link) is replaced with SSIS (migration)
    • ADHelper service
    • SOAP/HTTP endpoints, sys.http_endpoints, sys.endpoint_webmethods (Transact-SQL)
  2. Discontinued SQL Server Features in SQL Server 2008
    • Some SMO Classes has been discontinued
    • Surface Area Configuration Tool (SAC)
    • Discontinued Command Prompt Parameters for SQL Server Setup
  3. SQL Server Native Client has been upgraded with some behavior changes

How about some technical data and stats? To begin with, Microsoft team is no longer publicly presents any data which will show some benchmarks between versions, editions or between different database server platforms, so we have to relay on someone outside do the job.

Yes, comming with mathematical background I can agree with some arguments that this data is not accurate and always represent just what was tested in each particular case in each particular hardware configuration. With statistics being used by marketing department it can be very interesting.

Therefore, we have to be very sceptical about any stats presented anywhere. Nevertheless:

As far as my own experience with a new version - it has been positive and I would highly recommend to go with this upgrade.

TIP: Installing SQL Server 2008 – reboot required check fails

When installing SQL Server 2008 one can run into “‘Reboot required check failed” situation.

After seeing it few times already, I think a solution is worth mentioning.

“Why don’t you just reboot?”, you say… Well, most likely it would not help you, but try it first.
If this would not help, then try the following:

  1. Start regedit.exe
  2. Navigate to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager
  3. Locate PendingFileRenameOperations
  4. Remove any data associated with the value (you may want to export the value for later review)
  5. Reboot and rerun installation, check should pass.

Update. As a respond to some comments about solution above not always work you may consider the following:

  • Make sure that after you reboot entry in the registry does not reappear.
    If it does, remove it again, but do not reboot, just go ahead with installation process and click “ReRun”. Most likely you would be fine now.
  • You may also try and see that other ControlSet001, ControlSet002, etc do not suffer from the same problem (solution mentioned by Mike Hills below).
  • There was mentioning that some installations of Visual Studio 2008 could cause check to fail as well. So if nothing from above helped, uninstall Visual Studio (comes with light version of SQL Server), install SQL Server and then reinstall Visual Studio again.

Lets see why would entry reappear… This may happen if there is a driver or application which supports “recovery” mode or plain virus is around and after reboot it is trying to finish the deployment process again. In any situation try and look closer: what app file belongs to, if there are other solutions to the problem so that original process would finish properly, etc.

Update: Recently I was working on the unrelated setup automation for Visual Studio and stepped on a hidden gem which may work here as well: when running setup from command prompt, Windows Installer accepts a parameter called SkipRules. It worth mention the following “How to: Install SQL Server 2008 from the Command Prompt” first and then look at desired parameter

/SkipRules=VSShellInstalledRule RebootRequiredCheck

We can ignore first rule, since it is VS related, but second is the one you may want to try.

Valid XHTML 1.0 Transitional  Valid CSS!