Serge's Technology View

Talk about Technologies, Software Architecture and Management

Posts Tagged ‘sql server’

SQL tasks automation: apply scripts from folder

You will be amazed how common this question is:

I have a folder with arbitrary set of SQL scripts and want to apply them all automatically. How do I do that?

In other words:

  • Having a folder – ex: C:\AutoSQLScripts – with some unknown set of SQL scripts
  • And a SQL Instance where scripts need to be applied
  • Run a scheduled task to perform the operation automatically.

Solution:

First, lets figure out single operation - run SQL Script from command line in unattended mode using osql utility:

::
osql -S MYSERVER -d MYDATABASE -E -i MySqlScript.sql  -o report.txt
::

Where:

-S indicates server name
-d is an alternative to USE db_name inside the script
-E indicates that trusted connection would be used
-i identifies the file that contains a batch of SQL statements
-o identifies the file that receives output from osql

Simple so far…

Fun starts when we need to go through files in the folder. Using “magic” of Power Shell, it is also easy:

::
FOR /f "TOKENS=*" %%a IN ('dir /b "%1*.sql"') do ECHO %%a

::or

SET _PATH=%1
IF (NOT %_PATH%=="") SET _PATH = '-p "%_PATH%"'
FORFILES %_PATH% -s -m *.* -c "CMD /C ECHO @FILE"
::

Let’s ignore the end for now and concentrate on the syntax of the loop itself.

FOR /f indicates that we need to perform a loop against a set of files specified in IN. To produce the set we are using “raw” dir of the folder, where %1 is, if specified, an input param of the batch file call.

FORFILES is a little more complex and given here for comparison, yet produces the same result.

Note: commands above work only with “local” (drive: based) locations. If you need apply scripts from Network folder, map it first as a local drive then apply scripts.

Now let’s combine SQL call with the batch processing and we would get the following ApplyScript.cmd which could be used to apply any scripts from current or specific folder to specified SQL Server/Database instance:

::
@ECHO OFF
SET _SERVER=MYSERVER
SET _DB=MYDATABASE

FOR /f "TOKENS=*" %%a IN ('dir /b %1*.sql') DO CALL osql -S %_SERVER% -d %_DB% -E -i %%a  -o report.txt
::

Enjoy.

SQL Server 2008: moving forward

SQL Server 2008 has been available since Aug 6, 2008. This does not include CTP (Feb 2008) time-frame.
We are close to the year milestone now.

To my surprise there are still questions around if moving to a new version is necessary.

This is especially difficult question to discuss/suggest when SQL Server 2005 adoption cycle is just about being complete.

  • We all know about “have to wait until first SP is out” rule.
  • What is good about it to justify spending another chunk of IT budget?

To answer first question, there is a cumulative update 1 available since Nov, 2008. One could count it as SP1.

To answer second question could be more difficult.

There is Product Overview White Paper available along with Data sheet where main differences between 2005 and 2008 versions are highlighted.

As you could see many features are enterprise level, but many are to make life regular developers easier as well.

To summarise, while SQL Server 2008 could be considered just an extension to 2005, major enhancements are:

  1. Transparent Data Encryption – add protection to your data storage without custom and usually process extensive solutions
  2. External Key Management – integrate 3rd party security keys and hardware
  3. Enhanced Auditing – monitor data access and modification
  4. Advanced Database Mirroring – data corruption protection, improved performance, new performance counters
  5. Hot CPU management – change hardware configurations without downtime (for supported hardware platforms)
  6. Performance improvements, Service Broker Scalability, Integrated Full-Text Search, Sparse Columns (“zero storage”), no 8,000 byte limit - manage allocated resources and priorities, query plans optimizations, data compression, backup compression (finally :) ), extended performance monitoring tools, 64 bit platform optimization (RAM, hardware, etc)
  7. SQL Server Integration Services (SSIS) improvements
  8. Policy-Based Management
  9. Installation improvements
  10. ADO.Net Entity Framework and CLR Integration – Object-Oriented style support instead of Table/Field approach
  11. LINQ – even though some pieces (?temporary solutions?) were available before, now LINQ becomes first-class citizen for SQL Server. Forget your T-SQL and program using LINQ syntax.
  12. T-SQL improvements – even though LINQ may be a way to go, T-SQL still is not forgotten with added support for Table Value Parameters (TVP), Object Dependencies Views and Functions, new data types: DATE, TIME, DATETIMEOFFSET, DATETIME2 allowing conserve storage space and improve time precision, FILESTREAM, GEOGRAPHY and GEOMETRY  datatypes, and HIERARCHYID for better support of tree based data structures
    Compound operators – lovely “+=” syntax and in-line variable assignment DECLARE @myVar int = 5
    MERGE support – manage data inserts/updates in one SQL statement
  13. SQL Management Studio gets IntelliSense and other improvements and better Source Control integration
  14. SQL Management Studio T-SQL debugging – not as robust as it is in Visual Studio, but sufficient in many situations.
  15. Table Partitioning management
  16. Hot-swap data connectors – easier support for disconnected data applications
  17. Improved MS Reporting Services and report builder- MS Word/Excel/SharePoint integration improved
  18. OLAP support improvements

Even if you decided to stay with 2000/2005 version, keep in mind a few things which will make your future upgrades easier.

The list of features dropped/changed in SQL Server 2008.

  1. Deprecated SQL Server Features in SQL Server 2008
    • DTS support (link) is replaced with SSIS (migration)
    • ADHelper service
    • SOAP/HTTP endpoints, sys.http_endpoints, sys.endpoint_webmethods (Transact-SQL)
  2. Discontinued SQL Server Features in SQL Server 2008
    • Some SMO Classes has been discontinued
    • Surface Area Configuration Tool (SAC)
    • Discontinued Command Prompt Parameters for SQL Server Setup
  3. SQL Server Native Client has been upgraded with some behavior changes

How about some technical data and stats? To begin with, Microsoft team is no longer publicly presents any data which will show some benchmarks between versions, editions or between different database server platforms, so we have to relay on someone outside do the job.

Yes, comming with mathematical background I can agree with some arguments that this data is not accurate and always represent just what was tested in each particular case in each particular hardware configuration. With statistics being used by marketing department it can be very interesting.

Therefore, we have to be very sceptical about any stats presented anywhere. Nevertheless:

As far as my own experience with a new version - it has been positive and I would highly recommend to go with this upgrade.

Tip: Get list of columns for the table

When you dealing with dynamic SQL or unknown table structure, SQL meta data could help recover the missing information.

There are two main system tables which give you such information:

  • dbo.sysobjects
  • dbo.syscolumns

Example below shows how to gather column list for specified table.

DECLARE
  @TABLENAME varchar(256),
  @COLUMNS varchar(4000)

SET @TABLENAME = 'mytable'
SET @COLUMNS = ''

SELECT @COLUMNS = @COLUMNS + c.name + ', '
FROM syscolumns c
INNER JOIN sysobjects o ON o.id = c.id
WHERE o.name = @TABLENAME AND o.xtype='U'
ORDER BY colid

SET @COLUMNS = SUBSTRING(@COLUMNS, 1, Datalength(@COLUMNS) - 2)

SELECT @COLUMNS

or if you prefer more generic approach

DECLARE
  @TABLENAME varchar(256),
  @COLUMNS varchar(4000)

SET @TABLENAME = 'mytable'
SET @COLUMNS = ''

SET @COLUMNS = ''
SELECT @COLUMNS = @COLUMNS + COLUMN_NAME + ', '
FROM INFORMATION_SCHEMA.Columns
WHERE TABLE_NAME = @TABLENAME
ORDER BY ORDINAL_POSITION

SET @COLUMNS = SUBSTRING(@COLUMNS, 1, Datalength(@COLUMNS) - 2)
SELECT @COLUMNS

Result is a coma-separated list of columns in the table in order they appear.

Note. Check for xtype in the first example could be useful but not required, since you probably do not mix table name with any other object names. Otherwise, it would be necessary to ensure that you are looking at the table and not something else.
Other possible values for this column are:

  • S – System tables
  • U – User table
  • TR – Triggers
  • P – Stored procedure
  • RF – Replication filter stored procedure
  • X – Extended stored procedure
  • V – View
  • TF – Functions
  • C – CHECK constraint
  • D – DEFAULT constraint
  • F – FOREIGN KEY constraint
  • PK – PRIMARY KEY constraint (type is K)
  • UQ – UNIQUE constraint (type is K)
  • L – Log

Valid XHTML 1.0 Transitional  Valid CSS!