Serge's Technology View

Talk about Technologies, Software Architecture and Management

Archive for the ‘Fun stuff with SQL Server’ Category

Tip: Get list of columns for the table

When you dealing with dynamic SQL or unknown table structure, SQL meta data could help recover the missing information.

There are two main system tables which give you such information:

  • dbo.sysobjects
  • dbo.syscolumns

Example below shows how to gather column list for specified table.

DECLARE
  @TABLENAME varchar(256),
  @COLUMNS varchar(4000)

SET @TABLENAME = 'mytable'
SET @COLUMNS = ''

SELECT @COLUMNS = @COLUMNS + c.name + ', '
FROM syscolumns c
INNER JOIN sysobjects o ON o.id = c.id
WHERE o.name = @TABLENAME AND o.xtype='U'
ORDER BY colid

SET @COLUMNS = SUBSTRING(@COLUMNS, 1, Datalength(@COLUMNS) - 2)

SELECT @COLUMNS

or if you prefer more generic approach

DECLARE
  @TABLENAME varchar(256),
  @COLUMNS varchar(4000)

SET @TABLENAME = 'mytable'
SET @COLUMNS = ''

SET @COLUMNS = ''
SELECT @COLUMNS = @COLUMNS + COLUMN_NAME + ', '
FROM INFORMATION_SCHEMA.Columns
WHERE TABLE_NAME = @TABLENAME
ORDER BY ORDINAL_POSITION

SET @COLUMNS = SUBSTRING(@COLUMNS, 1, Datalength(@COLUMNS) - 2)
SELECT @COLUMNS

Result is a coma-separated list of columns in the table in order they appear.

Note. Check for xtype in the first example could be useful but not required, since you probably do not mix table name with any other object names. Otherwise, it would be necessary to ensure that you are looking at the table and not something else.
Other possible values for this column are:

  • S – System tables
  • U – User table
  • TR – Triggers
  • P – Stored procedure
  • RF – Replication filter stored procedure
  • X – Extended stored procedure
  • V – View
  • TF – Functions
  • C – CHECK constraint
  • D – DEFAULT constraint
  • F – FOREIGN KEY constraint
  • PK – PRIMARY KEY constraint (type is K)
  • UQ – UNIQUE constraint (type is K)
  • L – Log

SQL date operations: Last 90 days, Get Date from DateTime…

I am continuing putting some common examples of operations for Microsoft SQL Server.
In addition to some of the Date related code snippets published before, there are few more today:

-- Last 90 days:
select DATEADD(day, -90, GETDATE())
-- Get Date portion of DateTime value
select DATEADD(d, 0, DATEDIFF(d, 0, getdate()))

SQL Server Collation. An enemy or a friend?

Is SQL Server Collation setting your enemy or your friend? How to change server default collation?

In general, it is your friend, it will help SQL Server to figure out how to store character strings in different locale. It becomes even more important when data you are storing goes beyond regular ANSI charset (Chinese, Japanese, etc.).

In my case I, from time to time work with Cyrillic. Being born and raised in Russia, I am native Russian speaker. As a result I am keeping Cyrillic enabled on my machine as main locale setting.

It is OK in many instances, but when it comes to SQL Server or some other applications which are trying to be smart in this area, it becomes a problem.

If you install SQL Server instance in such environment, you would get something like that

Let check collation for some database I have

SELECT databasepropertyex(db_name(),'collation') AS collation_name

In my case result would be

 Cyrillic_General_CI_AS

If you are at home, it may be fine, but at work when your primary locale is SQL_Latin1_General_CP1_CI_AS
it could be a problem since you often will run into error when execution of the SQL code across different databases will return Collation error.

Too late? Not exactly.

In many cases your solution would be to adjust Collation setting for the database using to following code:

ALTER DATABASE DatabaseName COLLATE SQL_Latin1_General_CP1_CI_AS

Problem would still exist though, any new database created will get default Collation setting from … MASTER database for your SQL Server Instance.

 And here some hoops will be required. Following steps will help you overcome the issue:

  1. Detach all your user databases
  2. Stop your SQL Server
  3. In C:\Program Files\Microsoft SQL Server\80\Tools\Binn\ folder locate rebuildm.exe and run it. This utility which comes with SQL Server (you will need to have client tools to be installed) allows you to repair your master database from original installation CD/DVD/Folder and also change default Collation Setting for your SQL Server instance. Go to Settings and choose appropriate collation designator or SQL Collation.
    If you cannot find rebuildm.exe, it means you are using SQL Server 2005 or 2008. Then the following may help:
    for SQL Server 2005

    start /wait <CD or DVD Drive>\setup.exe /qn INSTANCENAME=<InstanceName> REINSTALL=SQL_Engine REBUILDDATABASE=1 SAPWD=<NewStrongPassword>

    for SQL Server 2008

    setup.exe  /QUIET  /ACTION=REBUILDDATABASE  /INSTANCENAME=instance_name  /SQLSYSADMINACCOUNTS= accounts   [/SAPWD=password]
    
  4. Rebuild your master database
  5. Reattach your user databases
  6. Check that collation for master and user databases is a desired using first query above.

This is much easier then to recreate a SQL Server instance from scratch.

Note. There is less safe way of getting the same result – adjust Windows Registry.
Your default collation name is stored under

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSSQLServer\MSSQLServer\
DefaultCollationName

Load text file data into SQL Server table

Last time we have seen how to manipulate text files from SQL Server code using Scripting object.

This time we would look at “Read File As Table” operation from a different angle – using BULK INSERT operation.

There are pros and cons in this approach. Lets look at them.

Advantages:

  • Getting data in table format, with properly associated columns
  • BULK INSERT is quicker operation if you know what you are importing
  • IDENTITY INSERT and other Batch optimizers
  • Transaction support
  • Is easiest way to copy data between servers

Disadvantages:

  • it is supported for variation of CSV formatted files only
  • structure of the file is strict and small error in formatting or a hidden character can cause problems with import
  • only members of the sysadmin and bulkadmin server roles can perform an operation
  • could be messy if not properly managed.

All saying, you should always try and see if it works for you, instead of creating custom import procedures or using DTS functionality.

Let fun begins:

We would be working with the following table:

CREATE TABLE StoreList (
StoreID INT,
StoreName VARCHAR(32),
  City VARCHAR(32),
  State VARCHAR(2),
  Country VARCHAR(32)
)

Sample data we would be using is stored in CSV file called c:\stores.csv

1111,Stone Age Books,Boston,MA,USA
2222,Harley & Davidson,Washington,DC,USA
3333,Infodata Algosystems,Berkeley,CA,USA

When table is created Bulk insert could be performed using the following command:

BULK INSERT dbo.StoreList FROM 'c:\stores.csv' WITH (
   FIELDTERMINATOR = ',',
   ROWTERMINATOR = '\n'
)

Notice that we are using coma as separator, it might not be appropriate in some situations and could be easily addressed for let say TAB-separated values using

BULK INSERT dbo.StoreList FROM 'c:\stores.csv'
WITH (
   ROWTERMINATOR = '\n'
)

As you can see that FIELDTERMINATOR has been removed. This is a case because ‘\t’ value otherwise used here is a default for the parameter. You can adjust Field and Row terminators to better suite format present in the file in each case.

Another thing to remember that some of the files, especially results from Excel export might include header rows which might make your import fail with conversion error

Bulk insert data conversion error (type mismatch) for…

To address the issue simply exclude header related rows from import

BULK INSERT dbo.StoreList FROM 'c:\stores.csv'
WITH (
    FIRSTROW = 2,
    ROWTERMINATOR = '\n'
)

There are other settings which might affect bulk import:

  • Turning Insert Triggers On while performing import
  • Keep NULL values – situation when there is nothing specified between delimiters
  • specifying LASTROW of the file to import
  • how many error to ignore when found in the file before invalidating a whole import

Please refer to SQL Server Book Online for details about such settings.

Valid XHTML 1.0 Transitional  Valid CSS!