Archive for the ‘Database’ Category

Some Useful SQL commands for MSSQL DB Admins

Monday, July 2nd, 2012

Sometimes we just can’t get our hands on that right “command” or the perfect “script”, that will simply “do my job (no bells or whistles required)”. After some hard googling, you may find one like a gift you have always wished or had to go the toiling way of writing, testing and putting into use, your own code for that particular need. In the following post I will discuss about some SQL commands and scripts (intended for MSSQL DB hosting) that comes handy in performing those tasks, that would otherwise take a lot of effort and time. You may find them scattered around the world wide web in fragments, but in my opinion finding them in one place have obvious advantage.

Take backup of a particular database and set it offline

This is a likely situation in a database migration scenario. You take backup of a database and immediately take it offline to avoid further data operations into it. You have to run the following SQL from sa login.

BACKUP DATABASE <dbname>
TO DISK = '<backup_path><dbname>.Bak'
WITH FORMAT,
NAME = 'Full Backup of <dbname> on <backup_time>, <backup_date>'
GO

ALTER DATABASE <dbname> SET OFFLINE
GO

<backup_path> may be something like “D:\migrate_database\” and <dbname> is the name of the database.

List tables and stored procedures with a specific schema

You may need to modify schema of tables and stored procedures in batch. This situation arises when you need to delete one user from the database who owns a lot of tables and stored procedures. In order to list those tables and procedures, run the following SQL commands.

USE <dbname>;

SELECT * FROM INFORMATION_SCHEMA.TABLES WHERE INFORMATION_SCHEMA.TABLES.TABLE_SCHEMA = '<dbuser>';

SELECT * FROM INFORMATION_SCHEMA.ROUTINES WHERE INFORMATION_SCHEMA.ROUTINES.SPECIFIC_SCHEMA = '<dbuser>';

Now you can prepare and run a batch SQL script with commands like the following one.

ALTER SCHEMA dbo TRANSFER <dbuser>.<tablename>;

Find log size of all the databases

The following procedure will list the size of log file used by each database.

SELECT INSTANCE_NAME AS [DATABASE],
(CNTR_VALUE/1000) AS Size_In_MB FROM MASTER.dbo.SYSPERFINFO
WHERE COUNTER_NAME LIKE '%Log File(s) Size (KB)%'
AND INSTANCE_NAME NOT IN ('_TOTAL','mssqlsystemresuorce')
ORDER BY Size_In_MB DESC

Find recovery model of all the databases

Run the following procedure to know the recovery model used in each database.

Select name, DatabasePropertyEx(name, 'Recovery') from master..sysdatabases

A database’s log file size is influenced by the recovery model used in it.

Find failed login attempts to ‘sa’ account

Your public SQL server may be under constant brute force attack for ‘sa’ account. The following script will list the numbers and IPs from where the attacks are originated. This may be helpful in implementing your firewall policies.

declare @path VARCHAR(512)
select @path = path from sys.traces where is_default = 1
SELECT loginname, IP, Max(starttime) as LastErrorTime, COUNT(*) AS AttempCounts
FROM (
SELECT t.loginname, REPLACE(SUBSTRING(t.textdata, CHARINDEX(':', t.textdata, CHARINDEX(':', t.textdata, 1) + 1) + 1, 16), ']', '') AS IP, t.starttime
FROM dbo.fn_trace_gettable(@path, DEFAULT) AS t INNER JOIN
sys.trace_events AS e ON t.eventclass = e.trace_event_id
WHERE (eventclass = 20)) AS A
GROUP BY loginname, IP
order by LastErrorTime desc

Find cached page counts and memory uses for each database

The following two SQL commands will show page counts and memory used for each database in a SQL server instance.

SELECT
(CASE WHEN ([is_modified] = 1) THEN 'Dirty' ELSE 'Clean' END) AS 'Page State',
(CASE WHEN ([database_id] = 32767) THEN 'Resource Database' ELSE DB_NAME (database_id) END) AS 'Database Name',
COUNT (*) AS 'Page Count'
FROM sys.dm_os_buffer_descriptors
GROUP BY [database_id], [is_modified]
ORDER BY [Page Count], [is_modified];
GO

SELECT count(*)AS cached_pages_count, (count(*) * 8)/1024 As Mbytes,  db_name(database_id)
FROM sys.dm_os_buffer_descriptors
GROUP BY db_name(database_id) ,database_id
ORDER BY cached_pages_count DESC;
GO

Get list of all current connections to SQL Server instance

The following SQL command lists all current connections to the SQL Server instance. It generates list by code and is flexible and easy to analyze.

SELECT DB_NAME(dbid)AS ConnectedToDB,
hostname, program_name,loginame,
cpu, physical_io, memusage, login_time,
last_batch, [status]
FROM master.dbo.sysprocesses
ORDER BY dbid, login_time, last_batch
GO

Also you can filter the results for a single user or a single database by providing filter criteria in where clause as required. For example to get just connections to database you may add following condition in where clause

WHERE dbid = DB_ID ('AdventureWorks')

Similarly you may filter the results with relevance to any column as required. Above script can be used on all versions of SQL Server.

Complete memory usage report of an SQL server

The following SQL script will generate a complete memory usage report for a SQL server instance.

DECLARE @pg_size INT, @Instancename varchar(50)

SELECT @pg_size = low from master..spt_values where number = 1 and type = 'E'

SELECT @Instancename = LEFT([object_name], (CHARINDEX(':',[object_name]))) FROM sys.dm_os_performance_counters WHERE counter_name = 'Buffer cache hit ratio'

PRINT '----------------------------------------------------------------------------------------------------'

PRINT 'Memory usage details for SQL Server instance ' + @@SERVERNAME  + ' (' + CAST(SERVERPROPERTY('productversion') AS VARCHAR) + ' - ' +  SUBSTRING(@@VERSION, CHARINDEX('X',@@VERSION),4)  + ' - ' + CAST(SERVERPROPERTY('edition') AS VARCHAR) + ')'

PRINT '----------------------------------------------------------------------------------------------------'

SELECT 'Memory Configuration on the Server visible to Operating System'

SELECT physical_memory_in_bytes/1048576.0 as [Physical Memory_MB], physical_memory_in_bytes/1073741824.0 as [Physical Memory_GB], virtual_memory_in_bytes/1048576.0 as [Virtual Memory MB] FROM sys.dm_os_sys_info

SELECT 'Buffer Pool Usage at the Moment'

SELECT (bpool_committed*8)/1024.0 as BPool_Committed_MB, (bpool_commit_target*8)/1024.0 as BPool_Commit_Tgt_MB,(bpool_visible*8)/1024.0 as BPool_Visible_MB  FROM sys.dm_os_sys_info

SELECT 'Total Memory used by SQL Server instance from Perf Mon '

SELECT cntr_value as Mem_KB, cntr_value/1024.0 as Mem_MB, (cntr_value/1048576.0) as Mem_GB FROM sys.dm_os_performance_counters WHERE counter_name = 'Total Server Memory (KB)'

SELECT 'Memory needed as per current Workload for SQL Server instance'

SELECT cntr_value as Mem_KB, cntr_value/1024.0 as Mem_MB, (cntr_value/1048576.0) as Mem_GB FROM sys.dm_os_performance_counters WHERE counter_name = 'Target Server Memory (KB)'

SELECT 'Total amount of dynamic memory the server is using for maintaining connections'

SELECT cntr_value as Mem_KB, cntr_value/1024.0 as Mem_MB, (cntr_value/1048576.0) as Mem_GB FROM sys.dm_os_performance_counters WHERE counter_name = 'Connection Memory (KB)'

SELECT 'Total amount of dynamic memory the server is using for locks'

SELECT cntr_value as Mem_KB, cntr_value/1024.0 as Mem_MB, (cntr_value/1048576.0) as Mem_GB FROM sys.dm_os_performance_counters WHERE counter_name = 'Lock Memory (KB)'

SELECT 'Total amount of dynamic memory the server is using for the dynamic SQL cache'

SELECT cntr_value as Mem_KB, cntr_value/1024.0 as Mem_MB, (cntr_value/1048576.0) as Mem_GB FROM sys.dm_os_performance_counters WHERE counter_name = 'SQL Cache Memory (KB)'

SELECT 'Total amount of dynamic memory the server is using for query optimization'

SELECT cntr_value as Mem_KB, cntr_value/1024.0 as Mem_MB, (cntr_value/1048576.0) as Mem_GB FROM sys.dm_os_performance_counters WHERE counter_name = 'Optimizer Memory (KB) '

SELECT 'Total amount of dynamic memory used for hash, sort and create index operations.'

SELECT cntr_value as Mem_KB, cntr_value/1024.0 as Mem_MB, (cntr_value/1048576.0) as Mem_GB FROM sys.dm_os_performance_counters WHERE counter_name = 'Granted Workspace Memory (KB) '

SELECT 'Total Amount of memory consumed by cursors'

SELECT cntr_value as Mem_KB, cntr_value/1024.0 as Mem_MB, (cntr_value/1048576.0) as Mem_GB FROM sys.dm_os_performance_counters WHERE counter_name = 'Cursor memory usage' and instance_name = '_Total'

SELECT 'Number of pages in the buffer pool (includes database, free, and stolen).'

SELECT cntr_value as [8KB_Pages], (cntr_value*@pg_size)/1024.0 as Pages_in_KB, (cntr_value*@pg_size)/1048576.0 as Pages_in_MB FROM sys.dm_os_performance_counters WHERE object_name= @Instancename+'Buffer Manager' and counter_name = 'Total pages'

SELECT 'Number of Data pages in the buffer pool'

SELECT cntr_value as [8KB_Pages], (cntr_value*@pg_size)/1024.0 as Pages_in_KB, (cntr_value*@pg_size)/1048576.0 as Pages_in_MB FROM sys.dm_os_performance_counters WHERE object_name=@Instancename+'Buffer Manager' and counter_name = 'Database pages'

SELECT 'Number of Free pages in the buffer pool'

SELECT cntr_value as [8KB_Pages], (cntr_value*@pg_size)/1024.0 as Pages_in_KB, (cntr_value*@pg_size)/1048576.0 as Pages_in_MB FROM sys.dm_os_performance_counters WHERE object_name=@Instancename+'Buffer Manager' and counter_name = 'Free pages'

SELECT 'Number of Reserved pages in the buffer pool'

SELECT cntr_value as [8KB_Pages], (cntr_value*@pg_size)/1024.0 as Pages_in_KB, (cntr_value*@pg_size)/1048576.0 as Pages_in_MB FROM sys.dm_os_performance_counters WHERE object_name=@Instancename+'Buffer Manager' and counter_name = 'Reserved pages'

SELECT 'Number of Stolen pages in the buffer pool'

SELECT cntr_value as [8KB_Pages], (cntr_value*@pg_size)/1024.0 as Pages_in_KB, (cntr_value*@pg_size)/1048576.0 as Pages_in_MB FROM sys.dm_os_performance_counters WHERE object_name=@Instancename+'Buffer Manager' and counter_name = 'Stolen pages'

SELECT 'Number of Plan Cache pages in the buffer pool'

SELECT cntr_value as [8KB_Pages], (cntr_value*@pg_size)/1024.0 as Pages_in_KB, (cntr_value*@pg_size)/1048576.0 as Pages_in_MB FROM sys.dm_os_performance_counters WHERE object_name=@Instancename+'Plan Cache' and counter_name = 'Cache Pages'  and instance_name = '_Total'

At Diadem Technologies we have been using these scripts in our day to day DBA work and have benefited from these code snippets.

You can download all the scripts shown on this page from here: SQL Scripts (.zip)

1 Star2 Stars3 Stars4 Stars5 Stars (1 votes, average: 5.00 out of 5)
Loading ... Loading ...

MS SQL Server backup & maintenance using Expressmaint

Friday, July 15th, 2011

Expressmaint utility is a very handy tool to backup and do different maintenance activities for SQL server 2005/2008 databases.

You can perform various kind of administrative jobs in your SQL server database without having a in depth knowledge of SQL server maintenance facilities available with it. The jobs that can be done using this small but powerful utility are:

  • Full Database Backup
  • Differential Database Backup
  • Log Backup
  • Housekeeping of backup files
  • Database Integrity Checks
  • Database Index Rebuilds
  • Database Index Reorganization
  • Database Statistics Update
  • Report Creation

The Expressmaint Utility can be downloaded from the below mentioned links:

ExpressMaint utility for SQL 2005: http://www.sqldbatips.com/samples/code/ExpressMaint.zip

ExpressMaint utility for SQL 2008: http://www.sqldbatips.com/samples/code/ExpressMaint2008.zip

Example Syntax

Note that the entire command should be on one line even though some examples span multiple lines for formatting purposes

1) Full Database Backup of all user databases to c:\backups, verify the backups and report to c:\reports keeping backups for 1 day and reports for 1 week and continue processing other databases if an error is encountered baking up a database

expressmaint -S (local) -D ALL_USER -T DB -R c:\reports -RU WEEKS -RV 1 -B c:\backups -BU DAYS -BV 1 -V -C

2) Full Database Backup of all system databases to c:\backups, verify the backups and report to c:\reports keeping backups for 1 week and reports for 1 week

expressmaint -S (local) -D ALL_SYSTEM -T DB -R c:\reports -RU WEEKS -RV 1 -B c:\backups -BU DAYS -BV 1 -V

3) Log Backup of all user databases to c:\backups, don’t verify the backups and report to c:\reports keeping backups for 1 day and reports for 1  day

expressmaint -S (local) -D ALL_USER -T LOG -R c:\reports -RU DAYS -RV 1 -B c:\backups -BU DAYS -BV 1

4) Check the integrity of the AdventureWorks database and report to c:\reports keeping reports for 1 week

expressmaint -S (local) -D AdventureWorks -T CHECKDB -R c:\reports -RU WEEKS -RV 1

5) Rebuild all indexes in the AdventureWorks database and report to c:\reports keeping reports for 1 day

expressmaint -S (local) -D AdventureWorks -T REINDEX -R c:\reports -RU DAYS -RV 1

6) Update all statistics with fullscan in the AdventureWorks database and report to c:\reports keeping reports for 1 day

expressmaint -S (local) -D AdventureWorks -T STATSFULL -R c:\reports -RU DAYS -RV 1

Automating backups using ExpressMaint

Since SQL Server 2005 Express Edition does not include SQL Agent, we need to rely on the Windows Task Scheduler to run our maintenance tasks. If you are not familiar with how to set up a scheduled task, it’s worth reviewing the Microsoft Knowledge Base article below

How to Schedule Tasks in Windows XP

  • Double-click Add Scheduled Task to start the Scheduled Task Wizard, and then click Next in the first dialog box
  • Click Browse, browse to Expressmaint.exe, and then click Open.
  • Type a name for the task e.g DAILY FULL BACKUP and then choose Daily from the scheduling options
  • Click Next, specify the information about the time to run the task e.g. 00:00, and then click Next
  • Type the name and password of the account that will execute this task. Make sure that you choose an account that is a syadmin for your instance
  • Click Next, select the checkbox to Open the Advanced Properties for this task and then click Finish
  • In the Run text box append the arguments you want to pass to the ExpressMaint utility. (You must leave a space after the existing contents)
  • Click OK. If prompted, supply the password for the account again
1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading ... Loading ...

Howto shrink MSSQL DB and transaction logs without Admin privileges

Friday, February 11th, 2011

If you do not have Administrator privilege on the MS SQL Database server (common for shared hosting environments), you may often face a problem with shrinking the database or the transaction log, specially if the disk space is limited for you. You can easily shrink your database and log files using the simple scripts given below, even if you have the restricted db_owner database role.

Script to shrink the database

USE <databasename>
DBCC SHRINKDATABASE (<databasename>, 0);
GO

NOTE: <databasename> is the name of the database which you want to shrink. The second parameter in the SHRINKDATABASE command denotes the percentage of free space you want to keep in the database. If you put 0 (Zero) it means the database will be shrunk to the minimum possible size.

Script to shrink the transaction log

USE <databasename>
GO
ALTER DATABASE <databasename> SET RECOVERY SIMPLE
DBCC SHRINKFILE(2,0)
ALTER DATABASE <databasename> SET RECOVERY FULL
GO

NOTE: <databasename> is the name of the database for which you want to shrink the transaction log. Recovery mode of the database needs to be changed to simple to shrink the log file successfully which must be changed back to full.
The first parameter in the DBCC SHRINKFILE command denotes the file which you want to shrink and the log file is defined by the numeric value 2. The second parameter is as same as the SHRINKDATABASE command where 0 (Zero) is shown to shrink the log file to its minimum size.

The above commands can be run through the SQL Query Analyser by connecting to the remote DB from your end. Hopefully this will help you to manage database files to keep them in size.

Howto automate hot backups of SQL Server 2005 databases

Monday, January 17th, 2011

There is often a need for a hot backup of a live SQL server database as a standby. To accomplish this a stored procedure can be created in the master database and called periodically using SQL server Jobs with proper arguments. The script provided below is for MSSQL server 2005 database but the same can be done for SQL server 2000 and SQL server 2008 databases too. I will explain the changes to be done for the same as well.

Run the script below in the Query window on the “Master” database to create the DB copy stored procedure.

SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO

USE master
GO

IF object_id(‘dbo.Dbcopy_sp’,'P’) IS NOT NULL
BEGIN
PRINT ‘Procedure already exists. So, dropping it’
DROP PROC dbo.Dbcopy_sp
END
GO

CREATE PROCEDURE Dbcopy_sp
(
@DB varchar(200),
@BackupFile varchar(2000),
@TestDB varchar(200),
@RestoreFile varchar(2000)
)

AS
BEGIN

DECLARE @query varchar(2000)

DECLARE @DataFile varchar(2000)

SET @DataFile = @RestoreFile + ‘.mdf’

DECLARE @LogFile varchar(2000)

SET @LogFile = @RestoreFile + ‘.ldf’

IF @DB IS NOT NULL

BEGIN

SET @query = ‘BACKUP DATABASE ‘ + @DB + ‘ TO DISK = ‘ + QUOTENAME(@BackupFile, ””)

EXEC (@query)

END

IF EXISTS(SELECT * FROM sysdatabases WHERE name = @TestDB)

BEGIN

SET @query = ‘DROP DATABASE ‘ + @TestDB

EXEC (@query)

END

RESTORE HEADERONLY FROM DISK = @BackupFile

DECLARE @File int

SET @File = @@ROWCOUNT

– This always returned 0 for me but the

– RESTORE call returned the number

– of rows associated with the backup. Strange…

DECLARE @Data varchar(500)

DECLARE @Log varchar(500)

SET @query = ‘RESTORE FILELISTONLY FROM DISK = ‘ + QUOTENAME(@BackupFile , ””)

CREATE TABLE #restoretemp

(

LogicalName nvarchar(500),

PhysicalName nvarchar(500),

Type varchar(10),

FilegroupName nvarchar(200),

Size numeric(30,0),

MaxSize numeric(30,0),

FileID bigint,

CreateLSN numeric(25,0),

DropLSN numeric(25,0),

UniqueId uniqueidentifier,

ReadOnlyLSN numeric(25,0),

ReadWriteLSN numeric(25,0),

BackupSizeInBytes bigint,

SourceBlockSize int,

FileGroupId int,

LogGroupGUID uniqueidentifier,

DifferentialBaseLSN numeric(25,0),

DifferentialBaseGUID uniqueidentifier,

IsReadOnly bit,

IsPresent bit

)

INSERT #restoretemp EXEC (@query)

SELECT @Data = LogicalName FROM #restoretemp WHERE type = ‘D’

SELECT @Log = LogicalName FROM #restoretemp WHERE type = ‘L’

PRINT @Data

PRINT @Log

TRUNCATE TABLE #restoretemp

DROP TABLE #restoretemp

SET @query = ‘RESTORE DATABASE ‘ + @TestDB + ‘ FROM DISK = ‘ + QUOTENAME(@BackupFile, ””) +

‘ WITH MOVE ‘ + QUOTENAME(@Data, ””) + ‘ TO ‘ + QUOTENAME(@DataFile, ””) + ‘, MOVE ‘ +

QUOTENAME(@Log, ””) + ‘ TO ‘ + QUOTENAME(@LogFile, ””) + ‘, FILE = 1, RECOVERY’

EXEC (@query)

END
GO

After you run the above script you can find a Stored procedure created in the name of  “Dbcopy_sp” in the master database. You can run the stored procedure manually or call it from scheduled jobs provided by SQL server agent for creating the database copy. The  variables that needs to be passed to the stored procedure are as below:

  • @DB = The original database name which you want to copy.
  • @BackupFile = The database backup file name that will be created along with the path( e.g : C:\Databases\dbcopy\Sourceddbackup.bak )
  • @TestDB = The backup database name that will be created
  • @RestoreFile = The new database file name without .mdf/.ldf (e.g  C:\Databases\destinationdb)

For scheduling the database copy creation create a job with below command:

EXEC   Dbcopy_sp    <sourcedatabase> ,  ‘C:\Databases\dbcopy\Sourceddbackup.bak’ ,  <destinationdatabase> ,  ‘C:\Databases\destinationdatabase’

This will create a database named destinationdatabase and the associated .mdf and .ldf files will be placed under the directory “C:\Databases”. You will need to alter the database name and backup/restore location according to your need.

NOTE: The procedure mentioned above uses the file level restore and hence the fields defined in the table “restoretemp” in the stored procedure has to be altered for SQL Server 2000 and SQL Server 2008/2008 R2. Please visit the below links for the exact fields and their data types.

For SQL Server 2000:

http://msdn.microsoft.com/en-us/library/aa238420%28v=sql.80%29.aspx

For SQL Server 2008/2008 R2

http://msdn.microsoft.com/en-us/library/ms173778.aspx

1 Star2 Stars3 Stars4 Stars5 Stars (2 votes, average: 4.50 out of 5)
Loading ... Loading ...

Tips to optimise MySQL server performance

Tuesday, January 11th, 2011

Here are a couple of tweaks which can improve your mysql server performance significantly. We implemented this on our hosted servers and the php/mysql queries for dynamic DB driven applications  improved by over 50%. The following are the relevant commands which can be inserted in the mysql conf file, which is normally located at /etc/my.cnf.

# Improves mysql server performance by disabling name resolution and remote mysql connections
skip-name-resolve
skip-networking

The skip-name-resolve option will disable name resolution for hostnames and the skip-networking option would disable remote connections to be made to the local MySQL server. This is a recommended setting and would improve your server security as well.

Feel free to post your queries and share any other optimisation tips as well!

1 Star2 Stars3 Stars4 Stars5 Stars (2 votes, average: 2.50 out of 5)
Loading ... Loading ...

Crystal Report issue with Plesk and installing runtime component

Thursday, March 12th, 2009

There is an issue with Crystal reports support with Plesk installation . We have found that in spite of  installing the crystal report runtime redistributable package the report won’t show in the web page on any hosted domain in Plesk, but if you run the report under a virtual directory under the default website the report will be displayed properly. The reason behind this is the IIS application pool launching user (identity). By default the “Default Application pool” launches using the “Network Service” privilege which has the privilege to access the Crystal report assemblies that are referenced in your web.config file.  By Default when a domain is hosted through Plesk control panel it creates a separate application pool for it and the user to launch its application pool is set to “IWPD_domain”. So to resolve the issue either one has to change the application pool Identity to “Network Service” or create a separate application pool using “DefaultApppol” as template and then change the application setting of the website to use that newly created application pool. Remember to set inheritable NTFS permission to the “Network Service” user on your web site.

You can download the crystal report redistributable packages from the link and install it:

https://websmp130.sap-ag.de/sap(bD1lbiZjPTAwMQ==)/bc/bsp/spn/bobj_download/main.htm.
A step by step pictorial representation is given below:

(more…)