bet.ucoz.co.uk

Posts Tagged ‘Performance’

Monitor High CPU with Extended Events in SQL Server 2012 June 28th, 2012

Vinod Kumar

Extended Events has always been fascinating for me to learn in the previous editions too. But the only drawback of not having an effective UI means that this was tough to configure and more so to use. With SQL Server 2012, this limitation goes out of the door and now we can use some basic UI to work with Extended Events.

In this post, let me take a simple tour of creating a simple Extended Event via SSMS and then we will add appropriate filters to the collection to collect a specific event of some specified value. This is no way exhaustive of what Extended Events can do but surely can be a great starter.

What is Extended Events?

As per BOL, “Extended Events has a highly scalable and highly configurable architecture that allows users to collect as much or as little information as is necessary to troubleshoot or identify a performance problem.” – Read completely on this topic over TechNet.

Step 1: Initializing a session

To get started, Open SQL Server 2012 Management Studio and get to the Management Node. Expand and click on the “Extended Events” node and right click at the Sessions Folder. Select “New Session”.

image

Step 2: Session Name to start

The next step is to give a “Session Name” in the Text Box. Enter a value of “High_CPU” for example. This screen has multiple other Checkbox which allows you to start this Extended Event session upon SQL Server Startup etc. For this blog, we will not select any of them. Feel free to use these advanced features once you got a hang of Extended Events with SQL Sever 2012.

image

Step 3: Add an Event and Filter Values

Once the above is complete, now select the “Events” tab from the left-hand side menu. This will bring you the Events Library to select. Remember we want to get the Queries based on some high CPU value.

Under the “Event Library” text box, apply the filter for ShowPlan. This will get you 3 records like below. Select the “query_post_execution_showplan”. This occurs after a SQL statement is executed. This event returns an XML representation of the actual query plan and we will use the CPU values from this event to get badly performing queries. Once you Select use the small “Right Arrow” to move the event to “Selected Events” section. The wizard will look like below.

image

Step 4: Configure details

Once you reached this step, Click on the “Configure” button on the top right corner. This brings the Event Configuration Options. Select the “Filter (Predicate)” Tab. Apply some filters. For the demo, I am adding two filters on cpu_time and database_name as below. These can be different for you.

image

Step 5: Configure Storage Options

Now the once the event and filter conditions are configured, we need to next configure where we be collecting these events. For this demonstration, we have used a “event_file” option and are writing to “c:\temp” folder with the options like below.

image

This is the final step and click “OK” button at the bottom. We are all set now.

Step 6: Start the session now

Right click on the “High_CPU” node and select start session.

image

Now the configuration is complete and we are all set to start collecting data. Play around with the various options, events and destinations to configure your own collection.

===================================================================

Step 7: Generate a High CPU query

For this blog I am querying a very large table and doing an aggregate to simulate some CPU cycles. Once this is done. Just "double click” the file “c:\temp\High_CPU_x_xxxxx.xel”.

This will Open up the SSMS and you will be able to view the file with some interesting data.

image

This shows you that I have had a Stored Procedure – “High_CPU1” that is causing some 3313000 ms of CPU time and this qualified for the event capture and hence was collected. You can also use Live trace if you from one of the options of right-click on the session. That will be functioning like a typical Profiler sorts with a filteration of event.

Clicking on the Query Plan also reveals the query that was used inside the Stored Procedure that caused this CPU spike.

There are nice ways to now group, filter etc on Extended Events. But I will write it for a future post.

Conclusion

Want to wrap up saying, with SQL Server 2012 Extended Events are way more powerful than the conventional way of collecting data via Profiler. We will get into some more options and advanced data collection in the future. Do feel free to pass me your comments (if any) for future posts too.

Continue reading...


 

Windows PowerShell Links to Bookmark June 24th, 2012

Vinod Kumar

powershell

After writing a bunch of links and resources on Sharepoint, thought it will be great to drop a line around all the links and resources on PowerShell. Hope the administrators who want to work with PowerShell will find these links useful and will use this page as a bookmark for reference. This is just my way to share all the bookmarks from my IE page :). Your feedbacks and comments are most welcome as you browse through this mega-list.

The list below is provided for those who wish to become familiar with Microsoft PowerShell 2.0. Top Level Site for PowerShell: http://www.microsoft.com/powershell

Windows PowerShell Training

Windows PowerShell Getting Started Guide

  • Scripting with Windows PowerShell

PowerShell Essentials for the Busy Admin (5 part video series)

  • Why You Should Learn PowerShell
  • How to Compound Windows PowerShell Commands for Fun and Profit
  • A Look at Windows PowerShell Providers
  • Windows PowerShell Does Event Logs
  • Using Windows PowerShell to Manage the Remote Desktop

Windows PowerShell – Learn it Now Before It’s an Emergency (5 part video series)

  1. Part 1
  2. Part 2
  3. Part 3
  4. Part 4
  5. Part 5

Windows PowerShell Owner’s Manual

  • Getting Started with Windows PowerShell
  • Customizing the Windows PowerShell Console
  • Windows PowerShell Shortcut Keys
  • Piping and the Pipeline
  • Running Windows PowerShell Scripts
  • The Windows PowerShell Profile
  • Windows PowerShell Aliases

Windows PowerShell User’s Guide

  • Windows PowerShell Basics
  • Object Pipeline
  • Windows PowerShell Navigation
  • Working with Objects
  • Using Windows PowerShell for Administration
  • Introducing the Windows PowerShell ISE
  • Appendix 1 – Compatibility Aliases
  • Appendix 2 – Creating a Custom Windows PowerShell Shortcut

Windows PowerShell Webcasts/Podcasts (a list of 76 different PowerShell presentations in multiple formats – to see the entire list select “MORE” on right-hand side opposite the heading that reads “IT Professionals (76)”)

  • Introduction to Windows PowerShell
  • Using Windows PowerShell 2.0
  • Windows PowerShell Basics for IT Professionals (2-part series)
  • TechEd 2011 Birds-of-a-Feather (Sessions 04): PowerShell: Best Practices From The Field
  • Managing Web Infrastructure Systems with Windows PowerShell 2.0
  • How Do I:
  • PowerShell – The Basics
  • PowerShell Variables
  • PowerShell Scripts and the Command Line
  • PowerShell and WMI
  • Creating Output with PowerShell
  • PowerShell – Object Manipulation
  • Pipelining

The VBScript-to-Windows PowerShell Conversion Guide

PowerShell Podcasts

These audios offer Windows PowerShell tips, news and interviews that you can listen to at your convenience.

Windows PowerShell Podcasts

TechNet Radio

Useful PowerShell Blogs

This is a sampling of some of the blogs that are being written on the subject of Windows PowerShell.  Some of these are independent blogs and not from Microsoft.

Hey, Scripting Guy! Blog

Windows PowerShell Team Blog

Get-PowerShellBlog

Keith Hill’s Blog

Richard Siddaway’s Blog

PowerShell Scripts

This section represents links to actual PowerShell scripts that have been developed for review and reuse.  These scripts are intended to keep administrators from having to “start from scratch” when developing scripts.  While they will almost certainly require some modification, the intent is to give admins a starting point to learn how to do various tasks related to PowerShell in their own environments.

Windows PowerShell Script Repository

Microsoft Script Center

PowerShell Forum

This forum is a place where administrators can ask question, read discussions among technical professionals, and engage with the online PowerShell community.

Windows PowerShell Forum

PowerShell Tools

Windows PowerShell (download)

The Windows PowerShell Toolbox

PowerShell Books

The following is a partial list of books that have proven useful as PowerShell resources, both to the new admin and the experienced developer. This is not intended as a complete list of PowerShell books, but is intended only as a sample for the admin looking to start building their library

Windows PowerShell 2.0 Best Practices

Microsoft Windows PowerShell 3.0 Step by Step

Windows PowerShell Scripting Guide

Windows PowerShell 2.0 Administrator’s Pocket Consultant

Additional PowerShell Resources

A variety of external resources offered to the PowerShell community in the form of blogs, sample scripts, wikis, forums, etc.

PowerShellCommunity.org

Windows PowerShell Wiki

CodePlex

Precision Computing

Under the Stairs

PowerShell v3 (beta)

These resources are specifically designed to help prepare for the newest version of PowerShell to be released in the near future.

PowerShell v3 Guide (TechNet)

  1. Introduction
  2. Getting PowerShell v3
  3. Installing PowerShell v3
  4. PowerShell v3 FAQ
  5. Featured Articles
  6. New Features
  7. Tips and Tricks

What’s New in PowerShell v3

PowerShell v3 Sneak Peek (Microsoft TechDays video)

Phew, that is a lot of links and resources you have in your hand to digest. So feel free to pass this page on your Social sites if you find it useful. Do drop a line if you would like to see for other technologies too.

Continue reading...


 

SharePoint 2010 Resources to Bookmark June 19th, 2012

Vinod Kumar

sharepointI know this is not the usual post of my techie but I thought this is worth a post based on some of the bookmarks available with me already. Can become an interesting and handy reference for sure for folks out there.

I just mentioned over my twitter (@vinodk_sql) about this awesome resource to start: SharePoint 2010 Resources for IT Professionals. This is like the root that I want to mention first and then take you through some of the detailed content that is available from there that you can use.

Beginning with SharePoint

Getting Started with SharePoint 2010 for IT Pros (Webcast Series)

  1. Part 1 – What Can SharePoint Server 2010 Do for Me?
  2. Part 2 – How SharePoint Server 2010 Works
  3. Part 3 – Simple Install for Testing in SharePoint Server 2010
  4. Part 4 – Search in SharePoint Server 2010
  5. Part 5 – Backups in SharePoint Server 2010
  6. Part 6 – Creating Your First Web Application, Site Collection and Web in SharePoint Server 2010
  7. Part 7 – Getting Started with Security in SharePoint Server 2010
  8. Part 8 – Windows PowerShell for SharePoint Server 2010 Administrators
  9. Part 9 – Tools to Optimize the Performance of SharePoint Server 2010
  10. Part 10 – Introduction to Upgrade in SharePoint Server 2010
  11. Part 11 – Service Applications and Topologies in SharePoint Server 2010
SharePoint 2010 Advanced IT Professional Training (Webcast Series)

Core Architecture of SharePoint 2010

  • Lesson 1: Understanding SharePoint 2010 Topology
  • Lesson 2: Planning and Leveraging SharePoint 2010 Service Applications
  • Lesson 3: Business Continuity Management Features in SharePoint 2010

Security in SharePoint 2010

  • Lesson 1: Using the New SharePoint 2010 Security Model – Part 1
  • Lesson 2: Using the New SharePoint 2010 Security Model – Part 2

IT Pro Management in SharePoint 2010

  • Lesson 1: Managing SharePoint 2010 Customizations for the IT Pro
  • Lesson 2: IT Pro Tools for Customizing SharePoint 2010

Upgrading to SharePoint 2010

  • Lesson 1: SharePoint 2010 Upgrade Overview
  • Lesson 2: Preparing to Upgrade to SharePoint 2010
  • Lesson 3: Testing Upgraded SharePoint 2010 Solutions
  • Lesson 4: Implementing and Validating Upgrade Plans for SharePoint 2010

Enterprise Search in SharePoint 2010

  • Lesson 1: Search Capabilities and Features in SharePoint 2010

Content Management in SharePoint 2010

  • Lesson 1: Document Management Capabilities and Features in SharePoint 2010
  • Lesson 2: Records Management in SharePoint 2010
  • Lesson 3: Governing Content Types, Policies, and Taxonomy Services in SharePoint 2010

Composite Solutions in SharePoint 2010

  • Lesson 1: Business Connectivity Services in SharePoint 2010 and Office 2010

Communities in SharePoint 2010

  • Lesson 1: Using Social Capabilities in SharePoint 2010

Business Intelligence in SharePoint 2010

  • Lesson 1: Business Intelligence Features in SharePoint 2010

SharePoint Online Overview

  • Lesson 1: SharePoint Online Overview

SharePoint Documentation

When I meet customers I get a lot of queries regarding some of the implementations and nuances. Trust me almost 80-90% of the time it is the official documentation site that gives me the answer and not the internal distribution lists. The following top-level link is for the SharePoint Tech Center on TechNet:

Microsoft SharePoint Server 2010

SharePoint Online Forums

Microsoft maintains an active community of SharePoint professionals who regularly post questions, answers, tips, and general information related to SharePoint. This is also an place where most of the best MVP’s and the support engineers hang around to help customers, partners and enthusiasts who work on SharePoint. This is an excellent way to become familiar with the professionals in the SharePoint community around the world, ask your own questions, and learn more about SharePoint:

SharePoint 2010 Forums

Microsoft SharePoint Certification

Microsoft offers several certification tests related to SharePoint 2010.  These tests contribute to the certifications listed under the “Credit Toward Certification” section listed for each test below:

Exam 70-667: TS: Microsoft SharePoint 2010, Configuring

The minimally qualified candidate typically:

  1. Is an IT administrator who implements and maintains SharePoint Online or an on-premise deployment of SharePoint 2010 Service Pack 1.
  2. Is proficient with IIS 7.0, DNS, Active Directory Domain Services, and Microsoft SQL Server 2008 as these technologies relate to SharePoint.
  3. Is proficient with the infrastructure and security of Windows Server 2008 or later.
  4. Has experience with business operations for IT, including data backup, restoration, and high availability.
  5. Has experience with Windows PowerShell 2.0 and command-line administration.
  6. Has a basic understanding of single sign-on and Active Directory Federation Services 2.0.

Exam 70-668: PRO: Microsoft SharePoint 2010, Administrator

Candidates typically plan, design, and maintain:

  1. Physical topologies and services architecture
  2. Disaster recovery (backup and restore) and availability
  3. Infrastructure capacity (users, licenses, and topology)
  4. Farm performance and availability
  5. Migration, coexistence, and upgrade  
  6. Security and compliance requirements
  7. Information architecture (interprets taxonomy)
  8. Information search strategy integration with other data sources (LOB, third-party products)
  9. Client application services deployment

Exam 77-866: SharePoint 2010

Candidates for this exam are users who provide structure for information, extend out-of-the-box site features, solve business problems through composite applications, and facilitate collaboration with other site users. Users perform all site user tasks. They optimize Web Part pages for team use by adding and configuring Web Parts from the Web Part gallery. They customize dashboards to target the information presented. Users focus on adapting SharePoint sites to their teams’ needs and improving productivity.

Exam 70-573: TS: Microsoft SharePoint 2010, Application Development

A Microsoft Certified Technology Specialist (MCTS) in Microsoft SharePoint 2010, Application Development should be able to perform the following tasks:

  1. Write code that extends SharePoint 2010
  2. Add and support code to an existing project
  3. Write code for and test custom features in a SharePoint solution such as a Visual Web Part or Event Receiver
  4. Implement a solution designed by lead SharePoint Developer

Exam 70-576: PRO: Designing and Developing Microsoft SharePoint 2010 Applications

The candidate is responsible for designing custom code for projects that are deployed to SharePoint servers. This includes technology selection across the many ways to build code in SharePoint, ensuring the team development environment is configured, creating a strategy for code deployment, versioning, configuration, or structure. The candidate also leads a team of SharePoint developers, has at least two years of SharePoint development experience, has at least three years of ASP.NET development experience

More Training Locations to Bookmark

These are few more locations that are worth to book are:

Introductory Developer Training for SharePoint 2010

SharePoint 2010 Developer Training Course

Developer Overview: video that shows some of the great new features for building custom applications and components in SharePoint 2010.

If you are Upgrading from SharePoint 2007:SharePoint 2010 Upgrade Resource Center.

To setup the environment on your machines: setting up the development environment for SharePoint 2010.

SharePoint 2010 101 Code Samples – Great download to learn the tricks

SharePoint 2010 Advanced Developer Training – Another set of videos to look for.

Finally, the Microsoft Certified Master certification also will be available for SharePoint Server 2010. For this certification, you must attend the training program and complete three weekly exams and a qualifying lab exam.

Hope you will find these set of links and resources useful and will bookmark for future reference. Please drop a line if you would like to see more of such consolidated lists of links from my useful links sections :).

Continue reading...


 

SQL Server 2012: Using SEQUENCE May 24th, 2012

Vinod Kumar

There have been multiple options of using running numbers and as developers we have used options of IDENTITY in the past. With SQL Server 2012, there is an interesting option to utilize called as SEQUENCE. I am not sure how many of you are using this new feature in your applications already. In this post wanted to quickly give you some ways to use this nice little addition into SQL Server.

Creating SEQUENCE

The syntax for SEQUENCE can be got from the documentation mentioned above. Let us create a simple table and Sequence to start with for our experiment.

image

In the above script the important part is the last command where we initialize the SEQUENCE. What we created is:

Data type: BIGINT
Starts at 1 which is greater than minvalue and less than maxvalue
MaxValue is max value of the type i.e. 9223372036854775807
MinValue is min value of the type i.e. -9223372036854775808
Increment is 1 – can also be negative.
No cycle – means it will error after maxvalue
Cache is used, database engine will select a size and selection might not be consistent.

‘CREATE SEQUENCE’ does not allow specifying the database name as a prefix to the object name.

Only word of caution is wrt cache. Using of cache can minimize some IO at generation time, but at the same time if any abrupt crash or power outage can mean the cache values can be lost and there can be breakage in the sequence values.

Get the SEQUENCE Metadata from inside SQL Server. Use the sys.sequence DMV for this task.

image

This confirms our initial understanding of the values and how they can be queried from the system. This DMV can be used to see what SEQUENCEs have been defined inside a Database.

Generating SEQUENCE

Generating sequence numbers is a simple step using the “next value” construct.

image

Since we are using the GO 10, there are 10 sequence numbers that get generated.

Quick Quiz: What will be the output from the below query if I run them now.

image

Will the values be:

  1. 10,11
  2. 11,12
  3. 11,11
  4. 12,12
  5. None of the above

Please post the answer over comments and would love to see your explanation for the same too. For correct answers, we will send you Pluralsight Codes to watch our courses.

Let us next, alter an existing SEQUENCE number to different values:

image

Only catch here is: Data type can’t be changed by alter statement. And “START WITH” cannot be used in an ALTER SEQUENCE command.

Given this constrain, decide on if you want to go with smallint, int, bigint or decimals/numeric (with 0 scale). I would suggest the choice would be between int and bigint – not to forget the storage or memory needs from a cache. Given these days few MB’s is fine from an overhead when the world is moving to TB’s of data :).

Using SEQUENCE Inside Transactions

The next aspect is to use the SEQUENCE values inside a transaction and see how they get affected. Use the below status check query to keep a tap on the progress every step.

image

Let us start by using an implicit transaction inside SSMS and insert one value. The output of the same is:

image

This seems to be one of the easiest of steps. Now if we use an Explicit transaction and if we rollback, how would the SEQUENCE Status look like?

image

Now you can see that the values as part of INSERT have been rolled back, yet there is a small difference in the status check – the current value is 11. This confirms that the SEQUENCE has jumped to the next value and doesn’t participate as part of transaction rollback. This is the same / similar behavior to how IDENTITY works inside SQL too. Hence if we insert a new row, it would have skipped a value of 11 as we gave a increment of 10 in the last alter statement.

It is sort of different from IDENTITY because even the IDENTITY can get reset using the TRUNCATE command, but SEQUENCE has to be reinitialized using the ALTER for the same.

image

Getting a range of values

Now that we are talking about sequences, there have been requirements to get a range of values. The same can be achieved using a stored procedure – sp_sequence_get_range. The idea here would be an application might need some range of values it can use from the sequence generated from the backend. In case of pessimistic locking lot of people ask me is it possible to know the IDENTITY before we insert so that they can send the parent and child (with foreign key populated) in one shot into SQL Server. Here is a typical way to achieve that.

image

Let me add the command text used above for easy access.

DECLARE @sequence_name nvarchar(100) = ‘SequenceID’,  @range_size int = 10, 
    @range_first_value sql_variant,  @range_last_value sql_variant,
    @range_cycle_count int,  @sequence_increment sql_variant, 
    @sequence_min_value sql_variant,  @sequence_max_value sql_variant;

EXEC sp_sequence_get_range  @sequence_name = @sequence_name, @range_size = @range_size,
@range_first_value = @range_first_value output,  @range_last_value = @range_last_value output, 
@range_cycle_count = @range_cycle_count output,  @sequence_increment = @sequence_increment output, 
@sequence_min_value = @sequence_min_value output,  @sequence_max_value = @sequence_max_value output;

SELECT RangeSize= @range_size,FirstValue=@range_first_value,LastValue= @range_last_value,CycleCount=@range_cycle_count,
  Increment=@sequence_increment,MinValue= @sequence_min_value,MaxValue= @sequence_max_value;

The documentation is pretty exhaustive and can be helpful for reference– MSDN/BOL.

SEQUENCE Usage

There are a number of ways to use SEQUENCE inside SQL Server. We have already seen how one might use the same inside a SELECT (Option 1) or INSERT Statement (Option 2) above. Let me give you some other options of how one might use SEQUENCE.

Option 3 – Pre-generation and Batching

Get the next value before inserting. If you have not taken the quiz / trivia above – do that. You will realize that the solution for that question is by doing a batch operation like one written below. Unlike in the previous case, all expressions that appear in the same logical query processing phase are evaluated as if at the same point in time hence giving it a batch window eliminates that problem.

image

Option 4 – OVER and Order by

This will be one of the common methods people might start using the SEQUENCEs. Your requirement is to get some sequence or order while you get the resultset and this can be yet another way to use. You can use the overall ranking functions also to get two numbered set one as a master and other as a subset ranking set. A typical example is:

image

Option 5 – Using inside Update statement

There can be a requirement you want to create some running sequence for an already existing values inside SQL Server as part of migrating it to SQL Server 2012. Also you don’t want any breaks in that sequence (like Invoice etc). You can use this technique to generate such running numbers and then reset / reseed if you are using the identity columns. If you get the drift then you can easily build the script using another column and achieve the same. Though I did this for one of the ISV’s keep in mind the Foreign key pitfalls as part of the activity.

image

Option 6 – Use it via DEFAULT Column value

This option is a great way to use in comparison to IDENTITY values. Some of the benefits of cache can surely eliminate the contention on the last page because of IDENTITY value generation. And I see this as a new way to maintain and create unique numbers inside SQL Server. In the example below, the default constraint makes sure the next value has been entered and see the entry of last 8th row automatically.

image

Why SEQUENCE when IDENTITY is there?

UPDATE: A lot of people are asking when to use sequence and when IDENTITY. The scenario is simple, if you need a unique sequence number to be generated across a single database then sequence can be of great help. IDENTITY can be used to generate a unique number for a given table and not across tables. That is the fundamental difference between both.

Though SEQUENCE can also be used to replace IDENTITY in the future, given the caching and faster retrieval, this is an option to explore and use. I would say, do a simple performance testing to know which one performs better. You will surely surprised :).

Finally

Your question in mind is – is it a replacement for IDENTITY? Well, it can be as one of the options. Do a small performance test to see which of these methods fair well for use before doing the replacement. I have seen acceptable performance with SEQUENCE usage on a decent workload – but you are better judge of the final results in your environments.

As time permits, feel free to pass me your comments.

Continue reading...


 

Upgrading to SQL Server 2012 May 17th, 2012

Vinod Kumar

As new versions of software get released, companies and products also get into this natural transition of upgrading and supporting the latest and greatest. And in the same space, SQL Server is no exception -almost as an predictable cycle, every 3 years we can expect a version of SQL Server getting released. This has been a trend since the SQL Server 2005 version for sure. As part of my job in meeting customers, I have to give recommendations and best practices to move to the latest versions. There are so many nuances to keep in mind as we make a successful upgrade. Let me take a moment to give you some of the suggestions I give to customers. Hope these will be helpful for you too.

Bible to start

You need to always start with preparation phase. This phase involves multiple steps and here are some of them for you to start.

  1. Keep the SQL Server 2012 Upgrade Technical Guide handy with you always. This is a very long document to read but it is worth the reference. I am outlining this as the FIRST reference because I will refrain from what this document covers as content. It is exhaustive and as I said with ~450 pages has a lot of material on how to upgrade. Also for reference SQL Server 2005 to 2008 Upgrade Whitepaper.
  2. If you still want to know WHY UPGRADE? I am attaching a link to some of the new features document for your reference. Knowing the new features is an important step before you upgrade. Later, you must be looking at how you can start maximizing the new features introduced with the new version.

Just upgrading by keeping the compatibility level at a older version is not real upgrade. Evaluate at using some of the new features to increase productivity, performance, maintainability and scalability.

As management takes the decisions for upgrade, DBA’s and developers need to also get themselves set for this task. Any upgrade process cannot be a simple overnight activity but more of a calculated informed step as line-of-business applications are at stake and it can bring down the efficiency of the teams working inside the organizations.

  1. Take a BACKUP of your databases. This is the first step irrespective of what you do next. A backup is required to make sure we have a fallback mechanism if things go wrong. Also make sure if you are using any specific system databases and objects in those DB, these are appropriately backed-up or scripted out.
    1. Take FULL Backups
    2. Transaction Log Backups
    3. Backup Database Master Keys
    4. Backup Service Master Keys
    5. Backup any external resource used (CLR DLLs, Extended SP) and anything that is required for the application outside the database for proper functioning.
  2. Before the backup process, just run a DBCC CHECKDB across the databases to make sure there is no corruption before the backup and they are consistent. This step can reduce any heartburns at a later point in time post upgrade :).
  3. As the upgrade talks happens, get started with tracking if any deprecated features are being used. There are multiple ways to do this – I highly recommend using the “SQL Server: Deprecated Features” Perfmon counter to track it like a background task. As your application runs, this gives you a rough idea of how many deprecated features you might be using inside your application that might have an impact to your upgrade process. There is also a Trace Event for capturing Deprecated features with SQL Server Profiler which you can use but that will be to nail down which code is causing that perfmon spike.
  4. Before initializing the upgrade process, please start by running the Upgrade Advisor as a first step. This surely guides you of potential problems upfront and is one of the essential step too for any SQL Server upgrade process.
  5. Keep track of the Supported Versions and Editions as you plan for Upgrades. With changes in licensing plans, LPIM (lock pages in memory), AWE gone etc. It is critical to keep track of how we upgrade between environments.
  6. Take a typical workload using profiler and keep it handy for a later test. Create a separate environment, bring the databases via a backup / restore operation to the new edition. Look for warnings, errors, changes in functionality and performance bottlenecks. Make sure to replay the workload collected before to see if anything unusual behaviors or errors. Document and then make sure these are mitigates during the actual run.
  7. Also, review the KB article that talks about “How to move databases between computers running SQL Server”. Especially, when we move between servers during upgrades we might need to change the location of files and hence have to account for the same. Also on a new setup, make sure the drivers for SAN or other hardware components are done properly and are the latest as suggested by the vendor.
    1. There are interesting ways to install as we move forward with SQL Server 2012 like, you can install SQL Server with SMB Fileshares as storage options.
    2. Another, is to install it on a Windows Server Core. This is interesting because now you don’t have an UI to manage SQL Server and it also reduces the surface area of attack and patch management is easy too.
    3. Considerations for SysPrep install for SQL Server – read documentation.
  8. Also, since this is an upgrade to a new box, make sure you are having enough space allocated for atleast 2-3 years of growth in mind for the database locations. Now is a good time for you to reassess some of the values defined for the auto-growth settings. Ideally, I suggest never allow SQL Server to grow but you pre-allocate it as part of your administration window cycle by properly monitoring SQL Server Data file free space.
  9. Before you install the Database on a new server, make a note of the Collation requirements for your applications. This is an important step and you don’t want get into a server which is Case-Sensitive while the application is built assuming case-insensitive.
  10. Don’t forget some of the critical addendum functions that you might have configured like replication, mirroring, clustering, service broker, maintenance plans, local user accounts, SQL Server Logins, service accounts, Linked servers, common backup path locations, configuration settings, DR strategies etc. All these need to be accounted and have to be changed accordingly.
  11. Whatever step you take, document them meticulously and run the document through a junior tester on a fresh SQL Server environment. This will test your upgrade process documentation and at the same time make sure that you are not missing any step to avoid last minute glitches. Document every single error encountered and the solution for the same – it is better to be prepared than to fail not being prepared.
  12. When I call out documentation, look out for hidden gems in your process like – Startup Processes, SQL Server Trace Flags that were enabled, Error Log settings, External DLL / CLR assemblies used, powershell scripts, any application Windows services that have been installed, external third party backup tools etc. Also make sure the server is secure by default and make sure any unnecessary services (SQL Browser, IIS, virus scanners, spooler etc) if running have been stopped.
  13. If you are planning to move into Virtualized environment, know what does it take to run SQL Server in Hyper-V Environment (though little outdated, is a worthy read). Also, read about running SQL Server with Dynamic Memory in HyperV Environments. I saw a session video in NA TechEd on this very topic which can be a good watch: Microsoft SQL Server Consolidation and Virtualization: Myths and Realities.
  14. More documents worthy for reading on the Virtualization subject are: Hyper-V is the Best Virtualization Solution for SQL Server, High Performance SQL Server Workloads on Hyper-V, Planning, Implementing, and Supporting SQL Server Virtualization with Windows Server 2008 R2 Hyper-V and Live Migration and Support Policy for SQL Server Virtualization.
  15. What are your plans for upgrade: in-place upgrade, side-by-side on the same box or side-by-side on a different box? Irrespective of the method used Keep in mind Point 1 :).
  16. Have a small test-stub workload where you are aware of the parameters and the expected results from the server. Make sure you run this pre and then post the upgrade and debug any anomalies based on the output. If you have automated test scripts, these are the best to be run against the server for a quick test with expected values.
  17. Have a failsafe strategy if things don’t go as per the plan. Your backup can come handy here and be careful especially if you plan to do a in-place upgrade as there is not much choice to come back other than using the backup route.

Know the fine prints

Please make sure you are aware of the specific call-outs mentioned in the Upgrade Technical reference mentioned above. During upgrade there can be specific features or settings that might need special attention. I am calling out some of them as I have seen at multiple customer upgrade process.

  1. The default setting for max worker threads in SQL Server 2000 is 255. When you upgrade from an instance of SQL Server 2000, the database engine keeps setting the value of max worker threads. However, it is recommended you change the value of max worker threads to 0 before the migration, to allow the Database Engine calculates the optimal number of threads.
  2. Changing the Compatibility Level while users are connected to the database can generate incorrect results for sets of active queries. To avoid inaccurate results, it is recommended to change the Compatibility Level with the database in order to a SINGLE_USER mode and then back to MULTI_USER. Do this post the upgrade activity.
  3. Look at server side settings from the older version, as you plan to move to new hardware’s some of these old values doesn’t make sense. Keep special note to MAXDOP settings, number of TempDB files etc.
  4. Just like the recommendation on Server Core, I highly recommend not to install any new components that you may not be planning to use in the immediate future like – Data Quality Services, PowerPivot, Master Data Services, Analysis Services etc.
  5. Look at the Security considerations and the protocols used for communication as part of upgrade. Each new version there are some changes and analyze the ports you have as standards open at the new environment. Security considerations during Install is documented and worth reading for every version upgrade.
  6. In earlier versions of SQL Server, the values ​​for the table and index row counts and page may be incorrect. Thus, databases that were created in versions prior to SQL Server 2005 may contain incorrect counts. After you upgrade a database from SQL Server 2000, it is recommended that you perform the DBCC UPDATEUSAGE to correct any invalid counts. This DBCC statement corrects the count of rows, used pages, reserved pages, leaf pages and data for each partition in a table or index.
  7. Update statistics for all Indexes after the Upgrade process. This is one of the most important step. Keep in mind also the Plan guides and hints that you have used in your application. Test these sections for any potential performance problems. Given the upgrade, SQL Server engine would have also changed and hence testing them matters. In case of Full-text catalogs, make sure these are also updated.
  8. As a sanity check, make sure to run a DBCC CHECKDB on each of the databases which have been part of the upgrade process.
  9. Over and above this, I also recommend checking each of the database properties which are critical. Check the recovery models, Auto-Close, Auto-Create-Statistics, Auto-Shrink, Auto-Update-Statistics and Auto-Update-statistics-Asynchronously. Also in this list, I highly recommend setting value of CHECKSUM for the Page Verify property at the DB properties level.
  10. Know and document the changes that need to be done on the application to start pointing to the new upgraded server. This might sometimes involve Connection String changes in Web Server deployment. Also, track if any reporting server and Sharepoint installations are pointing to the old servers as data sources. Even these need to be changed as part of movement.
  11. After all this, don’t forget to take a Backup of your system. You don’t want to do all the hard work and find out the HDD crashed the next day :). Backup and secure on a different media and location.

I think this very much sums-up what I had in mind based on various implementations that I have seen. These steps are very much towards migration of the SQL Server Relational engine but doesn’t talk about the SSAS, SSIS, SSRS workloads. I would again reiterate to look at the Upgrade Reference Guide mentioned above as a good guide to start.

A successful upgrade uses a cycle of: Plan, Document process, Test, Refine process, Test, Plan upgrade window, execute, verify upgrade and then opens for business.

Now that we have upgraded into the next version of SQL Server, now start looking actively at how we can start using the new capabilities available inside the next new release. I am just calling out some of them that are worth mentioning here for SQL Server 2012 release:

SQL Server: AlwaysOn Technologies

xVelocity in-memory technologies of PowerView and PowerPivot

BI Sematic Model

File Tables

SQL Server 2012: Express LocalDB

SQL Server ColumnStore Index

Data Quality Services

Security Enhancements – Contained Databases

Big Data Analytics and Hadoop Connectors

Microsoft SQL Server JDBC Driver 3.0

Microsoft Drivers 3.0 for SQL Server for PHP

Links worth keeping track as you upgrade:

Deprecated features: http://technet.microsoft.com/en-us/library/cc707789

Discontinued features: http://technet.microsoft.com/en-us/library/cc707782

Breaking changes in 2012: http://technet.microsoft.com/en-us/library/cc707784

Behavior changes in 2012: http://technet.microsoft.com/en-us/library/cc707785

Full-Text Search Backward Compatibility in SQL 2012: http://msdn.microsoft.com/en-us/library/ms143544

Hope these links and resources will be of help for you as you plan your upgrade and movement to yet another version of SQL Server. I would love to get your inputs and understand if there are any glaring steps that I would have missed mentioning as part of the checklist mentioned above.

Continue reading...


 

SQL Server 2012: ColumnStore Characteristics May 10th, 2012

Vinod Kumar

I was generally exploring for content to explain the nuances of using ColumnStore Index with SQL Server 2012. And interestingly during a conversation with Pinal, he had already written a number of blog posts on the same subject – fundamentals, ignoring columnstore, video demo of performance. Seeing this I thought maybe I want to write something different on the same subject and leave the basic learning to those posts as they make a good read.

Columnstore indexes group and store data for each column and then join all the columns to complete the whole index. These differ from traditional indexes that group and store data for each row and then join all the rows to complete the whole index. For some types of queries, the SQL Server query processor can take advantage of the columnstore layout to significantly improve query execution times. Columnstore indexes in SQL Server have key characteristics that differentiate it from a traditional row-based store. Let me list a few here, these are simple and good to know information:

  1. Just like the explanation I did above, unlike the traditional row-based organization of data (called rowstore format), in columnar database systems such as SQL Server with columnstore indexes, data is grouped and stored one column at a time. SQL Server query processing can take advantage of the new data layout and significantly improve query execution time. In this context you will hear the term xVelocity to describe the columnstore indexing engine.
  2. There is no concept of key columns in a columnstore index, so the limitation on the number of key columns in an index of 16 does not apply to columnstore indexes. But it cannot have more than 1024 columns.
  3. The index cannot be clustered index for columnstore. If a base table is a clustered index, all columns in the clustering key must be present in the non-clustered columnstore index. If a column in the clustering key is not listed in the create index statement, it will be added to the columnstore index automatically.
  4. It can neither be a primary key or foreign key and cannot have the INCLUDE keyword for adding columns.
  5. Columnstore indexes work with table partitioning. No change to the table partitioning syntax is required. A columnstore index on a partitioned table must be partition-aligned with the base table. Therefore, a non-clustered columnstore index can only be created on a partitioned table if the partitioning column is one of the columns in the columnstore index.
  6. The index key record size limitation of 900 bytes also does not apply to columnstore indexes. It cannot be unique Index and cannot include sparse columns. You cannot also include the ASC or DESC keyword as part of the index.
  7. Along with the columnstore index, SQL Server introduces batch processing to take advantage of the columnar orientation of the data. This can be seen in the Logical step of a Execution Plan called “Batch”.
  8. Only the columns needed must be read. Therefore, less data is read from disk to memory and later moved from memory to processor cache.
  9. Most queries do not touch all columns of the table to satisfy a query. Therefore, many columns will never be brought into memory. This, combined with excellent compression, improves buffer pool usage, which reduces total I/O.
  10. In this version of SQL Server 2012, the ColumnStore Index makes the table into ReadOnly mode. And it cannot be combined with Compression, replication, Change tracking, Change data capture and Filestreams.
    For Updates, there are always workarounds as discussed on MSDN.

DMV’s to keep track

sys.column_store_dictionaries: Contains a row for each column in a ColumnStore index.

sys.column_store_index_stats: Contains a row for each ColumnStore index.

Update: Seems like the this DMV will get deprecated, Pinal wrote about this.

sys.column_store_segments: Contains a row for each column in a ColumnStore index.

INDEXPROPERTY – Adds a property IsColumnstore to identify xVelocity columnstore.

This feature is strongly geared towards DW workloads and I had an opportunity to meet customers with TB’s of data to process in short time. And this feature was something they just loved using even given the restrictions. Some of the restrictions may be eliminated in future versions but this xVelocity concept is quite powerful to be ignored. If you do use this, drop a line !!!

Continue reading...


 

Access Denied: Run SSMS as Administrator May 4th, 2012

Vinod Kumar

I recently upgraded my machine to Windows 8 and there are so many changes that are worth mentioning as part of security enhancements for SQL Server 2012 and Windows 8 combinations. We will discuss in various blogs in the future, but here is one of the question I got from one of my ASP.NET developer that after he upgraded he started getting this below error for attaching an existing database.

image

Textually the error is something like:

TITLE: Microsoft SQL Server Management Studio
——————————

Attach database failed for Server ‘xxxxxx’.  (Microsoft.SqlServer.Smo)

——————————

Unable to open the physical file "C:\SQLSamples\AdventureWorks2008R2_Data.mdf". Operating system error 5: "5(Access is denied.)". (Microsoft SQL Server, Error: 5120)

——————————

Worst was, the link for help from SSMS was also not getting him anywhere. Here are the sequence of things that I asked:

  1. Which OS are you running Windows 7 or Windows 8? Ans: Windows 8
  2. Has UAC been turned off or on? Ans: Off (asked why … but that is a different discussion)
  3. Great, did you start SSMS in Administrator mode and try? Ans: Well, UAC is off right? Then why should I.
  4. Me: Trust me :)
  5. Ans: Woala, it is now working.

The solution is not specific to Windows 8 but can be on Windows 7 too, tough it will be interesting to see how we have made “Security by default” more restrictive on Windows 8. Simple solutions for simple tasks :)

Do tell me if you have encountered this. Hope this helps.

Continue reading...


 

SQL Server: Identify UnUsed Databases April 30th, 2012

Vinod Kumar

too-many

As part of my day job, I solve lot of interesting customer problems and recently one of our customers had an interesting question to stump me. It got me really thinking hard and I wanted to share some of the solutions I shared with them. Though these solutions were given, want to also see if you have a better way to solve this question.

The Enterprise I was talking to was on more than 70+ SQL Server server installations and had close to 500+ databases running in their datacenters for various businesses. The CIO asked the datacenter administrator how many of these databases are actually used by the business and how many were stale and not used but just there because of legacy reasons. And as a database expert in the room the datacenter Administrator just turned to me and asked – “Why don’t you answer this Vinod?”

Simple solution

For every complex problems there always is a simple solution in hand. I just thought over and gave them a method almost that was convincing enough for the team. My suggestion was simple, just query the index_usage_stats DMV for some stats and then use the same to decide if the database was used or not. I just gave a simple script to the DBA like below:

Sp_msforeachdb 

‘Select @@Servername As Servername, ”?” AS DBName,

IsSystemDatabase,     

MAX(last_user_lookup) last_user_lookup,      

MAX(last_user_scan) last_user_scan,      

MAX(last_user_seek) last_user_seek,      

MAX(last_user_update) last_user_update

from [?].sys.dm_db_index_usage_stats

INNER JOIN  (      

SELECT dtb.name,

CAST(case

when dtb.name in (”master”,”model”,”msdb”,”tempdb”)

then 1

else dtb.is_distributor end AS bit) IsSystemDatabase      

FROM master.sys.databases AS dtb ) systemdatabases

ON systemdatabases.name = ”?”

GROUP BY IsSystemDatabase’ 

At least this is giving you a great starting point for discussion with the stakeholders that the database has not been accessed for the past 5+ months and if there is still business justification. The Administrator did call me back after a month that this script (simple yet powerful) was really useful. Thought to share with you all today.

PS: Remember, the statistics values can get reset if SQL Server is restarted hence this is not 100% foolproof implementation though.

What are the other options?

There are obviously other solutions that can be tried out:

  1. Auditing is one of the most precise and obviously a deterministic solution.
  2. Can even use Logon-triggers (but remember the user can switch context of the database hence will be difficult).
  3. The above Usage_Stats is great for databases of version higher than SQL Server 2005+. BTW, are you still running SQL Server 2000 versions?

Tell me your story and options to answer such situations. Would love to learn from you too.

Continue reading...


 

SQL Server 2012: Offline Logs viewing April 23rd, 2012

Vinod Kumar

As part of WMI events in SQL Server 2012, I saw these two new additions for offline error log files:

SqlErrorLogFile: http://msdn.microsoft.com/en-us/library/ff963576.aspx

SqlErrorLogEvent: http://msdn.microsoft.com/en-us/library/ff963582.aspx

Suddenly this got my attention and this blog post gets inspired because of the above events. These WMI classes are used inside SSMS’s Log Viewer when SQL Server is offline or is not responding. If the Log Viewer determines that the connection to a SQL Server instance could not be established, it will fall back to using WMI to enumerate and read the SQL error logs.

Since WMI classes are used for log viewing, it requires the WMI Service be started and only privileged Windows users with access to SQL Server logs can view the log data. By default the privileges are given only to the Administrators group of the machine by the SQL Server setup. As a result, SSMS must be launched using the Run as administrator option.

Offline Log Viewer can read error logs from SQL Server 2005, SQL Server 2008, SQL Server 2008 R2 and SQL Server 2012 versions.

How to use SSMS

With Microsoft SQL Server 2012, you can view SQL Server log files from a local or remote instance of SQL Server when the target instance is offline or cannot start. Steps to get this is:

  1. Start by opening SQL Server Management Studio (SSMS) as an Administrator.
  2. Register a SQL Server 2012 instance from Registered Servers (more about Registered Server on MSDN)
  3. Make sure the instance is stopped or offline.
  4. Once the SQL Server instance is registered, right-clicking on a registered SQL Server instance displays the following context menu to “View SQL Server Log
    image
  5. The SQL Server ErrorLog is displayed like it normally would with an online instance.

Some of the things to consider before using the offline viewing capability are:

  1. The instance of SQL Server that you want to connect to should be registered in Registered Servers.
  2. The account that you use to connect must have the required permissions.

Final words

I found this feature quite helpful and for DBA’s this will be a great addition. You can access the offline log files from Registered Servers in SSMS (as in example above) or programmatically through WMI with WQL (WMI Query Language) queries. PowerShell can also leverage these WMI components to programmatically read the error log files. The other methods for access will be for a different blog post.

Feel free to drop your comments.

Continue reading...


 

Reliving #TechEdIn at Blr UG April 3rd, 2012

Vinod Kumar

Reliving TechEd 2012 with Vinod Kumar 1TechEd India 2012 has been an event to remember and we take great pride in bringing this event every year from Microsoft. As I posted before on my other blog post around recap it takes a lot of preparation and execution to do sessions at the TechEd. This time I was tweeting almost live at the event using the #TechEdIn tag and everyone following me will vouch for the same. Having said that, there are many out there who really want to relive the experience of TechEd again. So here we go again – I will be doing a session on April 1st (Saturday).

BDotNet.in and BITPro.in Brings you

The Bangalore Usergroups are a bunch of bubbling individuals who bring so much joy to the members by conducting regular Usergroup meets. If you are in Bangalore then I strongly urge you to join these groups to get an update on an constant basis.

  1. BDotNet.in – Bangalore .NET Usergroup
  2. BITPro.in – Bangalore ITPro Usergroup
  3. SQLBangalore - Bangalore SQL User Group

Not strange, being in this city for close to 12 years, I have had the opportunity to talk and be part of both the UG’s since they started. Still nostalgic !!!

Reliving TechEd Sessions

TechEd2012Switching gears, yes – if you want to hear to my sessions delivered at #TechEdIn now in a much more relaxed manner. This is the chance again. For folks who are wondering when it will happen in their UG – don’t worry. If I travel to your town, I promise to do them for sure. This UG meeting we plan to have 2 sessions (back-to-back) on Internals.

1. Understanding Windows with SysInternals Tools
2. Peeling the Onion: SQL Server Internals Demystified

I can assure you one thing for sure – it will be a learning and sharing experience for me in a free flow format :).

Feel free to comment if you want any specific area to be covered as part of the UG event. I will be more than happy to expand the scope to include them :). For it is reliving the experience and get great content – right?

Venue: Microsoft Corporation, Krishna Hall, Embassy Building, 3rd Floor, Domlur.

Register at our FB page : http://www.facebook.com/#!/events/139444029517882/

PS: Feel free to walk in too – it is your group and the session is completely FREE. Entry on a first-come-first-serve basis only.

Community talk: Manas Blogged on this event. Wonderful Community Networking opportunity. Track us via #ReLiveTechEdIn over twitter.

Continue reading...