SQL Server Internals Explained: An Introductory Guide

Understanding SQL Server's internal architecture and components, such as the Relational Engine, Storage Engine, indexes, and locking mechanisms, helps database administrators (DBAs) and developers optimise performance, troubleshoot issues, and ensure efficient data management. Each part of SQL Server, from how data is stored to how queries are processed, contributes to its overall efficiency and scalability. By diving deeper into SQL Server internals, one can make informed decisions to improve database performance and stability.

Here’s an overview of some of the key internal components and concepts of SQL Server:

Some common reasons why the transaction log cannot be reused automatically

Managing SQL Server databases may sometimes be challenging especially when it comes to dealing with transaction log file internal space usage. More specifically, today I am going to talk about SQL Server wait types that may prevent SQL Server from automatically reusing transaction log internal space and therefore resulting in running out of space and affecting detrimentally on the availability of the database. For instance, if the recovery model of the database is either Full or Bulk-Logged then you may see a LOG_BACKUP wait type while the next backup log is waited to be run. Consequently, if those databases do not have Log Backups tasks, their transaction log files will be growing without control and when they take all the disk space available the databases will stop working until the internal space of the transaction logs is truncated by executing log backups manually.

Boosting SQL Server Efficiency: Why You Should Update Statistics Manually

While SQL Server provides automatic statistics management, manually updating statistics allows for greater control over database performance. Regularly monitoring the state of your statistics and updating them as necessary can lead to more efficient query execution, especially in environments with dynamic data and high transaction volumes. By implementing a proactive strategy for managing statistics—whether through scheduled jobs, maintenance plans, or manual updates—you can ensure that your SQL Server instance continues to perform optimally over time.

Detailed Explanation of SQL Server Query Hints

SQL Server query hints are directives that can be included in your SQL queries to guide the SQL Server Query Optimiser in executing them. Typically, SQL Server automatically selects the most efficient execution plan based on the query and underlying data. However, query hints enable you to override the optimiser’s default choices, giving you greater control over specific aspects of query execution. 

While these hints can significantly enhance performance in particular scenarios, they must be used cautiously and strategically. Incorrect application of hints can lead to degraded performance or unpredictable outcomes. Therefore, it is essential to thoroughly test your queries both with and without hints to evaluate their impact. Ensure that hints are employed appropriately and suited to your specific workloads to achieve optimal results.

Below is a detailed look at common SQL Server query hints, examples of their usage, and the potential risks associated with each.

Understanding the SQL Server Query Optimiser: A Comprehensive Overview

The SQL Server Query Optimiser is a crucial component of Microsoft SQL Server, responsible for determining the most efficient way to execute a given query. The optimiser takes into consideration various factors, including the structure of the query, available indexes, statistics about the data, and the overall database schema. Here is a detailed breakdown of its functionalities, processes, and related concepts. 

1. Purpose of the Query Optimiser

The primary goals of the Query Optimiser include:

• Generating Execution Plans: The optimiser creates a roadmap that describes how SQL Server will access the data required by the query.
• Maximising Efficiency: By evaluating different potential execution paths, the optimiser selects the one that minimises resource utilisation, such as CPU, memory, and disk I/O.
• Improving Response Times: A well-optimised query results in faster execution, contributing to better overall system performance.

Detecting Poor Cursor Usage in SQL Server

Undoubtedly, poor cursor usage is a critical issue to review in order to mitigate the risk of CPU bottlenecks and to assess whether cursors are the most appropriate means for processing or if set-based operations would be more effective. It is well known that set-based operations are generally more efficient; however, if you choose to use cursors, you must ensure that they do not adversely affect the performance of the database.

Looking deeper into the physical & logical architecture - Transaction Log File

Beyond all doubt, it is essential to have a good understanding of the Transaction Log (T-Log) so that we can diagnose unforeseen performance issues related to it and I am sure that almost everyone had at least one. The T-Log is basically a record of all transactions happening to the database. All these transactions are actually first written to the physical T-Log file, and then after a CHECKPOINT, is written to the Data File via the Lazy Writer process. Some of the uses of T-Log are: as a point in time recovery (full recovery model), to record the Start and End of each transaction, every data modification (insert, update, delete) including system SP's, DDL statements to any table including system tables, every extent and page allocation and de-allocation operation, and creation or drop of tables and indexes.

Getting Important Information of SQL Server Backups

Regardless of the method employed to take SQL Server backups, they must be carefully monitored around the clock, particularly in critical database environments where data changes rapidly. Keeping backups current is vital, as they can save considerable time when it comes to recovering databases in the event of unforeseen incidents. Furthermore, it has been disappointing to observe numerous environments lacking a backup strategy, often with no one designated to oversee them. In this context, it is evident that these businesses were at significant risk of losing vast amounts of data and incurring substantial financial losses.

Sometimes, there is a belief that backing up the entire virtual machine is sufficient; however, this type of backup serves a different purpose than a dedicated database backup. Consequently, SQL Server database backups cannot be substituted with virtual machine backups, which can be time-consuming, inadequate, and impractical for databases. Thus, implementing an appropriate database backup strategy (for example, with Full + Differential + Log Backups) and closely monitoring them are essential tasks for ensuring the recovery process of databases.

AI-Driven SQL Server Database Management Service with Security in Mind

In the realm of SQL Server database management, the integration of artificial intelligence is transforming how organisations optimise their databases. AI enhances performance, automates processes, and significantly improves security. Below are specific applications of AI in SQL Server management and the benefits they offer.

Key Features and Benefits

• Automation and Efficiency:
• Intelligent Performance Monitoring: AI algorithms continuously track SQL Server performance metrics—such as CPU usage, memory consumption, and disk I/O. By identifying anomalies (e.g., slow query response times) before they affect users, SQL Server’s Intelligent Query Processing optimises performance automatically, reducing the need for manual intervention​
• Automated Backups and Restores: With AI, SQL Server can automate backup and restore processes, ensuring they occur without manual oversight. Features like SQL Server Managed Backup utilise AI to manage backups based on workload, guaranteeing data protection​
• Enhanced Security Measures:
• Advanced Threat Detection: SQL Server's Advanced Threat Protection leverages machine learning to monitor for suspicious activities. It can flag unusual login attempts or data access patterns, allowing for rapid responses to potential security threats​
• Data Encryption and Compliance: AI tools facilitate the management of Transparent Data Encryption (TDE) and Always Encrypted features, ensuring sensitive data is encrypted at rest and in transit. This is vital for compliance with regulations like GDPR​
• Scalability and Flexibility:
• Adaptive Resource Management: AI-driven systems in SQL Server can automatically adjust resource allocation based on workload demands. SQL Server’s Automatic Tuning feature uses AI to optimise memory and CPU allocations, ensuring performance during peak usage​
• Cloud Integration: In cloud environments, SQL Server’s AI capabilities facilitate seamless scaling of resources. Features like Azure SQL Database offer built-in intelligence to adjust resources based on usage patterns, enhancing cost and performance management​
• Cost-Effectiveness:
• Reduced Operational Costs: Automating routine tasks such as indexing, query optimisation, and monitoring with AI minimises the need for extensive manual intervention. This approach saves time for database administrators and cuts operational costs​

Applications of AI in SQL Server Database Management

Applying AI to SQL Server database management allows organisations to:

• Predictive Maintenance: AI can forecast potential database failures or performance degradation, enabling proactive measures to avoid downtime. Machine learning models can analyse historical performance data to identify patterns indicative of future issues​
• Query Optimization: AI algorithms can analyse query performance and suggest optimisations based on historical execution patterns. This can significantly enhance application responsiveness and reduce resource consumption​
• Anomaly Detection: By employing AI-driven analytics, SQL Server can detect anomalies in database behaviour, such as spikes in traffic or unusual data modifications, which could indicate security breaches or performance issues. This allows for timely interventions​
• User Behaviour Analytics: AI can analyse user interactions with the database, helping to optimise performance based on actual usage patterns. This understanding can drive further enhancements in user experience and resource allocation​

These resources provide a detailed overview of how AI is transforming SQL Server database management by automating performance tuning, enhancing security, and improving overall database reliability.

• Microsoft's Blog on AI Tuning in Azure SQL Databases: this blog discusses how AI is integrated into Azure SQL Database through features like automatic tuning. AI continuously monitors workloads, identifies performance bottlenecks, and applies tuning actions autonomously, improving performance without human intervention. It offers real-world examples of how businesses have benefited from these features. Learn more here​.
• The Role of AI in SQL Server Database Administration: this resource outlines how AI revolutionizes the role of a SQL Server DBA. Key areas include automated performance tuning, predictive analysis for capacity planning, and anomaly detection for early problem resolution. It also covers AI’s role in index management and security. Learn more​ here.
• Database Trends: How AI Is Impacting DBAs: this article explains the transformative role of AI in DBA tasks, such as automating routine maintenance, predictive analysis for performance optimization, and security enhancements through anomaly detection. AI allows DBAs to focus on more strategic activities. Read the full article here.​

Integrating an AI-driven SQL Server database management service not only enhances operational efficiency and performance but also fortifies data security against evolving threats. As organisations increasingly depend on SQL databases for critical functions, leveraging AI capabilities becomes essential for maintaining a competitive edge and ensuring data integrity.

New Features in SQL Server 2022 for Developers

SQL Server 2022 brings a comprehensive suite of features designed to enhance the development experience, improve performance, and strengthen security. By leveraging these capabilities, developers can create more efficient, scalable, and secure applications, positioning their organisations to effectively handle modern data challenges and requirements. The focus on intelligent processing, machine learning, data integration, and robust security aligns with the evolving landscape of application development and data management.

Here’s a detailed overview of the new features in SQL Server 2022 tailored for developers.

1. Intelligent Query Processing Enhancements

Overview: SQL Server 2022 continues to build on the Intelligent Query Processing (IQP) capabilities introduced in previous versions. The aim is to enhance query performance automatically without requiring significant code changes. This means that developers can achieve better performance for their applications without extensive rewrites.

Key Feature:

• Parameter Sensitive Plan Optimisation: This feature allows SQL Server to create and maintain multiple execution plans for the same query based on different parameter values. It identifies the most efficient plan based on runtime parameter values rather than relying solely on the static plan generated during compilation.

Example Scenario: Consider a financial application that retrieves transaction records based on different filters, such as dates and account numbers. With parameter-sensitive plan optimization, SQL Server will adaptively select the best execution plan based on the most commonly accessed account numbers or date ranges, leading to reduced execution times and better resource utilisation.

2. Built-in Machine Learning Services

Overview: SQL Server 2022 provides built-in support for machine learning services directly within the database engine. This integration allows developers to run R and Python scripts without the need for external tools.

Key Features:

• Direct Data Access: R and Python scripts can access SQL Server data directly, allowing for real-time analysis without the need to export data to another platform.
• Model Deployment: Developers can train and deploy machine learning models directly within SQL Server, making it easier to use these models in production applications.
• Scalability: Since machine learning is performed on the SQL Server instance, it can take advantage of the existing hardware resources, which may lead to improved performance for large datasets.

Example Scenario: A marketing department might use machine learning to predict customer churn. By training a model directly within SQL Server, they can continuously feed it fresh data from their transactional systems, enabling real-time predictions that can drive immediate business actions, such as targeted retention strategies.

3. SQL Server Ledger

Overview: SQL Server Ledger introduces a new feature that provides blockchain-like functionality, enabling developers to build applications with strong data integrity and immutability. This is particularly useful for industries that require stringent audit trails and data verification.

Key Features:

• Immutable Ledger Tables: These tables track all transactions in a way that prevents any modifications after they are committed, creating an audit trail that can be independently verified.
• Cryptographic Verification: Each transaction is cryptographically hashed, and the hashes are stored in the ledger, ensuring the integrity and authenticity of the data over time.
• Integration with Existing Systems: Developers can integrate ledger functionality with existing applications seamlessly, making it easier to adopt this technology without major architectural changes.

Example Scenario: In a healthcare application, patient treatment records can be stored in a ledger table. Each update to a patient’s treatment plan is recorded in a way that cannot be altered, providing a verifiable history that meets regulatory compliance and can be audited as needed.

4. Enhancements to Query Store

Overview: The Query Store feature is designed to help developers manage query performance by providing historical data on query execution, plan choices, and runtime statistics. The enhancements in SQL Server 2022 provide deeper insights and improved capabilities for tracking query performance over time.

Key Features:

• Query Performance Insights: The enhancements include improved reporting capabilities, allowing developers to visualize query performance trends over time and quickly identify any regressions.
• Automatic Plan Correction: SQL Server can automatically identify when a query's performance has degraded and revert to a previously optimal execution plan, reducing the need for manual intervention.
• Improved Visibility into Query Execution: Developers can see detailed information about query execution times, resource consumption, and plan changes, making it easier to diagnose performance issues.

Example Scenario: A web application might experience slowdowns after a new feature is deployed. Developers can use the Query Store to identify which queries have started to perform poorly, understand the reason behind the changes, and either optimise the queries or revert to a better-performing plan.

5. Azure Synapse Link for SQL Server

Overview: Azure Synapse Link provides a seamless integration between on-premises SQL Server databases and Azure Synapse Analytics, enabling developers to perform analytics on operational data without the need to move data to the cloud.

Key Features:

• Near Real-Time Analytics: Developers can run analytics on data as it is ingested, allowing for timely insights that can influence business decisions.
• Simplified Data Pipelines: This integration allows for the creation of streamlined data pipelines, enabling developers to focus on analysis rather than data movement.
• Enhanced Reporting: Azure Synapse provides powerful analytical capabilities that can be directly applied to operational data, enhancing reporting and decision-making.

Example Scenario: An e-commerce company can use Azure Synapse Link to analyse customer behaviour in real-time. By connecting their on-premises SQL Server to Azure Synapse, they can quickly generate reports on customer purchasing patterns, allowing for immediate marketing adjustments based on current trends.

6. Enhanced Security Features

Overview: SQL Server 2022 introduces several new security enhancements designed to protect sensitive data and improve overall database security posture.

Key Features:

• Always Encrypted with Secure Enclaves: This enhancement allows certain operations to be performed on encrypted data without exposing it in plaintext, enhancing the security of sensitive information.
• Dynamic Data Masking Enhancements: These improvements offer developers more options for controlling how sensitive data is presented to users, allowing for greater flexibility in displaying data according to user roles.
• Security Monitoring Tools: New tools provide real-time alerts and recommendations for improving database security, helping developers maintain compliance with best practices.

Example Scenario: In a banking application, sensitive information such as customer account details can be protected using Always Encrypted. Developers can configure dynamic data masking to ensure that only authorised personnel can view sensitive data, thus enhancing customer trust and compliance with regulations.

7. SQL Server Management Studio (SSMS) Enhancements

Overview: The latest version of SQL Server Management Studio includes significant enhancements designed to improve the developer experience, making it easier to manage and develop SQL Server databases.

Key Features:

• Improved IntelliSense: Enhanced IntelliSense capabilities help developers write SQL queries more efficiently by providing better auto-completion and suggestions.
• Integrated Monitoring Tools: New monitoring features allow developers to track database performance metrics directly within SSMS, enabling faster troubleshooting and optimisation.
• User-Friendly Interface: A more intuitive user interface makes it easier for developers to navigate complex database structures and management tasks.

Example Scenario: A developer working on a complex SQL query can rely on the improved IntelliSense features to quickly find the right syntax and suggestions, reducing errors and speeding up the query development process.

8. Support for UTF-8 Encoding

Overview: With the introduction of UTF-8 encoding support, SQL Server 2022 allows developers to store and manipulate multilingual data more effectively, which is crucial for global applications.

Key Features:

• Flexible Data Storage: Developers can specify UTF-8 encoding when creating string columns, enabling efficient storage of various character sets.
• Reduced Storage Requirements: By using UTF-8 encoding, developers can save space compared to UTF-16 encoding, particularly for texts primarily in English or other Latin-based languages.

That's all for now.

New Features in SQL Server 2022 for DBAs

SQL Server 2022 brings a wealth of new features that empower DBAs to enhance database performance, security, and usability. By leveraging these capabilities, organizations can optimise their data management strategies, ensuring better compliance, improved efficiency, and advanced analytics capabilities. The introduction of intelligent query processing, enhanced security features, and support for serverless architectures positions SQL Server 2022 as a robust choice for modern data solutions.

Here’s a summary of the key features and their benefits:

1. Intelligent Query Processing Enhancements

Overview: SQL Server 2022 expands on the Intelligent Query Processing (IQP) capabilities introduced in SQL Server 2019. New features include:

• Parameter Sensitive Plan Optimization: This feature allows SQL Server to create multiple execution plans for a single query based on the specific parameter values used during execution, leading to better performance.

Case Scenario: In a scenario where a stored procedure is called with highly variable input parameters (e.g., a sales report for different regions), the parameter-sensitive optimisation ensures that SQL Server selects the best execution plan tailored to the input, reducing execution time and resource consumption.

2. SQL Server Ledger

Overview: The SQL Server Ledger feature provides blockchain-like capabilities, ensuring data integrity and immutability through ledger tables. This feature uses cryptographic methods to verify data integrity, making it ideal for applications requiring audit trails.

Case Scenario: For financial institutions that need to maintain accurate records of transactions, SQL Server Ledger enables the creation of a ledger table where each transaction is logged. Any modifications are tracked, ensuring an immutable record that can be audited for compliance purposes.

3. Enhanced Security Features

Overview: SQL Server 2022 includes several security enhancements:

• Always Encrypted with Secure Enclaves: This allows more operations to be performed on encrypted data without exposing it in plaintext, enhancing security for sensitive information.
• Dynamic Data Masking Enhancements: Provides more flexible masking options, allowing DBAs to control how sensitive data is displayed to different users.

Case Scenario: In a healthcare application, patient data can be stored securely using Always Encrypted. The application can perform queries and calculations on encrypted data without revealing sensitive information to unauthorised users.

4. Query Store Enhancements

Overview: The Query Store feature has been enhanced to provide deeper insights into query performance over time. It now includes capabilities for identifying and managing query performance regressions.

Case Scenario: DBAs can utilise the enhanced Query Store to track performance changes after application deployments. If a new release introduces slow-running queries, the Query Store can help identify regressions, allowing quick remediation by reverting to previously optimised execution plans.

5. Built-in Machine Learning Services

Overview: SQL Server 2022 integrates built-in support for machine learning services, allowing DBAs to run R and Python scripts directly in the database engine. This feature enables data scientists and analysts to perform advanced analytics without moving data outside the SQL Server environment.

Case Scenario: A retail company can analyse customer purchasing patterns using machine learning algorithms stored in SQL Server. DBAs can schedule these analyses to run during off-peak hours, optimizing resource usage while delivering timely insights.

6. Serverless SQL Database

Overview: SQL Server 2022 introduces serverless capabilities for databases, allowing automatic scaling and cost-effective resource usage based on demand. This feature is ideal for workloads that experience variable usage patterns.

Case Scenario: A startup that experiences fluctuating traffic on its web application can benefit from serverless SQL databases, which automatically scale up during peak traffic and scale down during low usage periods, ensuring cost-efficiency.

7. Enhanced Availability Groups

Overview: New features for Always On Availability Groups improve failover performance and management. This includes automatic failover for databases in availability groups without the need for manual intervention.

Case Scenario: In a mission-critical application, the DBA can configure availability groups with automatic failover capabilities, ensuring minimal downtime during server maintenance or unexpected failures, thus enhancing application reliability.

8. Improved Performance Insights and Monitoring Tools

Overview: SQL Server 2022 provides enhanced monitoring tools with better insights into system performance. Features like the Database Health Monitor offer real-time health checks and performance tuning recommendations.

Case Scenario: DBAs can utilize the Database Health Monitor to receive alerts and recommendations for performance tuning based on workload analysis, enabling proactive management and optimization of database performance.

9. Support for JSON and XML Enhancements

Overview: SQL Server 2022 introduces improvements to support for JSON and XML data types, including better indexing and querying capabilities.

Case Scenario: A content management system that relies on JSON documents can leverage the new indexing features to improve query performance on JSON fields, allowing faster data retrieval and reporting.

That's all for now.

Knowing and Protecting Your Data

Computer circuitry has crept into nearly everything we use nowadays, and almost all of which gather information from us and about us. As a result, the present is immersed in copious amounts of data, which is stored somewhere and, most of the time, not securely protected. 

Today we are almost entirely reliant on IT departments at work; practically all organisations depend on enterprise-wide applications to support numerous key business processes which create a mountain of information. Needless to say, IT infrastructure and business applications are increasingly — and usually, unnecessarily — more complex; complexity is the worst enemy of security — and sometimes, of performance too. As a consequence, we are losing more control of storage and, therefore, security. The more data we share, the bigger the security risk is. 

Companies always strive to successfully harness the power of data, but unfortunately, this is not the case when it comes to securely accessing data. There are laws about protecting information, but most people are either blissfully unaware of them or careless about them. Many of us know what could happen when our data end up in the wrong hands.

Don't Place the Blame on SQL Server

I have never worked for Microsoft, but SQL Server has given me a lot in terms of learning, community and opportunities, all these together have helped me do a great job as a Database Administrator (DBA) for many years. Since I started working with SQL Server nearly 15 years ago, I have heard a lot of complaints about SQL Server being nowhere near as good as Oracle. Much as I would have liked to ignore these fruitless discussions, I couldn't see the point of comparing products in such a compulsive way. Is it not true that our skills are more crucial than the technology itself? — Or perhaps some people just try to find something to blame. Whatever the case, I am convinced that we, as database professionals, are compelled to make the most out of any specific database technology. 

No matter what technology we are working with, we are at the wheel — technology is just a tool — so it is not the best to blame technology on the ground of one's inefficiency.

Installing a stand-alone SQL Server 2017 instance step by step

Undoubtedly, many of us have the task of installing a new stand-alone SQL Server instance which includes the database engine service only. For instance, it can primarily be needed for dedicated and consolidated OLTP environments. Consequently, we can be asked to create a formal document for others so that they can easily follow it for future installations and standard configurations.

Today's post is going to outline the process of installing a basic stand-alone SQL Server 2017 instance. This process is just a basic guideline and, surely, not a rule for each installation, because it is fully understood that every environment is different and needs a customised installation to meet very specific requirements. You can read the whole tip about it at mssqltips here https://www.mssqltips.com/sqlservertip/5616/steps-to-install-a-standalone-sql-server-2017-instance. I hope you find it very useful and practical. That's all for now. Please let me know any remarks you may have. Stay tuned!

Configuring Read-Only Routing and load-balancing across Read-Only replicas

With the arrival of AlwaysOn Availability Group in SQL Server 2012, implementing HA+DR solutions have been an easier and not expensive task in comparison to legacy architectures such as Database Mirroring for HA and Log Shipping for DR, and FCI for HA and Database Mirroring for DR. Nevertheless, at the beginning not everyone has been fully aware of all the power of this technology so that some might not have made the most out of it. Naturally, this technology has been improved over the years, for instance, load-balancing across readable secondary replicas was added, and today in this post, I am coming with a script to configure it.

Quickly Checking for Stale Statistics in SQL Server

Monitoring the state of database statistics is crucial because they significantly influence overall performance. When statistics become outdated, performance degradation is inevitable, often leading to slow and inefficient systems—something nobody wants. This raises several important questions:

• How often have you faced unforeseen performance issues, even after completing your regular database maintenance tasks?
• Have you wondered why tasks like index rebuilding and statistics updates are sometimes insufficient?
• If everything was running smoothly before, why has performance suddenly declined?
• Why are indexes not being utilised as expected? Should more indexes be created?

The answer to many of these questions often lies in stale statistics. It's essential to remember that statistics contain vital information that the SQL Optimiser uses to generate optimal execution plans for queries. If these statistics are outdated, the impact on performance can be severe. Simply having the right indexes isn’t enough if the statistics are stale.

Avoid changing default ANSI database options

Not having another way of fixing some specific errors, at times some people may consider turning off any ANSI database options as a final solution. To be perfectly honest, I do recommend getting to the bottom of each problem and then fixing it at that level instead of changing default ANSI settings (unless it is just a quick fix or is truly necessary because you verified the benefits are significant). For instance, it is by no means uncommon to turn ANSI_WARRINGS off to fix the following error:

Msg 8152, Level 16, State 14, Line 5
String or binary data would be truncated.

How to make uniform all the collations of table columns for all databases

Clearly, it is of paramount importance to standardise the collations for all databases in a SQL Server instance in order to avoid dealing with unforeseen conflicts of page code compatibility. We may find ourselves in complicated situations because databases with different collations were migrated from other environments and the new consolidated environment was not prepared to host those new databases, but in one way or another we may need to consolidate them in only one server. It may be compounded by the fact that there may be many character columns of a database using different collations and another collation at database level. What’s more, the tempdb database may be using another different collation.  So, it may turn out to be not only a complex issue but also time-consuming.
To give you just an example, we can start finding out what character columns are using different collations from SQL_Latin1_General_CP1_CI_AS that we need to change in order to make everything uniform. I am going to display a useful script to do it. In this example I am assuming that we want to use SQL_Latin1_General_CP1_CI_AS for all objects in the database server.

EXEC sp_MSforeachdb '
USE [?]
select db_name(),c.name
from sys.columns c
inner join sys.types t on t.user_type_id= c.user_type_id
inner join sys.tables tb on  c.object_id=tb.object_id
where c.collation_name is not null 
and t.is_user_defined=0 and tb.is_ms_shipped=0 and tb.name<>''sysdiagrams''
and c.collation_name<>''SQL_Latin1_General_CP1_CI_AS''
order by tb.name, c.column_id'

After that,we can make the decision of changing all character columns for all databases and use SQL_Latin1_General_CP1_CI_AS, and then make that change at database level. Be cautious and make sure as well you have tempdb’s collation set SQL_Latin1_General_CP1_CI_AS.

EXEC sp_MSforeachdb '
USE [?]
if db_name() not in (''master'',''tempdb'',''msdb'',''model'')
begin
    select replace( REPLACE( ''ALTER TABLE '' + QUOTENAME(SCHEMA_NAME(tb.schema_id)) + ''.'' 
    + QUOTENAME(tb.name) + '' ALTER COLUMN '' + QUOTENAME(c.name) +  '' '' 
    + QUOTENAME(t.name) + ''('' + CAST( case when T.NAME=''NVARCHAR'' THEN  c.max_length/2 
      WHEN  T.NAME=''NCHAR'' THEN  c.max_length/2 ELSE c.max_length  END  AS VARCHAR(10)) +'')''  
    + '' COLLATE SQL_Latin1_General_CP1_CI_AS'' + CASE WHEN c.is_nullable =1 THEN '' NULL '' 
     else '' NOT NULL ;'' END, ''-1'', ''MAX'' ), ''[text](16)'', ''[varchar](max)'') as cmd
    INTO #TblTMP
    from sys.columns c
    inner join sys.types t on t.user_type_id= c.user_type_id
    inner join sys.tables tb on  c.object_id=tb.object_id
    where c.collation_name is not null 
    and t.is_user_defined=0 and tb.is_ms_shipped=0 and tb.name<>''sysdiagrams''
    and c.collation_name<>''SQL_Latin1_General_CP1_CI_AS''
    order by tb.name, c.column_id
    
  declare @cmd varchar(max)
  declare c_cmd cursor for 
     select cmd from  #TblTMP
  open c_cmd
  fetch next from c_cmd into @cmd
  while (@@fetch_status=0)
  begin    
    exec( @cmd)
    fetch next from c_cmd into @cmd
  end
  close c_cmd
  deallocate c_cmd
  drop table #TblTMP
end'

It is worth noting that while running the script above some errors may arise because of some indexes might be using one of the columns we are trying to alter. So, in this likely event, it is recommendable to drop those indexes and then run the script again. That is all for the time being. Let me know any remarks you may have.

MSSQL_ENG003165: An error was encountered while replication was being restored/removed. The database has been left offline

While restoring a replicated database without KEEP_REPLICATION option, SQL Server will remove replication settings by executing sp_restoredbreplication at the end of the process. The 'sp_restoredbreplication' system stored procedure will delete all replication metadata, that is, deletion of 'tr_MStran_alterschemaonly', 'tr_MStran_altertable', 'tr_MStran_altertrigger' and 'tr_MStran_alterview' tiggers (which were created to validate alterations on the replication of tables, triggers, views), disable user tables for replication, and deletion of subscription/publications/articles. Nevertheless, there might be some cases where 'sp_restoredbreplication' cannot be executed successfully and ends up leaving the database OFFLINE. I personally experienced that case and the error was something like this:

Msg 3165, Level 16, State 1, Line 1
Database ‘MyDB’ was restored, however an error was encountered while replication was being restored/removed. The database has been left offline. See the topic MSSQL_ENG003165 in SQL Server Books Online.
Msg 3167, Level 16, State 1, Line 1
RESTORE could not start database ‘MyDB’.
Msg 3013, Level 16, State 1, Line 1
RESTORE DATABASE is terminating abnormally.

Looking into this case, I could see that the cause was a DDL database trigger which existed inside the database. Let me expand on what I am saying. The database had that trigger to audit some schema changes which were supposed to save into an auditing table. Unfortunately, that auditing table did not exist in the server where the database was being restored, and the deletion of objects of replication settings were not completed, which means that 'sp_restoredbreplication' was not executed correctly. Consequently, the restoration was stopped and SQL Server decided to leave the database OFFLINE.

In order to restore a copy of this database, we need to disable all DDL database triggers before taking its backup. Only then will the database be restored successfully. The other method to deal with this issue is to change the status to ONLINE manually after the restoration finishes unsuccessfully and also execute 'sp_restoredbreplication'.

To sum up, we need to proceed with more cautiousness while working with databases linked to replication. That is all for now. Let me know any remarks you may have. Thanks for reading. Stay tuned.

Table-valued user-defined functions and the database collation

When it comes to altering database collations we may face some problems that tend to slow us down at the beginning. Nevertheless, digging into the message errors we might not spot the causes easily. For instance, while executing the following script to change the collation at database level an error may arise informing that some objects depends on it and it is not possible to make that change. Here is the script.

ALTER DATABASE MyDBUser SET RESTRICTED_USER WITH ROLLBACK IMMEDIATE; 
ALTER DATABASE MyDBUser COLLATE SQL_Latin1_General_CP1_CI_AS;  
ALTER DATABASE MyDBUser SET MULTI_USER WITH ROLLBACK IMMEDIATE;

As far as we know collations are heavily linked to character columns, and it includes columns of table-valued user-defined functions as they may have character columns on its definition. When theses functions are created they inherit the database collation by design for their columns. Here the error message:

Msg 5075, Level 16, State 1, Line 1
The object 'TVFUserTable' is dependent on database collation. The database collation cannot be changed if a schema-bound object depends on it. Remove the dependencies on the database collation and then retry the operation.
Msg 5072, Level 16, State 1, Line 1
ALTER DATABASE failed. The default collation of database 'MyDBUser' cannot be set to SQL_Latin1_General_CP1_CI_AS.

What we just need to do to be able to change the collation at database level is firstly drop every schema-bound objects, then make the change and finally create the objects again. That is all for now. Let me know any remarks you may have.