The Data Engineer Blog

Posted on September 29, 2023

This step-by-step technical article will explore how to perform data profiling on a database using SQL code. We will use a combination of SQL queries to gather various metrics and statistics about the data, such as column data types, null values, data length, and more. Ultimately, all SQL code portions... [Read More]

Posted on August 3, 2023

I will briefly remind you about the data wrangling concept in this article. Then, I will dive into the SQL Server functions used to create a custom data wrangling tool.

Posted on July 19, 2023

This article explains how implicit conversions affect SQL query performance and cause the query optimizer to perform an index scan instead of index seek while searching for a specific value.

Posted on June 30, 2023

Inadequate data quality can lead to significant financial or business opportunity losses. Still, many SQL Server data practitioners are unaware of the available data quality solutions. This article briefly explains SQL Server’s solutions to assess and improve data quality.

Posted on June 27, 2023

This article emphasizes the significance of data quality in data-driven strategies and highlights the consequences of implementing data warehousing scenarios that need more attention to data quality. It introduces the concept of data quality management and discusses the ISO/IEC 25012 standard’s fifteen data quality characteristics. The article explains the different... [Read More]

Posted on June 14, 2023

This article explains what database monitoring is and why it is essential. Then, it illustrates the different methods for monitoring MongoDB NoSQL databases.

Posted on June 13, 2023

The term data quality refers to measuring the fitness of data regarding the intended usage. Poor data quality leads to inadequate, inconsistent, and erroneous decisions that could escalate the computational cost, cause a decline in profits, and cause customer churn. Thus, data quality is crucial for researchers and industry practitioners.... [Read More]

Posted on June 13, 2023

In the big data domain, data quality assessment operations are often complex and must be implementable in a distributed and timely manner. This paper tries to generalize the quality assessment operations by providing a new ISO-based declarative data quality assessment framework (BIGQA). BIGQA is a flexible solution that supports data... [Read More]

Posted on June 3, 2023

In this article, we will examine the effectiveness of several popular AI detectors by subjecting them to a human-written paragraph and an AI-generated text. Subsequently, we will analyze and discuss the outcomes obtained from these tests.

Posted on April 24, 2023

When working with databases, it is common to encounter scenarios where you must search for data within a specific date range. Fortunately, Microsoft SQL Server provides several methods for performing these types of searches with T-SQL. This SQL tutorial illustrates some of the most common techniques for searching between two... [Read More]

Posted on April 5, 2023

In this article, we will be asking several questions about prompt engineering. It is worth noting that while ChatGPT has been trained over a vast amount of data up to 2021, prompt engineering has only recently become a buzzword in 2023.

Posted on March 29, 2023

Interviewing ChatGPT is an article series where we delve into various technical domains by conversing with ChatGPT. With its knowledge base and natural language processing capabilities, ChatGPT can provide insights and answers to various technical questions. In this article, we will be asking several questions about data engineering.

Posted on March 26, 2023

In this article, we will focus on three essential skills that every computer scientist should have in 2023 but are often overlooked: efficient Google searching, effective use of Stack Overflow for asking and answering questions, and having the ability to ask good questions to intelligent virtual assistants like ChatGPT By... [Read More]

Posted on March 15, 2023

This tutorial explains different data types that can be used to store date time values and illustrates when to use each one in scripts, stored procedures and more.

Posted on March 6, 2023

This article explains the different approaches used to insert multiple rows into SQL Server tables.

Posted on March 1, 2023

This tutorial, illustrates the different use cases and options available when using the WHERE clause and demonstrates them using the AdventureWorks2017 database.

Posted on February 10, 2023

I am very thankful for being awarded the MSSQLTips.com Rookie of the year. I am pleased that this award was based on the SSIS Bad Habits article series I published in 2022. A special thanks to Jeremy Kadlec, who encouraged me to complete this article series before the end of... [Read More]

Posted on February 6, 2023

This article illustrates how to use a Microsoft SQL Server Integration Services (SSIS) external configuration file to enable the properties of package objects to be dynamically updated at run time.

Posted on February 3, 2023

This article provides more than 50 questions that can be used within an SSIS job interview.

Posted on January 29, 2023

Tech Blog is a Medium publication that tends to publish technical articles, ideas, and tutorials. If you are interested in sharing your knowledge with us, please read the article writing guidelines and submission rules before submitting your first article.

Posted on January 20, 2023

In this article, we will reproduce the “Cannot convert between Unicode and non-Unicode string data types” exception and explain several solutions to this issue. [Read More]

Posted on January 17, 2023

This article gives an overview of the approaches to migrating SQL Server data to a MongoDB database. [Read More]

Posted on January 17, 2023

This article will show the SQL standard deviation function with several examples.

Posted on January 5, 2023

Dear Audience, 2022 was my first year as a contributing author for MSSQLTips.com. I have written an article series (6 articles) named “SSIS Bad Habits” where I explained several bad practices that decrease the SSIS packages performance. I am gload to annouce that I was recognized as a contender for... [Read More]

Posted on December 28, 2022

This article briefly explains the SQL group by clause, when it should be used, and what we should consider when using it.