The Data Engineer Blog

Posted on January 5, 2023

Dear Audience, 2022 was my first year as a contributing author for MSSQLTips.com. I have written an article series (6 articles) named “SSIS Bad Habits” where I explained several bad practices that decrease the SSIS packages performance. I am gload to annouce that I was recognized as a contender for... [Read More]

Posted on December 28, 2022

This article briefly explains the SQL group by clause, when it should be used, and what we should consider when using it.

Posted on December 1, 2022

This tip will explain some mistakes SSIS developers usually make when designing ETL and how they should be avoided.

Posted on November 18, 2022

This article explains what the SQL injection attack is, why it could be hazardous, and how to defend our SQL database from this attack using parameterized queries and some third-party tools.

Posted on August 4, 2022

This article explains when databases are not suitable for data staging and what formats SSIS developers should use to stage their data when ETL jobs are developed using SQL Server Integration Services.

Posted on June 22, 2022

This article explains the xp_cmdshell and sp_xp_cmdshell_proxy_account system stored procedures in SQL Server and how developers can use them to execute Windows commands.

Posted on June 7, 2022

This tutorial explains how lookups may highly decrease SSIS package performance and when they should only be used in Business Intelligence projects.

Posted on May 4, 2022

This tutorial will explain some of the mistakes that SSIS developers usually make when configuring the OLE DB component while designing SSIS packages and how they should avoid them.

Posted on April 19, 2022

This article explains data staging and staging methods in SSIS, and how to choose the appropriate approach while designing your data pipeline.

Posted on April 5, 2022

This tutorial explains the main cause of the performance problem faced while merging different data sources in SQL Server Integration Services and provides a few alternatives.

Posted on February 15, 2022

Is SSIS extension available for Visual Studio 2022?

Posted on January 18, 2022

This article gives an overview of DIFFERENCE and SOUNDEX SQL Server built-in system functions. It explains how to use those functions and how do they work.

Posted on January 16, 2022

This answer explains how to to loop over multiple flat files and insert them into seperate SQL Server tables.

Posted on January 13, 2022

Since many students and developers cannot pay for the certifications exams fees, I created a #GitHub repository to list all free professional exams provided by tech companies even if it was offered on a specific occasion (ex: Microsoft Ignite Cloud Skills Challenge). I will be grateful for any contribution. And... [Read More]

Posted on January 10, 2022

This article will explain how to create a SQL Server CLR function stored in a C# class library without creating a SQL Server database project. We will first briefly explain the concept of .NET common language runtime (CLR) and then build a C# class library using the .NET framework. Then,... [Read More]

Posted on December 30, 2021

This answer explains why the milliseconds part is removed when passing a DateTime value as a parameter in a OLE DB source SQL Command. Besides, it explains the different data types systems used in SSIS.

Posted on December 29, 2021

How to split a comma seperated value passed as a parameter to an SQL Command used in the SSIS OLE DB Source when the database does not support STRING_SPLIT() system function?

Posted on November 25, 2021

This article explains both the SSIS term extraction and term lookup components. Besides, it illustrates the use cases of each one of them.

Posted on November 19, 2021

This article explains the SSIS Script task and Script component, the scripting tools available in the SQL Server Integration Services (SSIS).

Posted on November 11, 2021

Is it Possible to Load data to MongoDB Using SSIS? If yes tools required to achieve? Does MongoDB Support all the SQL Operations?

Posted on November 3, 2021

This article explains the SSIS XML Source and the SSIS XML task, which are the XML-related components in the SQL Server Integration Services.

Posted on October 5, 2021

This article compares the SSIS OLE DB Destination SQL command with the “Table or View” data access mode

Posted on September 29, 2021

SSIS Merge Join and Merge transformations are two similar components used to combine two input data sources into one output. This article explains both transformations and illustrates when and how to use each one of them.

Posted on September 8, 2021

This article explains the SSIS pivot transformation and how it differs from the unpivot transformation.

Posted on August 31, 2021

How to filter data in Oracle based on a SQL Server query result?