The Data Engineer Blog

Posted on July 28, 2021

This article will explain how to use the SSIS lookup transformation and how it differs from the fuzzy lookup. This is a continuation of the SSIS features face-to-face series that we published previously on this website.

Posted on July 2, 2021

This article gives an overview of the SQL Server ISNUMERIC built-in system function. It explains how to use this function, illustrates its limitations, and provides some alternatives.

Posted on June 10, 2021

This article will explain ArangoDB, how to install it on Windows, and how to migrate a SQL Server graph database to this NoSQL database management system.

Posted on May 13, 2021

In this article, we will briefly explain the VS_NEEDSNEWMETADATA SSIS exception, one of the most popular exceptions that an ETL developer may face while using SSIS. Then, we will run an experiment that reproduces this error. Then, we will show how we can fix it.

Posted on May 10, 2021

This article briefly explains how to execute scripts written in Python in SQL Server and how Machine Learning services facilitate this process.

Posted on April 28, 2021

This article mentions some of the top free T-SQL training videos available on the internet.

Posted on April 16, 2021

This article explains how to use the sp_executesql system stored procedure to run static and dynamic SQL queries and how to pass input and output parameters while using it.

Posted on April 6, 2021

This article is an extension of the previsouly published article “Migrating SQL Server graph databases to Neo4j”, where we will explain how to export the supported indexes and constraints from SQL Server to the Neo4j graph database.

Posted on April 6, 2021

Professional certification recognizes your knowledge and experience and enhances your professional credibility. Many students and developers cannot get certified since they cannot pay the certifications exams fees. After a little research on the internet, I found several database certification exams available for free by leading database providers in the market.... [Read More]

Posted on March 30, 2021

This article will explain how to use the Business Intelligence Markup Language to create an SSAS tabular model.

Posted on March 16, 2021

This post mentions some SQL Server experts you need to know during your SQL Server journey. Those experts are Microsoft Most Valuable Professionals that highly contributed to SQL Server the community.

Posted on March 16, 2021

In this article, we will explain how to use the Business Intelligence Markup Language (Biml) to build SSAS OLAP cubes, noting that this article is the ninth article in the Learn Biml series.

Posted on March 9, 2021

This article provides a step-by-step guide for migrating SQL Server graph databases to Neo4j using C#.

Posted on February 5, 2021

In this article, we will talk briefly about graph databases and what Neo4j is. Then we will give a step-by-step guide on how to install and get started with Neo4j on Windows operating system.

Posted on December 25, 2020

In this article, we will explain how to install MongoDB on the Windows operating system, how to install and configure the ODBC driver and connector that allows us to import data from MongoDB to SQL Server using SSIS.

Posted on December 20, 2020

Even after the rise of Big Data technologies, Microsoft SQL Server Integration Services still one of the most popular data integration tools. Mainly, SSIS developers use Visual Studio to develop their data integration packages. One of the main challenges that face the SSIS developers is that they design tens of... [Read More]

Posted on December 9, 2020

As in the first release (1.0.0), SchemaMapper was developed to merge data from different file types (flat file, Excel, Access …) into one SQL table. SchemaMapper 1.1.0 was released after being improved to support reading from relational databases and writing data into more data formats. Also, SchemaMapper 1.1.0 is now... [Read More]

Posted on December 7, 2020

This article will briefly mention nine features added starting SQL Server 2008 that make SQL Server more than a traditional database management system.

Posted on October 14, 2020

In this article, we will explain how to build databases using BIML. We will be covering handling databases objects such as schemas, tables, columns, indexes, and other objects.

Posted on September 14, 2020

In this article, we will be talking about Manticore Search, which is an open-source search engine first released in 2017 as a fork of the Sphinx search engine. We will try to describe this search engine briefly, mention some of its differences from the Sphinx search engine, and we will... [Read More]

Posted on August 4, 2020

In this article, we will talk about building full-text indexes using Sphinx. We will be covering seven topics: The Sphinx configuration file Defining data sources Defining full-text indexes Indexer and Searchd services options Database preparation Indexing Using Linked servers to access indexes

Posted on July 11, 2020

In this article, we will be talking about the Sphinx search engine and how to use it to install it on the Windows operating system. This is the first article in the “Sphinx Search Engine” series, where we will explain how to install and use this search engine to create... [Read More]

Posted on May 19, 2020

In this article, we will talk briefly about data quality in SQL Server. Then, we will give a brief overview of the Melissa Data Quality for SQL Server Integration Services (SSIS), and we will demonstrate the components available in the community edition.

Posted on May 15, 2020

After working on several projects, I ended up with a consequence that it is not recommended to install Hadoop on windows operating system. In this article, I briefly described the main reasons for this consequence.

Posted on May 14, 2020

In this article, we will be talking about Hadoop Hive and Hadoop Pig Tasks. We will first give a brief overview of Apache Hive and Apache Pig. Then, we will illustrate the related SSIS Hadoop components and alternatives.