Study guide for Exam DP-203: Data Engineering

Handle Missing Data Microsoft DP-203

Implement Version Control for Pipeline Artifacts – Create and Manage Batch Processing and Pipelines

Bill Mettler 07/05/2024 0 Comment

You integrated your Azure Synapse Analytics workspace with GitHub in Exercise 3.6. Each time you clicked the Commit button, the configuration changes you made were stored into your selected...

Handle Failed Batch Loads Handle Missing Data Microsoft DP-203

Trigger Batches – Create and Manage Batch Processing and Pipelines

Bill Mettler 06/22/2024 0 Comment

Earlier in this chapter you learned a great deal about triggering pipelines. You can trigger pipelines manually, or you can create numerous different Azure Synapse Analytics trigger types that...

Handle Failed Batch Loads Microsoft DP-203 STORAGE EVENT TRIGGERS

Implement Azure Synapse Link and Query the Replicated Data – Create and Manage Batch Processing and Pipelines

Bill Mettler 05/30/2024 0 Comment

Chapter 2 briefly introduced data processing types such as OLTP, OLAP, and HTAP, and Chapter 10 covered them in a bit more detail. The overlying principle with these data...

Handle Failed Batch Loads Microsoft DP-203 Revert Data to a Previous State

Use PolyBase to Load Data to a SQL Pool – Create and Manage Batch Processing and Pipelines

Bill Mettler 03/16/2024 0 Comment

PolyBase is a very powerful technology that works behind the scenes to enable the execution of SQL queries on data files, where the data files are, for example, in...

Microsoft DP-203 Trigger Batches

Enable Copy Activity Data Consistency Verification – Create and Manage Batch Processing and Pipelines

Bill Mettler 02/26/2024 0 Comment

The Copy Data activity is a very powerful and convenient way to ingest data to your data lake. Exercise 3.13 was your first exposure to the Copy Data activity....

Handle Failed Batch Loads Microsoft DP-203

Manage Data Pipelines – Create and Manage Batch Processing and Pipelines

Bill Mettler 01/23/2024 0 Comment

When you attempt to manage something, you must first know in some detail what it is you are managing. In this context, you need to manage a pipeline in...

Handle Missing Data Microsoft DP-203 Revert Data to a Previous State

Handle Security and Compliance Requirements – Create and Manage Batch Processing and Pipelines

Bill Mettler 12/24/2023 0 Comment

The in‐depth coverage of security‐related concepts is coming in Chapter 8, “Keeping Data Safe and Secure.” The different aspects of security are vast, but here you will get a...

Handle Missing Data Microsoft DP-203 Trigger Batches

Directory Structure – Create and Manage Batch Processing and Pipelines

Bill Mettler 11/21/2023 0 Comment

The following directory structure should not be something new to you at this point. And perhaps you can already visualize how this directory can be used for the incremental...

Microsoft DP-203 Revert Data to a Previous State Trigger Batches

Design and Implement Incremental Data Loads – Create and Manage Batch Processing and Pipelines

Bill Mettler 10/19/2023 0 Comment

The design part of incremental loading was covered in Chapter 3, in the section “Design for Incremental Loading.” As shown in Figure 3.3, the serving layer is part of...

Microsoft DP-203 Revert Data to a Previous State Trigger Batches

Sink – Create and Manage Batch Processing and Pipelines

Bill Mettler 09/14/2023 0 Comment

The Settings tab of a sink in a data flow includes an attribute named Batch Size. This setting is valid in the context of columnar storage and caching, which...