Data ingestion is the process of collecting, importing, and transferring raw data from various sources into a storage or processing system for analysis. In Azure, this is a foundational step in any analytics workload, ensuring that data flows seamlessly from its origin to where it can be transformed and analyzed.

**Data Ingestion Concepts:**

Data ingestion can occur in two primary modes:

1. **Batch Ingestion** – Data is collected and transferred in scheduled intervals or large chunks. This is ideal for periodic reporting and historical analysis. Tools like Azure Data Factory support batch processing efficiently.

2. **Real-Time (Streaming) Ingestion** – Data is continuously ingested as it is generated. This is critical for scenarios like IoT telemetry, fraud detection, and live dashboards. Azure Event Hubs and Azure Stream Analytics are commonly used for streaming ingestion.

Key considerations during ingestion include data format compatibility, latency requirements, data volume, security, and error handling.

**Pipelines:**

A data pipeline is an orchestrated workflow that automates the movement and transformation of data from source to destination. Pipelines define the sequence of activities such as copying data, transforming it, and loading it into a target system.

**Azure Data Factory (ADF)** is the primary Azure service for building data pipelines. It provides a visual interface and supports:

- **Data Movement** – Copying data from on-premises or cloud sources into Azure storage (e.g., Azure Data Lake, Azure SQL Database).
- **Data Transformation** – Using data flows or integrating with services like Azure Databricks and HDInsight for processing.
- **Scheduling and Triggers** – Automating pipeline execution based on time or events.
- **Monitoring** – Tracking pipeline runs and handling failures.

**Azure Synapse Analytics** also includes built-in pipeline capabilities similar to ADF, enabling end-to-end analytics workflows.

In summary, data ingestion and pipelines form the backbone of Azure analytics workloads, enabling organizations to reliably collect, process, and prepare data for meaningful insights and decision-making.

Azure Data Factory (ADF) is a cloud-based data integration service provided by Microsoft Azure that enables the creation, scheduling, and orchestration of data-driven workflows, commonly known as pipelines. It is designed to handle the complex process of extracting, transforming, and loading (ETL) or extracting, loading, and transforming (ELT) data from various sources into centralized data stores for analytics purposes.

At its core, Azure Data Factory allows organizations to ingest data from a wide range of sources, including on-premises databases, cloud-based storage, SaaS applications, and more. It supports over 90 built-in connectors, making it highly versatile for diverse data environments.

Key components of Azure Data Factory include:

1. **Pipelines**: Logical groupings of activities that together perform a task, such as moving and transforming data.

2. **Activities**: Individual processing steps within a pipeline, such as copying data, running a stored procedure, or executing a Databricks notebook.

3. **Datasets**: Named views of data that point to the data you want to use in your activities.

4. **Linked Services**: Connection strings that define the connection information needed for Data Factory to connect to external resources.

5. **Triggers**: Units that determine when a pipeline execution should be initiated, such as on a schedule or in response to an event.

6. **Data Flows**: Visually designed transformation logic that allows users to build data transformation processes without writing code.

ADF plays a crucial role in analytics workloads on Azure by serving as the orchestration engine that moves raw data from source systems into data lakes, data warehouses like Azure Synapse Analytics, or other storage solutions. It supports both batch and real-time data movement scenarios.

Its integration with other Azure services, such as Azure Databricks, Azure HDInsight, and Azure Synapse Analytics, makes it a central component of modern cloud-based analytics architectures, enabling organizations to build comprehensive, scalable, and automated data pipelines for business intelligence and advanced analytics.

Data Warehouses and Data Lakehouses are two key architectural approaches used in analytics workloads on Azure.

**Data Warehouses** are centralized repositories designed to store structured, processed data optimized for analytical queries and reporting. They follow a schema-on-write approach, meaning data is cleaned, transformed, and organized before being loaded. Azure Synapse Analytics is Microsoft's primary data warehousing solution. Data warehouses use relational database principles with structured tables, defined schemas, and SQL-based querying. They excel at providing fast query performance for business intelligence (BI) workloads, dashboards, and structured reporting. Data is typically organized using star or snowflake schemas with fact and dimension tables, making it highly optimized for aggregations and complex analytical queries.

**Data Lakehouses** combine the best features of data lakes and data warehouses into a unified architecture. A data lake stores vast amounts of raw data in various formats (structured, semi-structured, and unstructured), while a data lakehouse adds a structured management layer on top of this raw storage. This enables both flexible data storage and high-performance analytical querying. Azure Synapse Analytics and Microsoft Fabric support lakehouse architectures. Data lakehouses use a schema-on-read approach, allowing data to be stored in its native format and structured when needed. They support ACID transactions, schema enforcement, and governance features typically associated with data warehouses while retaining the scalability and cost-effectiveness of data lakes.

**Key Differences:** Data warehouses are best for structured, curated data with predictable query patterns, while data lakehouses offer greater flexibility by handling diverse data types and supporting both data science and traditional BI workloads. Data lakehouses reduce data duplication by eliminating the need to maintain separate lake and warehouse systems.

In Azure, services like Azure Synapse Analytics, Azure Data Lake Storage, and Microsoft Fabric provide integrated solutions for building both data warehouse and data lakehouse architectures to meet diverse analytical needs.

Azure Synapse Analytics is a comprehensive, integrated analytics service provided by Microsoft Azure that brings together enterprise data warehousing and Big Data analytics into a single unified platform. It enables organizations to ingest, prepare, manage, and serve data for immediate business intelligence and machine learning needs.

At its core, Azure Synapse Analytics combines several key capabilities:

1. **SQL Pools (formerly SQL Data Warehouse):** Synapse offers both dedicated SQL pools and serverless SQL pools. Dedicated SQL pools provide provisioned compute resources for high-performance data warehousing workloads, while serverless SQL pools allow on-demand querying of data without infrastructure management, making it cost-effective for ad-hoc analysis.

2. **Apache Spark Pools:** Built-in Apache Spark integration enables big data processing, machine learning, and data transformation using languages like Python, Scala, SQL, and .NET.

3. **Synapse Pipelines:** These are data integration tools similar to Azure Data Factory, allowing users to build ETL/ELT workflows to orchestrate data movement and transformation across various sources.

4. **Synapse Studio:** A unified web-based workspace where data engineers, data scientists, and analysts can collaborate. It provides a single interface for managing SQL scripts, notebooks, data flows, and monitoring pipelines.

5. **Integration with Power BI and Azure Machine Learning:** Synapse seamlessly connects with Power BI for data visualization and Azure ML for advanced analytics scenarios.

Azure Synapse supports querying both relational and non-relational data, including structured data in data warehouses and unstructured data stored in Azure Data Lake Storage. This makes it ideal for implementing modern data lakehouse architectures.

Key benefits include massive scalability, pay-as-you-go pricing models, enterprise-grade security, and reduced time to insight by eliminating data silos. Organizations use Synapse Analytics for reporting, dashboarding, advanced analytics, and real-time analytics workloads, making it a cornerstone solution for building end-to-end analytics solutions on Azure.

Azure Databricks is a powerful, cloud-based analytics platform built on Apache Spark, designed to process and analyze massive volumes of data efficiently. It is a fully managed service offered by Microsoft Azure in collaboration with Databricks, providing a unified workspace for data engineers, data scientists, and business analysts to collaborate seamlessly.

At its core, Azure Databricks leverages Apache Spark clusters to perform large-scale data processing, machine learning, and streaming analytics. It supports multiple programming languages, including Python, Scala, SQL, and R, making it versatile for various analytical workloads.

Key features of Azure Databricks include:

1. **Collaborative Workspace**: It provides interactive notebooks where teams can share code, visualizations, and insights in real time, fostering collaboration across different roles.

2. **Scalability**: Azure Databricks automatically scales compute resources up or down based on workload demands, ensuring cost-efficiency and optimal performance.

3. **Integration with Azure Services**: It seamlessly integrates with other Azure services such as Azure Data Lake Storage, Azure Synapse Analytics, Azure Blob Storage, Power BI, and Azure Machine Learning, enabling end-to-end data pipelines.

4. **Delta Lake Support**: Azure Databricks supports Delta Lake, an open-source storage layer that brings reliability, performance, and ACID transactions to data lakes, enabling both batch and streaming data processing.

5. **Machine Learning**: It offers built-in MLflow integration for managing the complete machine learning lifecycle, including experiment tracking, model deployment, and monitoring.

6. **Security and Compliance**: Azure Databricks provides enterprise-grade security with role-based access control, encryption, and compliance with industry standards.

In the context of analytics workloads, Azure Databricks is ideal for big data processing, ETL (Extract, Transform, Load) operations, real-time streaming analytics, and advanced machine learning tasks. It bridges the gap between data engineering and data science, enabling organizations to derive actionable insights from their data at scale while maintaining a unified and efficient analytics environment.

Microsoft Fabric is a unified, end-to-end analytics platform offered by Microsoft Azure that brings together a comprehensive suite of data services into a single integrated environment. It is designed to streamline the entire analytics workflow, from data ingestion and transformation to advanced analytics and visualization, eliminating the need for managing multiple disconnected tools.

At its core, Microsoft Fabric integrates key services such as Data Engineering, Data Factory, Data Science, Data Warehouse, Real-Time Analytics, and Power BI into one cohesive platform. This convergence enables data professionals—including data engineers, data scientists, data analysts, and business users—to collaborate seamlessly within a shared environment.

One of the defining features of Microsoft Fabric is its foundation on OneLake, a unified data lake that serves as a single repository for all organizational data. OneLake eliminates data silos by providing one centralized storage layer, ensuring that all users and services access the same consistent data without unnecessary duplication. This approach simplifies data governance, security, and management.

Microsoft Fabric operates on a Software-as-a-Service (SaaS) model, which means it reduces the overhead associated with infrastructure management. Users can focus on deriving insights from data rather than provisioning and maintaining underlying systems. The platform also leverages a consumption-based pricing model through capacity units, making it scalable and cost-efficient.

Key workloads supported by Microsoft Fabric include building data pipelines, creating lakehouses and data warehouses, performing machine learning experiments, conducting real-time analytics on streaming data, and generating rich visualizations through Power BI. All these workloads are tightly integrated, allowing smooth transitions between stages of the analytics lifecycle.

In summary, Microsoft Fabric represents Microsoft's vision for a unified analytics platform that simplifies complex data workflows, promotes collaboration, centralizes data storage through OneLake, and provides a comprehensive set of tools to support modern analytics workloads on Azure.

Batch data processing is a method of collecting and processing large volumes of data at scheduled intervals rather than in real-time. In the context of Azure analytics workloads, batch processing is fundamental for handling massive datasets efficiently.

**Core Concepts:**

Batch processing involves gathering data over a period of time and processing it as a single group or 'batch.' Unlike stream processing, which handles data in real-time, batch processing operates on bounded datasets with a defined start and end point.

**Key Characteristics:**

1. **High Volume:** Batch processing excels at handling large amounts of data, often terabytes or petabytes, making it ideal for big data scenarios.

2. **Scheduled Execution:** Jobs are typically scheduled to run at specific times—hourly, daily, or weekly—depending on business requirements.

3. **Latency Tolerance:** Since data is collected over time before processing, there is inherent latency. This approach suits scenarios where immediate results are not critical.

4. **Complex Transformations:** Batch processing allows for extensive data transformations, aggregations, and computations that would be too resource-intensive for real-time processing.

**Azure Services for Batch Processing:**

- **Azure Data Lake Storage:** Serves as a scalable repository for storing large volumes of raw data before batch processing.
- **Azure Synapse Analytics:** Provides powerful batch processing capabilities using SQL pools and Apache Spark pools.
- **Azure Databricks:** Offers collaborative Apache Spark-based analytics for large-scale data engineering and transformation.
- **Azure Data Factory:** Orchestrates and automates batch data movement and transformation through pipelines.
- **Azure HDInsight:** Supports batch processing using open-source frameworks like Hadoop and Spark.

**Common Use Cases:**

Batch processing is widely used for ETL (Extract, Transform, Load) operations, generating periodic reports, training machine learning models, processing log files, and performing historical data analysis.

**ELT vs ETL:**

Modern batch architectures often use ELT (Extract, Load, Transform), where raw data is first loaded into a data lake or warehouse and then transformed, leveraging the processing power of cloud-based systems for efficient large-scale data handling.

Stream data processing is a method of handling and analyzing data in real-time as it is generated or received, rather than storing it first and processing it later (which is batch processing). In Azure, stream processing is essential for scenarios requiring immediate insights from continuously flowing data.

**Key Concepts:**

1. **Real-Time Data Ingestion:** Stream processing begins with capturing data from various sources such as IoT devices, social media feeds, application logs, and clickstreams. Azure Event Hubs and Azure IoT Hub serve as primary ingestion services for high-throughput, real-time data streams.

2. **Event Processing:** Each piece of data in a stream is called an event. Events are processed individually or in small time windows as they arrive. Azure Stream Analytics is a fully managed service that enables real-time analytics using SQL-like queries on streaming data.

3. **Windowing Functions:** Since streaming data is unbounded, windowing functions group events into finite time segments (tumbling, hopping, sliding, or session windows) for aggregation and analysis.

4. **Low Latency:** A primary advantage of stream processing is its ability to deliver results with minimal delay—often in milliseconds or seconds—enabling immediate decision-making, fraud detection, alert systems, and live dashboards.

5. **Temporal Processing:** Stream processing inherently deals with time-ordered data, allowing analysis based on event time or arrival time to detect patterns, trends, and anomalies in real time.

6. **Integration with Output Sinks:** Processed streaming data can be directed to various destinations including Azure SQL Database, Power BI dashboards, Azure Cosmos DB, Blob Storage, or Azure Synapse Analytics for further analysis or visualization.

7. **Apache Technologies:** Azure also supports open-source stream processing through Apache Kafka (via Azure HDInsight or Confluent) and Apache Spark Structured Streaming (via Azure Databricks).

Stream processing complements batch processing in modern analytics architectures, often combined in a **Lambda architecture**, where real-time and historical data processing coexist to provide comprehensive analytical capabilities.

Azure Stream Analytics is a real-time analytics and complex event-processing engine designed to analyze and process high volumes of streaming data from multiple sources simultaneously. It is a fully managed (PaaS) service on Microsoft Azure that enables users to set up real-time analytic computations on data streaming from applications, devices, sensors, websites, social media, and other sources.

At its core, Azure Stream Analytics uses a SQL-like query language, making it accessible to users familiar with SQL. This allows developers and data analysts to define transformations, aggregations, and filters on streaming data without needing deep programming expertise. It supports temporal operations such as windowed aggregates, temporal joins, and temporal analytic functions, which are essential for time-based data analysis.

Key features of Azure Stream Analytics include:

1. **Real-Time Processing**: It processes data in real time with sub-second latencies, enabling instant insights and actions based on incoming data streams.

2. **Multiple Input Sources**: It can ingest data from Azure Event Hubs, Azure IoT Hub, and Azure Blob Storage, making it versatile for various streaming scenarios.

3. **Multiple Outputs**: Processed results can be directed to destinations such as Azure SQL Database, Azure Cosmos DB, Power BI, Azure Data Lake Storage, and more.

4. **Built-in Machine Learning**: It integrates with Azure Machine Learning, allowing users to apply ML models to streaming data for predictive analytics and anomaly detection.

5. **Scalability**: The service can scale up or down based on workload demands, handling from kilobytes to gigabytes of events per second.

6. **Reliability**: It guarantees exactly-once event processing and at-least-once delivery of events, ensuring data integrity.

Common use cases include IoT telemetry analysis, real-time dashboarding, fraud detection, log monitoring, and clickstream analysis. Azure Stream Analytics is a critical component of analytics workloads on Azure, bridging the gap between raw streaming data and actionable insights in real time.

Real-Time Intelligence and Event Processing in Azure refers to the capability of capturing, analyzing, and acting upon data as it is generated, rather than waiting for it to be stored and processed in batches. This is critical for scenarios requiring immediate insights, such as fraud detection, IoT monitoring, live dashboards, and stock trading.

At the core of real-time processing in Azure is **Azure Stream Analytics**, a fully managed event-processing engine that enables real-time analytics on multiple streams of data. It uses a SQL-like query language to filter, aggregate, and analyze streaming data from sources like IoT devices, applications, and social media feeds.

**Azure Event Hubs** serves as a big data streaming platform and event ingestion service, capable of receiving and processing millions of events per second. It acts as a front door for an event pipeline, decoupling event producers from event consumers.

**Apache Kafka on Azure (Azure HDInsight or Azure Event Hubs with Kafka endpoint)** provides another robust option for distributed event streaming, supporting publish-subscribe messaging patterns for high-throughput data pipelines.

**Microsoft Fabric Real-Time Intelligence** is a newer offering that provides end-to-end real-time analytics, allowing users to ingest, process, and visualize streaming data seamlessly within the Fabric ecosystem using tools like Eventstreams and KQL (Kusto Query Language) databases.

The typical real-time processing architecture follows this pattern: data sources generate events, which are ingested through Event Hubs or similar services, processed by Stream Analytics or Fabric Real-Time Intelligence, and then output to dashboards (Power BI), storage (Azure Data Lake), or trigger actions (Azure Functions).

Key benefits include low-latency insights, proactive decision-making, anomaly detection, and the ability to respond to changing conditions instantly. Real-time intelligence transforms raw event data into actionable information, enabling organizations to move from reactive to proactive operations, ultimately driving better business outcomes through timely and informed decisions.

Power BI is Microsoft's comprehensive business analytics platform that comes in two primary forms: Power BI Desktop and Power BI Service, each offering distinct capabilities for data analysis and visualization.

**Power BI Desktop** is a free Windows application installed locally on your computer. It serves as the primary development and authoring tool where analysts create reports and data models. Key capabilities include:

- **Data Connectivity:** Connect to hundreds of data sources including Azure SQL Database, Azure Synapse Analytics, Excel, CSV files, and REST APIs.
- **Data Transformation:** Using Power Query Editor, users can clean, shape, and transform data through an intuitive interface without writing complex code.
- **Data Modeling:** Create relationships between tables, define calculated columns, measures using DAX (Data Analysis Expressions), and build comprehensive data models.
- **Report Authoring:** Design interactive visualizations, charts, graphs, maps, and dashboards with drag-and-drop functionality and rich formatting options.
- **Advanced Analytics:** Incorporate AI-powered features like Q&A visuals, decomposition trees, and key influencer charts.

**Power BI Service** is a cloud-based SaaS (Software as a Service) platform accessible through a web browser at app.powerbi.com. It focuses on collaboration, sharing, and consumption. Key capabilities include:

- **Publishing and Sharing:** Publish reports from Desktop and share them across the organization through workspaces and apps.
- **Dashboards:** Create real-time dashboards by pinning visuals from multiple reports onto a single canvas.
- **Collaboration:** Enable team collaboration through shared workspaces, commenting, and role-based access control.
- **Scheduled Data Refresh:** Configure automatic data refresh schedules to keep reports current.
- **Data Governance:** Implement row-level security, sensitivity labels, and compliance features.
- **Natural Language Queries:** Ask questions about your data using everyday language.
- **Alerts and Subscriptions:** Set data-driven alerts and email subscriptions for automated report distribution.

Together, Power BI Desktop and Service form a complete analytics workflow: author in Desktop, publish and collaborate in the Service, enabling organizations to derive actionable insights from their Azure data workloads.

In Power BI, data models and relationships form the backbone of effective data analysis and reporting. A data model is a structured representation of data that defines how different tables, columns, and measures are organized and interconnected within a Power BI dataset.

**Data Models:**
Power BI uses a tabular data model based on the Vertipaq engine, which stores data in a highly compressed, in-memory columnar format. A data model consists of tables, columns, measures, calculated columns, and hierarchies. Tables can be imported from various sources such as Azure SQL Database, Azure Synapse Analytics, Excel files, or other cloud and on-premises sources. The model serves as the semantic layer between raw data and visualizations.

**Relationships:**
Relationships in Power BI define how tables are connected to each other, enabling cross-table analysis. They are established by linking columns (keys) between tables. Key aspects include:

- **Cardinality:** Relationships can be one-to-one (1:1), one-to-many (1:*), or many-to-many (*:*). One-to-many is the most common, linking a dimension table to a fact table.

- **Cross-filter Direction:** This determines how filters propagate between tables — either single direction (from one side to the many side) or bidirectional (both ways).

- **Active vs. Inactive Relationships:** Only one active relationship can exist between two tables at a time, but inactive relationships can be activated using DAX functions like USERELATIONSHIP.

**Star Schema:**
Power BI works best with a star schema design, where a central fact table (containing measurable data like sales amounts) is surrounded by dimension tables (containing descriptive attributes like dates, products, or customers). This design optimizes query performance and simplifies report building.

**Importance:**
Properly defined relationships ensure accurate aggregations, filtering, and slicing of data across visualizations. Without correct relationships, reports may produce misleading results. Power BI can auto-detect relationships, but manual configuration is often necessary for accuracy and performance optimization.

Power BI is Microsoft's powerful business analytics and visualization platform that enables users to connect to various data sources, transform data, and create interactive reports and dashboards. It plays a central role in Azure analytics workloads by turning raw data into meaningful insights.

**Power BI Visualizations** include a wide range of chart types and visual elements:
- **Bar and Column Charts**: Ideal for comparing categorical data across groups.
- **Line Charts**: Used for showing trends over time.
- **Pie and Donut Charts**: Display proportional data distributions.
- **Tables and Matrices**: Present detailed tabular data with drill-down capabilities.
- **Maps**: Geographic visualizations for location-based data analysis.
- **Cards and KPIs**: Highlight single key metrics or performance indicators.
- **Scatter Plots**: Show relationships between two numerical variables.
- **Treemaps and Gauges**: Represent hierarchical data or progress toward goals.
- **Custom Visuals**: Power BI supports marketplace visuals and custom-built components for specialized needs.

**Report Types in Power BI** include:
1. **Interactive Reports**: Multi-page reports with various visualizations that allow users to filter, slice, and drill into data dynamically. These are the most common report type.
2. **Paginated Reports**: Pixel-perfect, print-ready reports designed for precise formatting, often used for invoices, financial statements, or operational reports. These are built using Power BI Report Builder.
3. **Dashboards**: Single-page canvases that pin key visuals from multiple reports, providing a consolidated high-level overview of important metrics.
4. **Mobile Reports**: Optimized layouts designed specifically for mobile device consumption.

Power BI integrates seamlessly with Azure services like Azure Synapse Analytics, Azure Data Lake, and Azure SQL Database, allowing analysts to query massive datasets directly. Reports can be published to the Power BI Service (cloud), shared across organizations, embedded in applications, and scheduled for automatic data refresh, making it a comprehensive end-to-end analytics solution within the Azure ecosystem.

Power BI Dashboards and Data Sharing are essential components of Microsoft Azure's analytics workload ecosystem, enabling organizations to visualize, monitor, and collaborate on data-driven insights effectively.

**Power BI Dashboards:**
A Power BI dashboard is a single-page canvas, often called a canvas, that uses visualizations (called tiles) to tell a story. Dashboards are created within the Power BI service and consolidate key metrics and KPIs from multiple reports and datasets into one unified view. Unlike reports, which can span multiple pages, dashboards provide a high-level overview that allows users to monitor business performance at a glance. Each tile on a dashboard is linked to its underlying report or dataset, enabling users to drill down into detailed data when needed. Dashboards support real-time data streaming, natural language queries (Q&A feature), and alerts that notify users when data changes beyond defined thresholds.

**Data Sharing:**
Power BI offers robust data sharing capabilities that promote collaboration across teams and organizations. Users can share dashboards and reports directly with specific individuals, publish them to workspaces for team access, or distribute them through Power BI Apps — packaged collections of dashboards and reports designed for broader audiences. Organizations can also embed Power BI content into applications, websites, or Microsoft Teams. Row-Level Security (RLS) ensures that shared data respects access permissions, so users only see data relevant to their roles. Additionally, Power BI supports exporting data to formats like PDF, Excel, and PowerPoint for external sharing.

Key sharing methods include:
- **Workspaces** for team collaboration
- **Apps** for organization-wide distribution
- **Publish to Web** for public access
- **Embed** for integration into custom applications

Together, Power BI Dashboards and Data Sharing empower organizations to democratize data access, foster collaboration, and drive informed decision-making across all levels of the business, making them integral to Azure's comprehensive analytics workload strategy.

Data Transformation with Power Query is a fundamental concept in Microsoft Azure analytics workloads that enables users to shape, clean, and prepare data before loading it into a data model for analysis. Power Query is a data connectivity and transformation engine built into tools like Power BI, Excel, and Azure Data Factory.

Power Query uses a user-friendly, low-code interface called the Power Query Editor, which allows users to perform a wide range of data transformation tasks without writing complex code. It uses a functional language called M (Power Query Formula Language) behind the scenes to execute transformations.

Key transformation capabilities include:

1. **Filtering and Sorting**: Removing unnecessary rows and organizing data based on specific criteria.

2. **Column Management**: Adding, removing, renaming, reordering, or splitting columns to restructure data.

3. **Data Type Conversion**: Changing data types such as text, numbers, dates, and booleans to ensure consistency.

4. **Merging and Appending**: Combining multiple data sources through joins (merge) or stacking datasets (append) to create unified tables.

5. **Pivoting and Unpivoting**: Reshaping data by converting rows to columns (pivot) or columns to rows (unpivot) for better analytical structure.

6. **Aggregation and Grouping**: Summarizing data using functions like sum, average, count, and grouping by specific categories.

7. **Handling Nulls and Errors**: Replacing or removing null values and error entries to ensure data quality.

8. **Custom Columns and Conditional Logic**: Creating calculated columns using custom formulas and if-then-else logic.

Power Query follows an ETL (Extract, Transform, Load) or ELT approach, where data is extracted from various sources such as Azure SQL Database, Azure Blob Storage, APIs, or flat files, transformed through a series of applied steps, and then loaded into the destination model. Each transformation step is recorded and can be modified, reordered, or deleted, providing full transparency and reproducibility in the data preparation process. This makes Power Query essential for building reliable analytics workloads on Azure.